Re: facet.offset issue (previosly: [solr 3.4] anomaly during distributed facet query with 102 shards)

2013-04-30 Thread Toke Eskildsen 
On Mon, 2013-04-29 at 18:22 +0200, Dmitry Kan wrote:
> Does it even make sense to paginate in facet searches, if we require deep
> paging?

Whether it makes sense logically is up to you.

Technically deep paging in facets could be implemented in approximately
the same way as deep paging in search results: By providing previously
received maximum count and value, the faceting module could skip ahead
to the subsequent entries and only return those.

It would be fairly simple to implement for single core, but tricky for
distributed search. I am not aware of any initiatives in that direction.

- Toke Eskildsen, State and University Library

Re: createNodeSet

2013-04-30 Thread Arkadi Colson 

I'm using the latest solr 4.2.1 with apache-tomcat-7.0.33

What exactly is the purpose of the linkconfig commmand? When I run it 
before I creating the collection, it creates a file in zookeeper. After 
creating the collection, the file is gone and a directory comes in place.


On 04/29/2013 05:36 PM, Mark Miller wrote:

What version of Solr? That should work in Jetty in 4.2 and not before and in 
Tomcat in 4.3 and not before.

- Mark

On Apr 29, 2013, at 10:19 AM, Arkadi Colson  wrote:


When I first do a linkconfig the route:implicit seems to be gone! So recreating 
the collection will solve this. The problem that I cannot request a collection 
that does not exists on that host is still there.

Arkadi

On 04/29/2013 03:31 PM, Arkadi Colson wrote:

The strange thing is that I created some time ago 2 other collections and there 
the router:implicit has not been set. Is it possible to create a collection 
withour the router:implicit?

http://solr01:8983/solr/admin/collections?action=CREATE&name=lvs&numShards=5&replicationFactor=2&collection.configName=smsc
 


VERSIONS

Solr 4.2.1

java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)

Met vriendelijke groeten

Arkadi Colson

Smartbit bvba • Hoogstraat 13 • 3670 Meeuwen
T +32 11 64 08 80 • F +32 11 64 08 81

On 04/29/2013 03:24 PM, Michael Della Bitta wrote:

That means that documents will be indexed and stored on the node
they're sent to. It shouldn't keep Solr Cloud from loadbalancing
reads. Fixing that won't address the problem you're asking about, but
it may clear up other unintended behaviors.

What version of Solr are you using, and what servlet container?

Michael Della Bitta


Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271

www.appinions.com

Where Influence Isn’t a Game


On Mon, Apr 29, 2013 at 9:20 AM, Arkadi Colson  wrote:

I found this in the zookeeper directory /collections/collectionX/

{
   "configName":"smsc",
   "router":"implicit"}


Is "router:implicit" the cause of this? Is it possible to fix?

Thx!


On 04/29/2013 01:24 PM, Arkadi Colson wrote:

Is it correct that if I create a collection B with parameter createNodeSet
= hostB and I query on hostA something for collectionA it could not be
found?

BR,
Arkadi

Re: Update on shards

2013-04-30 Thread Arkadi Colson 

Anyone an idea how to debug this?

Thx!

On 04/25/2013 09:18 AM, Arkadi Colson wrote:

Hi

It seems not to work in my case. We are using the solr php module for 
talking to Solr. Currently we have 2 collections 'intradesk' and 'lvs' 
for 10 solr hosts (shards: 5 - repl: 2). Because there is no more disc 
space I created 6 new hosts for collection 'messages' (shards: 3 - 
repl: 2).


'intradesk + lvs':
solr01-dcg
solr01-gs
solr02-dcg
solr02-gs
solr03-dcg
solr03-gs
solr04-dcg
solr04-gs
solr05-dcg
solr05-gs

'messages':
solr06-dcg
solr06-gs
solr07-dcg
solr07-gs
solr08-dcg
solr08-gs

So when doing a select, I can talk to any host. When updating I must 
talk to a host with at least 1 shard on it.


I created the new messages shard with the following command to get 
them on the new hosts (06 -> 08): 
http://solr01-dcg.intnet.smartbit.be:8983/solr/admin/collections?action=CREATE&name=messages&numShards=3&replicationFactor=2&collection.configName=smsc&createNodeSet=solr06-gs.intnet.smartbit.be:8983_solr,solr06-dcg.intnet.smartbit.be:8983_solr,solr07-gs.intnet.smartbit.be:8983_solr,solr07-dcg.intnet.smartbit.be:8983_solr,solr08-gs.intnet.smartbit.be:8983_solr,solr08-dcg.intnet.smartbit.be:8983_solr 



They are all in the same config set 'smsc'.

Below is the code:

$client = new SolrClient(
array(
'hostname'  => "solr01-dcg.intnet.smartbit.be 
",

'port'  => "8983",
'login' => "***",
'password'  => "***",
'path'  => "solr/messages",
'wt'=> "json"
)
);

$doc = new 
SolrInputDocument();

$doc->addField('id', $uniqueID);
$doc->addField('smsc_ssid', $ssID);
$doc->addField('smsc_module',   $i['module']);
$doc->addField('smsc_modulekey', $i['moduleKey']);
$doc->addField('smsc_courseid', $courseID);
$doc->addField('smsc_description', $i['description']);
$doc->addField('smsc_content',  $i['content']);
$doc->addField('smsc_lastdate', $lastdate);
$doc->addField('smsc_userid',   $userID);

$client->addDocument($doc);

The exception I get look like this:
exception 'SolrClientException' with message 'Unsuccessful update 
request. Response Code 200. (null)'


Nothing special to find in the solr log.

Any idea?


Arkadi

On 04/24/2013 08:43 PM, Mark Miller wrote:
Sorry - need to correct myself - updates worked the same as read 
requests - they also needed to hit a SolrCore in order to get 
forwarded to the right node. I was not thinking clearly when I said 
this applied to just reads and not writes. Both needed a SolrCore to 
do their work - with the request proxying, this is no longer the 
case, so you can hit Solr instances with no SolrCores or with 
SolrCores that are not part of the collection you are working with, 
and both read and write side requests are now proxied to a suitable 
node that has a SolrCore that can do the search or forward the update 
(or accept the update).


- Mark

On Apr 23, 2013, at 3:38 PM, Mark Miller  wrote:


We have a 3rd release candidate for 4.3 being voted on now.

I have never tested this feature with Tomcat - only Jetty. Users 
have reported it does not work with Tomcat. That leads one to think 
it may have a problem in other containers as well.


A previous contributor donated a patch that explicitly flushes a 
stream in our proxy code - he says this allows the feature to work 
with Tomcat. I committed this feature - the flush can't hurt, and 
given the previous contributions of this individual, I'm fairly 
confident the fix makes things work in Tomcat. I have no first hand 
knowledge that it does work though.


You might take the RC for a spin and test it our yourself: 
http://people.apache.org/~simonw/staging_area/lucene-solr-4.3.0-RC3-rev1470846/


- Mark

On Apr 23, 2013, at 3:20 PM, Furkan KAMACI  
wrote:



Hi Mark;

All in all you say that when 4.3 is tagged at repository (I mean 
when it is

ready) this feature will work for Tomcat too at a stable version?


2013/4/23 Mark Miller 


On Apr 23, 2013, at 2:49 PM, Shawn Heisey  wrote:


What exactly is the 'request proxying' thing that doesn't work on
tomcat?  Is this something different from basic SolrCloud 
operation where
you send any kind of request to any server and they get directed 
where they

need to go? I haven't heard of that not working on tomcat before.

Before 4.2, if you made a read request to a node t

Re: facet.offset issue (previosly: [solr 3.4] anomaly during distributed facet query with 102 shards)

2013-04-30 Thread Dmitry Kan 
It is logical to use facet pagination for us, if it would work. It just
doesn't, probably due the data amount we store and imposed RAM settings.




On Tue, Apr 30, 2013 at 10:39 AM, Toke Eskildsen 
wrote:

> On Mon, 2013-04-29 at 18:22 +0200, Dmitry Kan wrote:
> > Does it even make sense to paginate in facet searches, if we require deep
> > paging?
>
> Whether it makes sense logically is up to you.
>
> Technically deep paging in facets could be implemented in approximately
> the same way as deep paging in search results: By providing previously
> received maximum count and value, the faceting module could skip ahead
> to the subsequent entries and only return those.
>
> It would be fairly simple to implement for single core, but tricky for
> distributed search. I am not aware of any initiatives in that direction.
>
> - Toke Eskildsen, State and University Library
>
>

SolrCloud terminology - Why leader is listed as slave and replica is listed as master at admin gui?

2013-04-30 Thread Furkan KAMACI 
When I look at admin gui I see that for a leader:

Replication (Slave) Version Gen Size
Master: 1367309548534 84 779.87 MB
Slave:  1367307649512 82 784.44 MB

and that for a replica:

Replication (Master) Version Gen Size
Master: 1367309548534 84 779.87 MB

isn't it confusing leader is a slave and replica is a master or is it true
for SolrCloud terminology?

Maximum number of facet query ina single query

2013-04-30 Thread vicky desai 
Hi,

Is there any upper limit on the number of facet queries I can include in a
single query. Also is there any performance hit if I include too many facet
queries in a single query

Any help would be appreciated



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Maximum-number-of-facet-query-ina-single-query-tp4059926.html
Sent from the Solr - User mailing list archive at Nabble.com.

Custom sorting of Solr Results

2013-04-30 Thread Sandeep Mestry 
Dear Experts,

>
> I have a requirement for the exact matches and applying alphabetical
> sorting thereafter.
>
> To illustrate, the results should be sorted in exact matches and all later
> alphabetical.
>
> So, if there are 5 documents as below
>
> Doc1
> title: trees
>
> Doc 2
> title: plum trees
>
> Doc 3
> title: Money Trees (Legendary Trees)
>
> Doc 4
> title: Cork Trees
>
> Doc 5
> title: Old Trees
>
> Then, if user searches with query term as 'trees', the results should be
> in following order:
>
> Doc 1 trees - Highest Rank
> Doc 4 Cork Trees - Alphabetical afterwards..
> Doc 3 Money Trees (Legendary Trees)
> Doc 5 Old Trees
> Doc 2 plum trees
>
> I can achieve the alphabetical sorting by adding the title sort parameter, 
> However,
> Solr relevancy is higher for Doc 3 (due to matches in 2 terms and so it 
> arranges
> Doc 3 above Doc 4, 5 and 2).
> So, it looks like:
>
> Doc 1 trees - Highest Rank
> Doc 3 Money Trees (Legendary Trees)
> Doc 4 Cork Trees - Alphabetical afterwards..
> Doc 5 Old Trees
> Doc 2 plum trees
>
> Can you tell me an easy way to achieve this requirement please?
>
> I'm using Solr 4.0 and the *title *field is defined as follows:
>
>  positionIncrementGap="100" >
> 
> 
>  stemEnglishPossessive="0" generateWordParts="1" generateNumberParts="1"
> catenateWords="1" catenateNumbers="1" catenateAll="1" splitOnCaseChange="1"
> splitOnNumerics="0" preserveOriginal="1" />
> 
> 
> 
> 
>  stemEnglishPossessive="0" generateWordParts="1" generateNumberParts="1"
> catenateWords="1" catenateNumbers="1" catenateAll="1" splitOnCaseChange="1"
> splitOnNumerics="0" preserveOriginal="1" />
> 
> 
> 
>
>
>
> Many Thanks in advance,
> Sandeep
>

Re: Bloom filters and optimized vs. unoptimized indices

2013-04-30 Thread Michael McCandless 
Be sure to test the bloom postings format on your own use case ... in
my tests (heavy PK lookups) it was slower.

But to answer your question: I would expect a single segment index to
have much faster PK lookups than a multi-segment one, with and without
the bloom postings format, but bloom may make the many-segment case
faster (just be sure to test it yourself).


Mike McCandless

http://blog.mikemccandless.com


On Tue, Apr 30, 2013 at 1:05 AM, Otis Gospodnetic
 wrote:
> Hi,
>
> I was looking at
> http://lucene.apache.org/core/4_2_1/codecs/org/apache/lucene/codecs/bloom/BloomFilteringPostingsFormat.html
> and this piece of text:
> "
> A PostingsFormat useful for low doc-frequency fields such as primary
> keys. Bloom filters are maintained in a ".blm" file which offers
> "fast-fail" for reads in segments known to have no record of the key.
> "
>
> Is this implying that if you are doing PK lookups AND you have a large
> index (i.e. slow queries) it may actually be better to keep the index
> unoptimized, so whole index segments can be skipped?
>
> Thanks,
> Otis
> --
> SOLR Performance Monitoring - http://sematext.com/spm/index.html

Re: createNodeSet

2013-04-30 Thread Arkadi Colson 

I'm getting this error in tomcat log:

Apr 30, 2013 11:21:44 AM org.apache.solr.common.SolrException log
SEVERE: null:org.apache.solr.common.SolrException: Error trying to proxy 
request for url: 
http://solr03.officemeeuwen.smartbit.be:8983/solr/intradesk/select/
at 
org.apache.solr.servlet.SolrDispatchFilter.remoteQuery(SolrDispatchFilter.java:478)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:253)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at 
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:931)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at 
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at 
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at 
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:722)
Caused by: java.io.IOException: Server returned HTTP response code: 400 
for URL: 
http://solr03.officemeeuwen.smartbit.be:8983/solr/intradesk/select/?version=2.2&indent=on&wt=xml
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at 
sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1674)
at 
sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1672)

at java.security.AccessController.doPrivileged(Native Method)
at 
sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1670)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1243)
at 
org.apache.solr.servlet.SolrDispatchFilter.remoteQuery(SolrDispatchFilter.java:466)

... 17 more
Caused by: java.io.IOException: Server returned HTTP response code: 400 
for URL: 
http://solr03.officemeeuwen.smartbit.be:8983/solr/intradesk/select/?version=2.2&indent=on&wt=xml
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1625)
at 
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
at 
org.apache.solr.servlet.SolrDispatchFilter.remoteQuery(SolrDispatchFilter.java:454)

... 17 more

On 04/30/2013 09:57 AM, Arkadi Colson wrote:

I'm using the latest solr 4.2.1 with apache-tomcat-7.0.33

What exactly is the purpose of the linkconfig commmand? When I run it 
before I creating the collection, it creates a file in zookeeper. 
After creating the collection, the file is gone and a directory comes 
in place.


On 04/29/2013 05:36 PM, Mark Miller wrote:
What version of Solr? That should work in Jetty in 4.2 and not before 
and in Tomcat in 4.3 and not before.


- Mark

On Apr 29, 2013, at 10:19 AM, Arkadi Colson  wrote:

When I first do a linkconfig the route:implicit seems to be gone! So 
recreating the collection will solve this. The problem that I cannot 
request a collection that does not exists on that host is still there.


Arkadi

On 04/29/2013 03:31 PM, Arkadi Colson wrote:
The strange thing is that I created some time ago 2 other 
collections and there the router:implicit has not been set. Is it 
possible to create a collection withour the router:implicit?


http://solr01:8983/solr/admin/collections?action=CREATE&name=lvs&numShards=5&replicationFactor=2&collection.configName=smsc 
 



VERSIONS

Solr 4.2.1

java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)

Met vriendelijke groeten

Arkadi Colson

Smartbit bvba • Hoogstraat 13 • 3670 Meeuwen
T +32 11 64 08 80 • F +32 11 64 08 81

On 04/29/2013 03:24

Re: createNodeSet

2013-04-30 Thread Arkadi Colson 

This what I get on the solr host where the query is proxy-ed to:

Apr 30, 2013 11:36:28 AM org.apache.solr.core.SolrCore execute
INFO: [intradesk_shard1_replica1] webapp=/solr path=/admin/ping/ 
params={indent=on&wt=json&version=2.2} hits=0 status=0 QTime=1

Apr 30, 2013 11:36:28 AM org.apache.solr.core.SolrCore execute
INFO: [intradesk_shard1_replica1] webapp=/solr path=/admin/ping/ 
params={indent=on&wt=json&version=2.2} status=0 QTime=1

Apr 30, 2013 11:36:28 AM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: invalid boolean value: 1
at org.apache.solr.common.util.StrUtils.parseBool(StrUtils.java:239)
at org.apache.solr.common.params.SolrParams.getBool(SolrParams.java:95)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:172)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:1797)
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:637)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at 
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at 
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009)
at 
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at 
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Apr 30, 2013 11:36:28 AM org.apache.solr.core.SolrCore execute
INFO: [intradesk_shard1_replica1] webapp=/solr path=/select/ 
params={sort=score+desc&fl=id,smsc_module,smsc_modulekey,smsc_userid,smsc_ssid,smsc_description,smsc_description_ngram,smsc_content,smsc_content_ngram,smsc_courseid,smsc_lastdate,score,metadata_stream_size,metadata_stream_source_info,metadata_stream_name,metadata_stream_content_type,last_modified,author,title,subject&debugQuery=1&defaultOperator=AND&indent=on&start=0&q=(smsc_content:dag+||+smsc_content_ngram:dag+||+smsc_description:dag+||+smsc_description_ngram:dag)+%26%26+(smsc_lastdate:[2013-03-30T13:30:14Z+TO+2013-04-30T13:30:14Z])&collection=intradesk,lvs&wt=xml&rows=50&version=2.2} 
status=400 QTime=1


The last post is from the host who does the proxying...


On 04/30/2013 01:40 PM, Arkadi Colson wrote:

I'm getting this error in tomcat log:

Apr 30, 2013 11:21:44 AM org.apache.solr.common.SolrException log
SEVERE: null:org.apache.solr.common.SolrException: Error trying to 
proxy request for url: 
http://solr03.officemeeuwen.smartbit.be:8983/solr/intradesk/select/
at 
org.apache.solr.servlet.SolrDispatchFilter.remoteQuery(SolrDispatchFilter.java:478)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:253)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at 
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:931)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at 
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at 
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at 
o

How to Recovery Backup Snapshot at SolrCloud?

2013-04-30 Thread Furkan KAMACI 
Hi Folks;

I can backup my indexes at SolrCloud via
http://_master_host_:_port_/solr/replication?command=backup
and it creates a file called snapshot. I know that I should pull that
directory any other safe place (a backup store) However what should I do to
make a recovery from that backup file?

Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Furkan KAMACI 
I use SolrCloud, 4.2.1 of Solr.

Here is a detail from my admin page:

Replication (Slave) Version Gen Size
Master: 1367309548534 84 779.87 MB
Slave: 1367307649512 82 784.44 MB

When I use command=abortfetch still file size are not same. Any idea?

Re: createNodeSet

2013-04-30 Thread Arkadi Colson 
Never mind about these last 2 posts. debugQuery parameter must be false 
or true instead of 0 or 1


On 04/30/2013 01:54 PM, Arkadi Colson wrote:

This what I get on the solr host where the query is proxy-ed to:

Apr 30, 2013 11:36:28 AM org.apache.solr.core.SolrCore execute
INFO: [intradesk_shard1_replica1] webapp=/solr path=/admin/ping/ 
params={indent=on&wt=json&version=2.2} hits=0 status=0 QTime=1

Apr 30, 2013 11:36:28 AM org.apache.solr.core.SolrCore execute
INFO: [intradesk_shard1_replica1] webapp=/solr path=/admin/ping/ 
params={indent=on&wt=json&version=2.2} status=0 QTime=1

Apr 30, 2013 11:36:28 AM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: invalid boolean value: 1
at org.apache.solr.common.util.StrUtils.parseBool(StrUtils.java:239)
at 
org.apache.solr.common.params.SolrParams.getBool(SolrParams.java:95)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:172)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)

at org.apache.solr.core.SolrCore.execute(SolrCore.java:1797)
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:637)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at 
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at 
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009)
at 
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at 
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:662)

Apr 30, 2013 11:36:28 AM org.apache.solr.core.SolrCore execute
INFO: [intradesk_shard1_replica1] webapp=/solr path=/select/ 
params={sort=score+desc&fl=id,smsc_module,smsc_modulekey,smsc_userid,smsc_ssid,smsc_description,smsc_description_ngram,smsc_content,smsc_content_ngram,smsc_courseid,smsc_lastdate,score,metadata_stream_size,metadata_stream_source_info,metadata_stream_name,metadata_stream_content_type,last_modified,author,title,subject&debugQuery=1&defaultOperator=AND&indent=on&start=0&q=(smsc_content:dag+||+smsc_content_ngram:dag+||+smsc_description:dag+||+smsc_description_ngram:dag)+%26%26+(smsc_lastdate:[2013-03-30T13:30:14Z+TO+2013-04-30T13:30:14Z])&collection=intradesk,lvs&wt=xml&rows=50&version=2.2} 
status=400 QTime=1


The last post is from the host who does the proxying...


On 04/30/2013 01:40 PM, Arkadi Colson wrote:

I'm getting this error in tomcat log:

Apr 30, 2013 11:21:44 AM org.apache.solr.common.SolrException log
SEVERE: null:org.apache.solr.common.SolrException: Error trying to 
proxy request for url: 
http://solr03.officemeeuwen.smartbit.be:8983/solr/intradesk/select/
at 
org.apache.solr.servlet.SolrDispatchFilter.remoteQuery(SolrDispatchFilter.java:478)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:253)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at 
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:931)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at 
org.apache.coyote.http11.AbstractHttp11Processor.process(A

Re: createNodeSet - proxy request solrCloud

2013-04-30 Thread Arkadi Colson 
I've set up a new test situation with the same results. 3 solr nodes and 
3 collections with 2 shards and 0 replica's.


Collection Intradesk is on solr01 and solr02. When querying to solr03 I 
got no results:


Uncaught exception 'SolrClientException' with message 'Unsuccessful 
query request : Response Code 200. (null)'


*L**ogging solr03:

*Apr 30, 2013 12:06:49 PM sun.net.www.protocol.http.HttpURLConnection 
writeRequests
FINE: sun.net.www.MessageHeader@53d265528 pairs: {HEAD 
/solr/intradesk/admin/ping/?version=2.2&indent=on&wt=json HTTP/1.1: 
null}{user-agent: PHP Solr Client 1.0.3-alpha}{accept: 
*/*}{accept-charset: utf-8}{Cache-Control: no-cache}{Pragma: 
no-cache}{Host: solr01.officemeeuwen.smartbit.be:8983}{Connection: 
keep-alive}
Apr 30, 2013 12:06:49 PM sun.net.www.protocol.http.HttpURLConnection 
getInputStream
FINE: sun.net.www.MessageHeader@19fa157c5 pairs: {null: HTTP/1.1 200 
OK}{Server: Apache-Coyote/1.1}{Content-Type: 
text/plain;charset=UTF-8}{Content-Length: 0}{Date: Tue, 30 Apr 2013 
12:05:07 GMT}
Apr 30, 2013 12:06:49 PM sun.net.www.protocol.http.HttpURLConnection 
writeRequests
FINE: sun.net.www.MessageHeader@509f662e11 pairs: {POST 
/solr/intradesk/select/?version=2.2&indent=on&wt=xml HTTP/1.1: 
null}{authorization: Basic c29scjpzb2xy}{user-agent: PHP Solr Client 
1.0.3-alpha}{accept: */*}{accept-charset: utf-8}{content-type: 
application/x-www-form-urlencoded;charset=UTF-8}{Cache-Control: 
no-cache}{Pragma: no-cache}{Host: 
solr01.officemeeuwen.smartbit.be:8983}{Connection: 
keep-alive}{Content-Length: 656}
Apr 30, 2013 12:06:49 PM sun.net.www.protocol.http.HttpURLConnection 
getInputStream
FINE: sun.net.www.MessageHeader@54bec43f5 pairs: {null: HTTP/1.1 200 
OK}{Server: Apache-Coyote/1.1}{Content-Type: 
application/xml;charset=UTF-8}{Transfer-Encoding: chunked}{Date: Tue, 30 
Apr 2013 12:05:07 GMT}



*Logging solr01**:*

Apr 30, 2013 12:08:39 PM org.apache.solr.core.SolrCore execute
INFO: [intradesk_shard1_replica1] webapp=/solr path=/admin/ping/ 
params={indent=on&wt=json&version=2.2} hits=0 status=0 QTime=0

Apr 30, 2013 12:08:39 PM org.apache.solr.core.SolrCore execute
INFO: [intradesk_shard1_replica1] webapp=/solr path=/admin/ping/ 
params={indent=on&wt=json&version=2.2} status=0 QTime=1
Apr 30, 2013 12:08:39 PM 
org.apache.http.impl.conn.PoolingClientConnectionManager requestConnection
FINE: Connection request: [route: 
{}->http://solr01.officemeeuwen.smartbit.be:8983][total kept alive: 3; 
route allocated: 1 of 20; total allocated: 3 of 1]
Apr 30, 2013 12:08:39 PM 
org.apache.http.impl.conn.PoolingClientConnectionManager requestConnection
FINE: Connection request: [route: 
{}->http://solr02.officemeeuwen.smartbit.be:8983][total kept alive: 3; 
route allocated: 1 of 20; total allocated: 3 of 1]
Apr 30, 2013 12:08:39 PM 
org.apache.http.impl.conn.PoolingClientConnectionManager leaseConnection
FINE: Connection leased: [id: 4][route: 
{}->http://solr01.officemeeuwen.smartbit.be:8983][total kept alive: 2; 
route allocated: 1 of 20; total allocated: 3 of 1]
Apr 30, 2013 12:08:39 PM 
org.apache.http.impl.conn.PoolingClientConnectionManager leaseConnection
FINE: Connection leased: [id: 5][route: 
{}->http://solr02.officemeeuwen.smartbit.be:8983][total kept alive: 1; 
route allocated: 1 of 20; total allocated: 3 of 1]
Apr 30, 2013 12:08:39 PM 
org.apache.http.impl.client.DefaultRequestDirector execute

FINE: Stale connection check
Apr 30, 2013 12:08:39 PM 
org.apache.http.impl.client.DefaultRequestDirector execute

FINE: Stale connection check
Apr 30, 2013 12:08:39 PM 
org.apache.http.impl.client.DefaultRequestDirector execute

FINE: Stale connection detected
Apr 30, 2013 12:08:39 PM 
org.apache.http.impl.client.DefaultRequestDirector execute

FINE: Stale connection detected
Apr 30, 2013 12:08:39 PM 
org.apache.http.impl.conn.DefaultClientConnection close

FINE: Connection 0.0.0.0:52385<->10.1.0.201:8983 closed
Apr 30, 2013 12:08:39 PM 
org.apache.http.impl.conn.DefaultClientConnection close

FINE: Connection 0.0.0.0:38275<->10.1.0.202:8983 closed
Apr 30, 2013 12:08:39 PM 
org.apache.http.impl.conn.DefaultClientConnectionOperator openConnection

FINE: Connecting to solr01.officemeeuwen.smartbit.be:8983
Apr 30, 2013 12:08:39 PM 
org.apache.http.client.protocol.RequestAddCookies process

FINE: CookieSpec selected: best-match
Apr 30, 2013 12:08:39 PM 
org.apache.http.client.protocol.RequestAuthCache process

FINE: Auth cache not set in the context
Apr 30, 2013 12:08:39 PM 
org.apache.http.impl.conn.DefaultClientConnectionOperator openConnection

FINE: Connecting to solr02.officemeeuwen.smartbit.be:8983
Apr 30, 2013 12:08:39 PM 
org.apache.http.client.protocol.RequestTargetAuthentication process

FINE: Target auth state: UNCHALLENGED
Apr 30, 2013 12:08:39 PM 
org.apache.http.client.protocol.RequestProxyAuthentication process

FINE: Proxy auth state: UNCHALLENGED
Apr 30, 2013 12:08:39 PM 
org.apache.http.client.protocol.RequestAddCookies process

FINE: CookieSpec sele

Re: Maximum number of facet query ina single query

2013-04-30 Thread Erik Hatcher 
There's no fixed limit of facet.query's.  But certainly there is a performance 
impact, which is often mitigated by warming and caching.  You'll need to test 
the impact in your environment.

Erik

On Apr 30, 2013, at 02:22 , vicky desai wrote:

> Hi,
> 
> Is there any upper limit on the number of facet queries I can include in a
> single query. Also is there any performance hit if I include too many facet
> queries in a single query
> 
> Any help would be appreciated
> 
> 
> 
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Maximum-number-of-facet-query-ina-single-query-tp4059926.html
> Sent from the Solr - User mailing list archive at Nabble.com.

More than one sort criteria

2013-04-30 Thread Peter Sch�tt 
Hallo,

I have more than one sortable fields. Example:

I have address data and I want to sort by zip code and street name.
Now I have both fields as sortable declared in the schema and I 
can sort by them.

q=*&sort=zip+asc

or 

q=*&sort=street+asc

But how can I sort by both (first zip code then street).

q=*&sort=zip+asc&sort=street+asc

I got no error message, but the result is not sorted by the second 
criteria.

The params are:


  
   zip asc
   street desc
  
  true
  *
  xml


What is my mistake?

Thanks for any hint.

Ciao
  Peter Schütt

Re: solr query- get results without scanning files

2013-04-30 Thread Erick Erickson 
Please review:
http://wiki.apache.org/solr/UsingMailingLists

You haven't given us near enough information to answer your question.
My guess is you're trying to return very large data sets, something
Solr isn't designed to do, but that's only a guess.

Possibly you have not set lazy field loading to on.

Best
Erick

On Mon, Apr 29, 2013 at 8:26 AM, dafnashkedy  wrote:
> I would like to execute a solr query and get only the uniquKey I've defined.
> The documents are very big so defining fl='my_key' is not fast enough - all
> the matching documents are still scanned and the query can take hours (even
> though the search itself was fast - numFound takes few seconds to return).
> I should mention that all the data is stored, and creating a new index is
> not an option.
>
> One idea I had was to get the docIds of the results and map them to my_key
> in the code.
> I used fl=[docid], thinking it doesn't need scanning to get this info, but
> it still takes too long to return.
>
> Is there a better way to get the docIds?
> Or a way to unstore certain fields without reindexing?
> Or perhapse a compeletly different way to get the results without scanning
> all the fields?
>
> Thanks,
>
> Dafna
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/solr-query-get-results-without-scanning-files-tp4059798.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: why does "*" affect case sensitivity of query results

2013-04-30 Thread Erick Erickson 
Actually, look at the referenced JIRA
https://issues.apache.org/jira/browse/SOLR-2438 and you'll see it's
changed in 3.6.

Best
Erick

On Mon, Apr 29, 2013 at 9:36 AM, geeky2  wrote:
> here is the jira link:
>
> https://issues.apache.org/jira/browse/SOLR-219
>
>
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/why-does-affect-case-sensitivity-of-query-results-tp4059801p4059814.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: Exact and Partial Matches

2013-04-30 Thread Erick Erickson 
I don't think you can do that. You're essentially
trying to mix ordering of the result set. You
_might_ be able to kludge some of this with
grouping, but I doubt it.

You'll need two queries I'd guess.

Best
Erick

On Mon, Apr 29, 2013 at 9:44 AM, Sandeep Mestry  wrote:
> Dear Experts,
>
> I have a requirement for the exact matches and applying alphabetical
> sorting thereafter.
>
> To illustrate, the results should be sorted in exact matches and all later
> alphabetical.
>
> So, if there are 5 documents as below
>
> Doc1
> title: trees
>
> Doc 2
> title: plum trees
>
> Doc 3
> title: Money Trees (Legendary Trees)
>
> Doc 4
> title: Cork Trees
>
> Doc 5
> title: Old Trees
>
> Then, if user searches with query term as 'trees', the results should be in
> following order:
>
> Doc 1 trees - Highest Rank
> Doc 4 Cork Trees - Alphabetical afterwards..
> Doc 3 Money Trees (Legendary Trees)
> Doc 5 Old Trees
> Doc 2 plum trees
>
> I can achieve the alphabetical sorting by adding the title sort
> parameter, However,
> Solr relevancy is higher for Doc 3 (due to matches in 2 terms and so
> it arranges
> Doc 3 above Doc 4, 5 and 2).
> So, it looks like:
>
> Doc 1 trees - Highest Rank
> Doc 3 Money Trees (Legendary Trees)
> Doc 4 Cork Trees - Alphabetical afterwards..
> Doc 5 Old Trees
> Doc 2 plum trees
>
> Can you tell me an easy way to achieve this requirement please?
>
> I'm using Solr 4.0 and the *title *field is defined as follows:
>
> >
> 
> 
>  stemEnglishPossessive="0" generateWordParts="1" generateNumberParts="1"
> catenateWords="1" catenateNumbers="1" catenateAll="1" splitOnCaseChange="1"
> splitOnNumerics="0" preserveOriginal="1" />
> 
> 
> 
> 
>  stemEnglishPossessive="0" generateWordParts="1" generateNumberParts="1"
> catenateWords="1" catenateNumbers="1" catenateAll="1" splitOnCaseChange="1"
> splitOnNumerics="0" preserveOriginal="1" />
> 
> 
> 
>
>
>
> Many Thanks in advance,
> Sandeep

Re: 4.2.1 Tutorial

2013-04-30 Thread Jon Strayer 
I haven't found that, but I did find this:

Apr 30, 2013 9:38:10 AM org.apache.solr.core.CoreContainer create
INFO: Creating SolrCore 'collection1' using instanceDir: solr/collection1
Apr 30, 2013 9:38:10 AM org.apache.solr.core.SolrResourceLoader 
INFO: new SolrResourceLoader for directory: 'solr/collection1/'
Apr 30, 2013 9:38:10 AM org.apache.solr.core.SolrConfig initLibs

And then this:
Apr 30, 2013 9:38:11 AM org.apache.solr.core.SolrCore 
INFO: [collection1] Opening new SolrCore at solr/collection1/,
dataDir=/Users/jon.strayer/solr/solr-4.2.1/example/solr/collection1/data/

And a little later:
Apr 30, 2013 9:38:11 AM org.apache.solr.core.CachingDirectoryFactory get
INFO: return new directory for
/Users/jon.strayer/solr/solr-4.2.1/example/solr/collection1/data forceNew:
false
Apr 30, 2013 9:38:11 AM org.apache.solr.core.SolrCore getNewIndexDir
INFO: New index directory detected: old=null
new=/Users/jon.strayer/solr/solr-4.2.1/example/solr/collection1/data/index/
Apr 30, 2013 9:38:11 AM org.apache.solr.core.CachingDirectoryFactory get
INFO: return new directory for
/Users/jon.strayer/solr/solr-4.2.1/example/solr/collection1/data/index
forceNew: false

It looks to me as if the core does exist (and in fact I can add documents
to it).  But for some reason parts of the system don't know about it.


On Mon, Apr 29, 2013 at 5:20 PM, Furkan KAMACI wrote:

> Check your logs when you startup Solr if you get that error: "There exists
> no core with the name "collection1"". Do you get any error as like
> core:collection1 could not create or something like that?
>
> 2013/4/29 Jon Strayer 
>
> > I can't be the only person to run into this, but I can't find any mention
> > of it anywhere.
> >
> > I have Solr 4.2.1 installed under OSX 10.8.3.  I'm working my way through
> > the tutorial.
> >
> > When I click on this link:
> http://localhost:8983/solr/#/collection1/queryI get the error message
> "There exists no core with the name "collection1"".
> >
> > This link works:
> > http://localhost:8983/solr/collection1/select?q=solr&wt=xml
> >
> > What am I doing wrong?
> >
> > [image: Inline image 1]
> >
> > --
> > To *know* is one thing, and to know for certain *that* we know is
> another.
> > --William James
> >
>



-- 
To *know* is one thing, and to know for certain *that* we know is another.
--William James

Re: why does "*" affect case sensitivity of query results

2013-04-30 Thread geeky2 
hello erik,

thank you for the info - yes - i did notice ;)

one more reason for us to upgrade from 3.5.

thx
mark




--
View this message in context: 
http://lucene.472066.n3.nabble.com/why-does-affect-case-sensitivity-of-query-results-tp4059801p406.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Shawn Heisey 
On 4/30/2013 6:13 AM, Furkan KAMACI wrote:
> I use SolrCloud, 4.2.1 of Solr.
> 
> Here is a detail from my admin page:
> 
> Replication (Slave) Version Gen Size
> Master: 1367309548534 84 779.87 MB
> Slave: 1367307649512 82 784.44 MB
> 
> When I use command=abortfetch still file size are not same. Any idea?

I won't know what abortfetch means unless I look it up, but I think this
is normal.

I have a 4.2.1 SolrCloud where things are working perfectly.  My two
replicas are not identical in size, but I have not seen any evidence of
a difference in contents:

Version Gen Size
Master: 1367280043801   15331   1,013.07 MB
Slave:  1367280083570   15331   1 GB

Version Gen Size
Master: 1367280083570   15331   1 GB
Slave:  1367280043801   15331   1,013.07 MB

For anyone who is wondering why these numbers are displayed differently,
these are powers of 2, not powers of 10.  Solr won't switch to 1GB until
you hit 1024MB, and each MB is 1048576 bytes, not 100.

Thanks,
Shawn

Re: SolrCloud terminology - Why leader is listed as slave and replica is listed as master at admin gui?

2013-04-30 Thread Shawn Heisey 
On 4/30/2013 2:51 AM, Furkan KAMACI wrote:
> When I look at admin gui I see that for a leader:
> 
> Replication (Slave) Version Gen Size
> Master: 1367309548534 84 779.87 MB
> Slave:  1367307649512 82 784.44 MB
> 
> and that for a replica:
> 
> Replication (Master) Version Gen Size
> Master: 1367309548534 84 779.87 MB
> 
> isn't it confusing leader is a slave and replica is a master or is it true
> for SolrCloud terminology?

Master info gets listed as soon as the replication handler has started.
 If that core has ever been used as a replication target since the last
restart, then it will show the slave information as well.

On the last restart, your current leader probably replicated itself from
the other replica, which would have been leader at the time.  Your other
replica has never (since the last restart) been used as a replication
target, it has only ever kept itself in sync by doing SolrCloud
distributed indexing.

There is a bug in the replication info display, fixed in 4.3.  The bug
info says it affects version 4.2, but I would not be surprised to learn
that it affects all previous 4.x releases as well.  This doesn't cause
any actual problem with replication, just the confusing information on
the slave:

https://issues.apache.org/jira/browse/SOLR-4661

The 4.3 release is on release candidate 4, hoping for a release just
after Lucene Revolution 2013 ends.

Thanks,
Shawn

Re: SolrCloud terminology - Why leader is listed as slave and replica is listed as master at admin gui?

2013-04-30 Thread Shawn Heisey 
On 4/30/2013 8:16 AM, Shawn Heisey wrote:
> Master info gets listed as soon as the replication handler has started.
>  If that core has ever been used as a replication target since the last
> restart, then it will show the slave information as well.

After thinking about it. I'm not 100% sure that what I said was correct.
 I don't have time right now to do any experimentation.

Thanks,
Shawn

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Michael Della Bitta 
In Solr Cloud, commits can happen at different times across replicas. Which
means merges also may happen at different times. So there's no expectation
of the cores of different replicas being totally similar.


Michael Della Bitta


Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271

www.appinions.com

Where Influence Isn’t a Game


On Tue, Apr 30, 2013 at 8:13 AM, Furkan KAMACI wrote:

> I use SolrCloud, 4.2.1 of Solr.
>
> Here is a detail from my admin page:
>
> Replication (Slave) Version Gen Size
> Master: 1367309548534 84 779.87 MB
> Slave: 1367307649512 82 784.44 MB
>
> When I use command=abortfetch still file size are not same. Any idea?
>

Re: Not In query

2013-04-30 Thread André Maldonado 
Thank's Jan for your reply.

My application has thousands of users and I don't know yet how many of them
will use this feature. They can exclude one document from their search
results or can exclude 200.000 documents. It's much more natural that they
exclude something like 50~300 documents. More than this will be strange.

However, I don't know how cache will work because we have a large number of
users who can use this feature. Even that query for user 1 be cached, it
won't work for other users.

Do you see another solution for this case?

Thank's



*
--
*
*"E conhecereis a verdade, e a verdade vos libertará." (João 8:32)*

 *andre.maldonado*@gmail.com 
 (11) 9112-4227




   
  

  



On Fri, Apr 26, 2013 at 6:18 PM, Jan Høydahl  wrote:

> I would start with the way you propose, a negative filter
>
> q=foo bar&fq=-id:(123 729 640 112...)
>
> This will effectively hide those doc ids, and a benefit is that it is
> cached so if the list of ids is long, you'll only take the performance hit
> the first time. I don't know your application, but if it is highly likely
> that a single user will add excludes for several thousand ids then you
> should perhaps consider other options and benchmark up front.
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
> Solr Training - www.solrtraining.com
>
> 26. apr. 2013 kl. 21:50 skrev André Maldonado :
>
> > Hi all.
> >
> > We have an index with 300.000 documents and a lot, a lot of fields.
> >
> > We're planning a module where users will choose some documents to exclude
> > from their search results. So, these documents will be excluded for UserA
> > and visible for UserB.
> >
> > So, we have some options to do this. The simplest way is to do a "Not In"
> > query in document id. But we don't know the performance impact this will
> > have. Is this an option?
> >
> > There is another reasonable way to accomplish this?
> >
> > Thank's
> >
> > *
> >
> --
> > *
> > *"E conhecereis a verdade, e a verdade vos libertará." (João 8:32)*
> >
> > *andre.maldonado*@gmail.com 
> > (11) 9112-4227
> >
> > 
> > 
> > 
> >   <
> http://www.delicious.com/andre.maldonado>
> >  
> > 
> >  
>
>

Re: How to Recovery Backup Snapshot at SolrCloud?

2013-04-30 Thread Michael Della Bitta 
That directory is the data directory for the core... you'd just swap it in.


Michael Della Bitta


Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271

www.appinions.com

Where Influence Isn’t a Game


On Tue, Apr 30, 2013 at 8:06 AM, Furkan KAMACI wrote:

> Hi Folks;
>
> I can backup my indexes at SolrCloud via
> http://_master_host_:_port_/solr/replication?command=backup
> and it creates a file called snapshot. I know that I should pull that
> directory any other safe place (a backup store) However what should I do to
> make a recovery from that backup file?
>

Re: Exact and Partial Matches

2013-04-30 Thread Sandeep Mestry 
Thanks Erick,

I tried grouping and it appears to work okay. However, I will need to
change the client to parse the output..

&fq=title:(tree)&group=true&group.query=title:(trees) NOT
title_ci:trees&group.query=title_ci:blair&group.sort=title_sort
desc&sort=score desc,title_sort asc

I used the actual query as the filter query so my scores will be 1 and then
used 2 group queries - one which will give me exact matches and other that
will give me partial minus exact matches.
I have tried this with operators too and it seems to be doing the job I
want, do you see any issue in this?

Thanks again for your reply and by the way thanks for SOLR-4662.

-S


On 30 April 2013 15:06, Erick Erickson  wrote:

> I don't think you can do that. You're essentially
> trying to mix ordering of the result set. You
> _might_ be able to kludge some of this with
> grouping, but I doubt it.
>
> You'll need two queries I'd guess.
>
> Best
> Erick
>
> On Mon, Apr 29, 2013 at 9:44 AM, Sandeep Mestry 
> wrote:
> > Dear Experts,
> >
> > I have a requirement for the exact matches and applying alphabetical
> > sorting thereafter.
> >
> > To illustrate, the results should be sorted in exact matches and all
> later
> > alphabetical.
> >
> > So, if there are 5 documents as below
> >
> > Doc1
> > title: trees
> >
> > Doc 2
> > title: plum trees
> >
> > Doc 3
> > title: Money Trees (Legendary Trees)
> >
> > Doc 4
> > title: Cork Trees
> >
> > Doc 5
> > title: Old Trees
> >
> > Then, if user searches with query term as 'trees', the results should be
> in
> > following order:
> >
> > Doc 1 trees - Highest Rank
> > Doc 4 Cork Trees - Alphabetical afterwards..
> > Doc 3 Money Trees (Legendary Trees)
> > Doc 5 Old Trees
> > Doc 2 plum trees
> >
> > I can achieve the alphabetical sorting by adding the title sort
> > parameter, However,
> > Solr relevancy is higher for Doc 3 (due to matches in 2 terms and so
> > it arranges
> > Doc 3 above Doc 4, 5 and 2).
> > So, it looks like:
> >
> > Doc 1 trees - Highest Rank
> > Doc 3 Money Trees (Legendary Trees)
> > Doc 4 Cork Trees - Alphabetical afterwards..
> > Doc 5 Old Trees
> > Doc 2 plum trees
> >
> > Can you tell me an easy way to achieve this requirement please?
> >
> > I'm using Solr 4.0 and the *title *field is defined as follows:
> >
> >  positionIncrementGap="100"
> >>
> > 
> > 
> >  > stemEnglishPossessive="0" generateWordParts="1" generateNumberParts="1"
> > catenateWords="1" catenateNumbers="1" catenateAll="1"
> splitOnCaseChange="1"
> > splitOnNumerics="0" preserveOriginal="1" />
> > 
> > 
> > 
> > 
> >  > stemEnglishPossessive="0" generateWordParts="1" generateNumberParts="1"
> > catenateWords="1" catenateNumbers="1" catenateAll="1"
> splitOnCaseChange="1"
> > splitOnNumerics="0" preserveOriginal="1" />
> > 
> > 
> > 
> >
> >
> >
> > Many Thanks in advance,
> > Sandeep
>

Re: Solr - WordDelimiterFactory with Custom Tokenizer to split only on Boundires

2013-04-30 Thread meghana 
Thanks Jack Krupansky, Its very helpful :)

Jack Krupansky-2 wrote
> The WDF "types" will treat a character the same regardless of where it 
> appears.
> 
> For something conditional, like dot between letters vs. dot lot preceded
> and 
> followed by a letter, you either have to have a custom tokenizer or a 
> character filter.
> 
> Interesting that although the standard tokenizer messes up embedded
> hyphens, 
> it does handle the embedded dot vs. trailing dot case as you wish (but 
> messes up "U.S.A." by stripping the trailing dot) - but that doesn't help 
> your case.
> 
> A character filter like the following might help your case:
>  positionIncrementGap="100">
>   
> 
> 
>  pattern="([\w\d])[\._&]+($|[^\w\d])" replacement="$1 $2" />
> 
>  pattern="(^|[^\w\d])[\._&]+($|[^\w\d])" replacement="$1 $2" />
> 
>  pattern="(^|[^\w\d])[\._&]+([\w\d])" replacement="$1 $2" />
> 
> 
>   
> 
> 
> I'm not a regular expression expert, so I'm not sure whether/how those 
> patterns could be combined.
> 
> Also, that doesn't allow the case of a single ".", "&", or "_" as a word - 
> but you didn't specify how that case should be handled.
> 
> 
> 
> -- Jack Krupansky
> -Original Message- 
> From: meghana
> Sent: Wednesday, April 24, 2013 6:49 AM
> To: 

> solr-user@.apache

> Subject: Solr - WordDelimiterFactory with Custom Tokenizer to split only
> on 
> Boundires
> 
> I have configured WordDelimiterFilterFactory for custom tokenizers for '&'
> and '-' , and for few tokenizer (like . _ :) we need to split on boundries
> only.
> 
> e.g.
> test.com (should tokenized to test.com)
> newyear.  (should tokenized to newyear)
> new_car (should tokenized to new_car)
> ..
> ..
> 
> Below is defination for text field
>  positionIncrementGap="100">
>   
> 
>  
> 
>  
>  words="stopwords.txt" enablePositionIncrements="false" />
>  
>  splitOnCaseChange ="0"
> splitOnNumerics ="0"
> stemEnglishPossessive ="0"
> generateWordParts="1"
> generateNumberParts="1"
> catenateWords="0"
> catenateNumbers="0"
> catenateAll="0"
> preserveOriginal="0"
> protected="protwords_general.txt"
> types="wdfftypes_general.txt"
> />
> 
> 
>   
> 
>   
> 
> 
> 
> 
>  words="stopwords.txt" enablePositionIncrements="false" />
> 
>  splitOnCaseChange ="0"
> splitOnNumerics ="0"
> stemEnglishPossessive ="0"
> generateWordParts="1"
> generateNumberParts="1"
> catenateWords="0"
> catenateNumbers="0"
> catenateAll="0"
> preserveOriginal="0"
> protected="protwords_general.txt"
> types="wdfftypes_general.txt"
> />
> 
>  ignoreCase="true" expand="true"/>
> 
> 
>   
> 
> 
> 
> below is wdfftypes_general.txt content
> 
> & => ALPHA
> - => ALPHA
> _ => SUBWORD_DELIM
> : => SUBWORD_DELIM
> . => SUBWORD_DELIM
> 
> types can be used in worddelimiter  are LOWER, UPPER, ALPHA, DIGIT,
> ALPHANUM, SUBWORD_DELIM . there's no description available for use of each
> type. as per name, i thought type SUBWORD_DELIM may fulfill my need, but
> it
> doesn't seem to work.
> 
> Can anybody suggest me how can i set configuration for worddelimiter
> factory
> to fulfill my requirement.
> 
> Thanks.
> 
> 
> 
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Solr-WordDelimiterFactory-with-Custom-Tokenizer-to-split-only-on-Boundires-tp4058557.html
> Sent from the Solr - User mailing list archive at Nabble.com.





--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-WordDelimiterFactory-with-Custom-Tokenizer-to-split-only-on-Boundires-tp4058557p4060011.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Furkan KAMACI 
I think that replication occurs after commit by default. It has been long
time however there is still mismatch between leader and replica
(approximately 5 MB). I tried to pull indexes from leader but it is still
same.

2013/4/30 Michael Della Bitta 

> In Solr Cloud, commits can happen at different times across replicas. Which
> means merges also may happen at different times. So there's no expectation
> of the cores of different replicas being totally similar.
>
>
> Michael Della Bitta
>
> 
> Appinions
> 18 East 41st Street, 2nd Floor
> New York, NY 10017-6271
>
> www.appinions.com
>
> Where Influence Isn’t a Game
>
>
> On Tue, Apr 30, 2013 at 8:13 AM, Furkan KAMACI  >wrote:
>
> > I use SolrCloud, 4.2.1 of Solr.
> >
> > Here is a detail from my admin page:
> >
> > Replication (Slave) Version Gen Size
> > Master: 1367309548534 84 779.87 MB
> > Slave: 1367307649512 82 784.44 MB
> >
> > When I use command=abortfetch still file size are not same. Any idea?
> >
>

Re: More than one sort criteria

2013-04-30 Thread Gustav 
Peter, try sorting them only using one &sort parameter, separating the fields
by comma.

&sort=zip+asc,street+asc 



--
View this message in context: 
http://lucene.472066.n3.nabble.com/More-than-one-sort-criteria-tp4059989p4060015.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Michael Della Bitta 
I'm a little confused. Are you using Solr Cloud, or ordinary replication?


Michael Della Bitta


Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271

www.appinions.com

Where Influence Isn’t a Game


On Tue, Apr 30, 2013 at 10:33 AM, Furkan KAMACI wrote:

> I think that replication occurs after commit by default. It has been long
> time however there is still mismatch between leader and replica
> (approximately 5 MB). I tried to pull indexes from leader but it is still
> same.
>
> 2013/4/30 Michael Della Bitta 
>
> > In Solr Cloud, commits can happen at different times across replicas.
> Which
> > means merges also may happen at different times. So there's no
> expectation
> > of the cores of different replicas being totally similar.
> >
> >
> > Michael Della Bitta
> >
> > 
> > Appinions
> > 18 East 41st Street, 2nd Floor
> > New York, NY 10017-6271
> >
> > www.appinions.com
> >
> > Where Influence Isn’t a Game
> >
> >
> > On Tue, Apr 30, 2013 at 8:13 AM, Furkan KAMACI  > >wrote:
> >
> > > I use SolrCloud, 4.2.1 of Solr.
> > >
> > > Here is a detail from my admin page:
> > >
> > > Replication (Slave) Version Gen Size
> > > Master: 1367309548534 84 779.87 MB
> > > Slave: 1367307649512 82 784.44 MB
> > >
> > > When I use command=abortfetch still file size are not same. Any idea?
> > >
> >
>

Issue with fuzzy search in Distributed Search

2013-04-30 Thread meghana 
I have created 2 versions of Solr core in different servers. one is simple
core having all records in one core. And other is shards core, distributed
over 3 cores on server.

Simple core :

http://localhost:8080/sorl/core0/select?q=text:hoers~1

Distributed core :

http://192.168.1.91:8080/core0/select?shards=http://192.168.1.91:8080/core0,http://192.168.1.91:8080/core1,http://192.168.1.91:8080/core2&q=text:hoers~1

data, schema and other configuration is similar in both the cores.

but while doing fuzzy search like hoers~1 one core returns many
records(about 450), while other core return only 1 record.

While this issue does not seem related to Distributed Search, as Although i
do not use distributed search, then also it do not return more rows.

as http://192.168.1.91:8080/core0/select?q=text:hoers~1

below is schema definition for my field.


  
  






  
  








  


Not sure, what is wrong with this. Can anybody help me on this??




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Issue-with-fuzzy-search-in-Distributed-Search-tp4060022.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Issue with fuzzy search in Distributed Search

2013-04-30 Thread Jack Krupansky 
A fuzzy query itself does not know about distributed search - Lucene simply 
scores the query results based on the local index. Then, Solr is merging the 
merging the query results from different nodes.


Try the query locally for each node and set debugQuery=true and see how each 
document gets scored.


I'm actually not sure what the specific "problem" (symptom) is that you are 
seeing. I mean, maybe there is only 1 result on that node - how do you know 
otherwise?? Or maybe one node has more exact matches.


-- Jack Krupansky

-Original Message- 
From: meghana

Sent: Tuesday, April 30, 2013 7:51 AM
To: solr-user@lucene.apache.org
Subject: Issue with fuzzy search in Distributed Search

I have created 2 versions of Solr core in different servers. one is simple
core having all records in one core. And other is shards core, distributed
over 3 cores on server.

Simple core :

http://localhost:8080/sorl/core0/select?q=text:hoers~1

Distributed core :

http://192.168.1.91:8080/core0/select?shards=http://192.168.1.91:8080/core0,http://192.168.1.91:8080/core1,http://192.168.1.91:8080/core2&q=text:hoers~1

data, schema and other configuration is similar in both the cores.

but while doing fuzzy search like hoers~1 one core returns many
records(about 450), while other core return only 1 record.

While this issue does not seem related to Distributed Search, as Although i
do not use distributed search, then also it do not return more rows.

as http://192.168.1.91:8080/core0/select?q=text:hoers~1

below is schema definition for my field.


 
 
   
   
   
   
   
   
 
 
   
   
   
   
   
   
   
   
 
   

Not sure, what is wrong with this. Can anybody help me on this??




--
View this message in context: 
http://lucene.472066.n3.nabble.com/Issue-with-fuzzy-search-in-Distributed-Search-tp4060022.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: How to Recovery Backup Snapshot at SolrCloud?

2013-04-30 Thread Furkan KAMACI 
Should I stop the node first? And what will happen to transaction logs?
Should I backup it too?

2013/4/30 Michael Della Bitta 

> That directory is the data directory for the core... you'd just swap it in.
>
>
> Michael Della Bitta
>
> 
> Appinions
> 18 East 41st Street, 2nd Floor
> New York, NY 10017-6271
>
> www.appinions.com
>
> Where Influence Isn’t a Game
>
>
> On Tue, Apr 30, 2013 at 8:06 AM, Furkan KAMACI  >wrote:
>
> > Hi Folks;
> >
> > I can backup my indexes at SolrCloud via
> > http://_master_host_:_port_/solr/replication?command=backup
> > and it creates a file called snapshot. I know that I should pull that
> > directory any other safe place (a backup store) However what should I do
> to
> > make a recovery from that backup file?
> >
>

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Furkan KAMACI 
I use Solr 4.2.1 as SolrCloud

2013/4/30 Michael Della Bitta 

> I'm a little confused. Are you using Solr Cloud, or ordinary replication?
>
>
> Michael Della Bitta
>
> 
> Appinions
> 18 East 41st Street, 2nd Floor
> New York, NY 10017-6271
>
> www.appinions.com
>
> Where Influence Isn’t a Game
>
>
> On Tue, Apr 30, 2013 at 10:33 AM, Furkan KAMACI  >wrote:
>
> > I think that replication occurs after commit by default. It has been long
> > time however there is still mismatch between leader and replica
> > (approximately 5 MB). I tried to pull indexes from leader but it is still
> > same.
> >
> > 2013/4/30 Michael Della Bitta 
> >
> > > In Solr Cloud, commits can happen at different times across replicas.
> > Which
> > > means merges also may happen at different times. So there's no
> > expectation
> > > of the cores of different replicas being totally similar.
> > >
> > >
> > > Michael Della Bitta
> > >
> > > 
> > > Appinions
> > > 18 East 41st Street, 2nd Floor
> > > New York, NY 10017-6271
> > >
> > > www.appinions.com
> > >
> > > Where Influence Isn’t a Game
> > >
> > >
> > > On Tue, Apr 30, 2013 at 8:13 AM, Furkan KAMACI  > > >wrote:
> > >
> > > > I use SolrCloud, 4.2.1 of Solr.
> > > >
> > > > Here is a detail from my admin page:
> > > >
> > > > Replication (Slave) Version Gen Size
> > > > Master: 1367309548534 84 779.87 MB
> > > > Slave: 1367307649512 82 784.44 MB
> > > >
> > > > When I use command=abortfetch still file size are not same. Any idea?
> > > >
> > >
> >
>

What Happens When I Unload A Core?

2013-04-30 Thread Furkan KAMACI 
I use Solr 4.2.1

What happens if I unload a core, I mean what does Solr do? Because solr.xml
didn't change and I think that Solr should writes something to somewhere or
deletes something from somewhere?

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Furkan KAMACI 
Shawn, why they don't have same data byte per byte? Can I force slave to
pull them, I tried but didn't work.

2013/4/30 Furkan KAMACI 

> I use Solr 4.2.1 as SolrCloud
>
>
> 2013/4/30 Michael Della Bitta 
>
>> I'm a little confused. Are you using Solr Cloud, or ordinary replication?
>>
>>
>> Michael Della Bitta
>>
>> 
>> Appinions
>> 18 East 41st Street, 2nd Floor
>> New York, NY 10017-6271
>>
>> www.appinions.com
>>
>> Where Influence Isn’t a Game
>>
>>
>> On Tue, Apr 30, 2013 at 10:33 AM, Furkan KAMACI > >wrote:
>>
>> > I think that replication occurs after commit by default. It has been
>> long
>> > time however there is still mismatch between leader and replica
>> > (approximately 5 MB). I tried to pull indexes from leader but it is
>> still
>> > same.
>> >
>> > 2013/4/30 Michael Della Bitta 
>> >
>> > > In Solr Cloud, commits can happen at different times across replicas.
>> > Which
>> > > means merges also may happen at different times. So there's no
>> > expectation
>> > > of the cores of different replicas being totally similar.
>> > >
>> > >
>> > > Michael Della Bitta
>> > >
>> > > 
>> > > Appinions
>> > > 18 East 41st Street, 2nd Floor
>> > > New York, NY 10017-6271
>> > >
>> > > www.appinions.com
>> > >
>> > > Where Influence Isn’t a Game
>> > >
>> > >
>> > > On Tue, Apr 30, 2013 at 8:13 AM, Furkan KAMACI <
>> furkankam...@gmail.com
>> > > >wrote:
>> > >
>> > > > I use SolrCloud, 4.2.1 of Solr.
>> > > >
>> > > > Here is a detail from my admin page:
>> > > >
>> > > > Replication (Slave) Version Gen Size
>> > > > Master: 1367309548534 84 779.87 MB
>> > > > Slave: 1367307649512 82 784.44 MB
>> > > >
>> > > > When I use command=abortfetch still file size are not same. Any
>> idea?
>> > > >
>> > >
>> >
>>
>
>

Re: 4.2.1 Tutorial

2013-04-30 Thread Stefan Matheis 
Jon

Did you upgrade from an earlier Solr-Installation? If so, clearing your browser 
cache might help. There is fix for 4.3 in place.

If it does not, what is the output of 
http://localhost:8983/solr/admin/cores?wt=json ? Does it contain the 
"collection1" core?
To try a basic thing, what do you see / what does your browser throw in your 
direction if you open http://localhost:8983/solr/ (without any further params) ?

Stefan 


On Tuesday, April 30, 2013 at 4:07 PM, Jon Strayer wrote:

> I haven't found that, but I did find this:
> 
> Apr 30, 2013 9:38:10 AM org.apache.solr.core.CoreContainer create
> INFO: Creating SolrCore 'collection1' using instanceDir: solr/collection1
> Apr 30, 2013 9:38:10 AM org.apache.solr.core.SolrResourceLoader 
> INFO: new SolrResourceLoader for directory: 'solr/collection1/'
> Apr 30, 2013 9:38:10 AM org.apache.solr.core.SolrConfig initLibs
> 
> And then this:
> Apr 30, 2013 9:38:11 AM org.apache.solr.core.SolrCore 
> INFO: [collection1] Opening new SolrCore at solr/collection1/,
> dataDir=/Users/jon.strayer/solr/solr-4.2.1/example/solr/collection1/data/
> 
> And a little later:
> Apr 30, 2013 9:38:11 AM org.apache.solr.core.CachingDirectoryFactory get
> INFO: return new directory for
> /Users/jon.strayer/solr/solr-4.2.1/example/solr/collection1/data forceNew:
> false
> Apr 30, 2013 9:38:11 AM org.apache.solr.core.SolrCore getNewIndexDir
> INFO: New index directory detected: old=null
> new=/Users/jon.strayer/solr/solr-4.2.1/example/solr/collection1/data/index/
> Apr 30, 2013 9:38:11 AM org.apache.solr.core.CachingDirectoryFactory get
> INFO: return new directory for
> /Users/jon.strayer/solr/solr-4.2.1/example/solr/collection1/data/index
> forceNew: false
> 
> It looks to me as if the core does exist (and in fact I can add documents
> to it). But for some reason parts of the system don't know about it.
> 
> 
> On Mon, Apr 29, 2013 at 5:20 PM, Furkan KAMACI  (mailto:furkankam...@gmail.com)>wrote:
> 
> > Check your logs when you startup Solr if you get that error: "There exists
> > no core with the name "collection1"". Do you get any error as like
> > core:collection1 could not create or something like that?
> > 
> > 2013/4/29 Jon Strayer mailto:j...@strayer.org)>
> > 
> > > I can't be the only person to run into this, but I can't find any mention
> > > of it anywhere.
> > > 
> > > I have Solr 4.2.1 installed under OSX 10.8.3. I'm working my way through
> > > the tutorial.
> > > 
> > > When I click on this link:
> > http://localhost:8983/solr/#/collection1/queryI get the error message
> > "There exists no core with the name "collection1"".
> > > 
> > > This link works:
> > > http://localhost:8983/solr/collection1/select?q=solr&wt=xml
> > > 
> > > What am I doing wrong?
> > > 
> > > [image: Inline image 1]
> > > 
> > > --
> > > To *know* is one thing, and to know for certain *that* we know is
> > > 
> > 
> > another.
> > > --William James
> > 
> > 
> 
> 
> 
> 
> -- 
> To *know* is one thing, and to know for certain *that* we know is another.
> --William James
> 
>

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Michael Della Bitta 
Then there is no replication, and no slaves nor masters. There's a leader
and followers. Documents themselves are sent from the leader to followers,
not cores or segments. You should not expect the bits on the disk across
leaders and followers to be the same because of the reasons I mentioned
already.


Michael Della Bitta


Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271

www.appinions.com

Where Influence Isn’t a Game


On Tue, Apr 30, 2013 at 11:04 AM, Furkan KAMACI wrote:

> I use Solr 4.2.1 as SolrCloud
>
> 2013/4/30 Michael Della Bitta 
>
> > I'm a little confused. Are you using Solr Cloud, or ordinary replication?
> >
> >
> > Michael Della Bitta
> >
> > 
> > Appinions
> > 18 East 41st Street, 2nd Floor
> > New York, NY 10017-6271
> >
> > www.appinions.com
> >
> > Where Influence Isn’t a Game
> >
> >
> > On Tue, Apr 30, 2013 at 10:33 AM, Furkan KAMACI  > >wrote:
> >
> > > I think that replication occurs after commit by default. It has been
> long
> > > time however there is still mismatch between leader and replica
> > > (approximately 5 MB). I tried to pull indexes from leader but it is
> still
> > > same.
> > >
> > > 2013/4/30 Michael Della Bitta 
> > >
> > > > In Solr Cloud, commits can happen at different times across replicas.
> > > Which
> > > > means merges also may happen at different times. So there's no
> > > expectation
> > > > of the cores of different replicas being totally similar.
> > > >
> > > >
> > > > Michael Della Bitta
> > > >
> > > > 
> > > > Appinions
> > > > 18 East 41st Street, 2nd Floor
> > > > New York, NY 10017-6271
> > > >
> > > > www.appinions.com
> > > >
> > > > Where Influence Isn’t a Game
> > > >
> > > >
> > > > On Tue, Apr 30, 2013 at 8:13 AM, Furkan KAMACI <
> furkankam...@gmail.com
> > > > >wrote:
> > > >
> > > > > I use SolrCloud, 4.2.1 of Solr.
> > > > >
> > > > > Here is a detail from my admin page:
> > > > >
> > > > > Replication (Slave) Version Gen Size
> > > > > Master: 1367309548534 84 779.87 MB
> > > > > Slave: 1367307649512 82 784.44 MB
> > > > >
> > > > > When I use command=abortfetch still file size are not same. Any
> idea?
> > > > >
> > > >
> > >
> >
>

Re: How to Recovery Backup Snapshot at SolrCloud?

2013-04-30 Thread Michael Della Bitta 
Presumably you'd only be restoring a backup in the face of a catastrophe.

Yes, you'd need to stop the node. And the transaction logs may not be
useful in this case. You'd have trouble reconciling them with the version
of the index in your backup I would think.

Anybody who knows more about this want to chime in?


Michael Della Bitta


Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271

www.appinions.com

Where Influence Isn’t a Game


On Tue, Apr 30, 2013 at 11:03 AM, Furkan KAMACI wrote:

> Should I stop the node first? And what will happen to transaction logs?
> Should I backup it too?
>
> 2013/4/30 Michael Della Bitta 
>
> > That directory is the data directory for the core... you'd just swap it
> in.
> >
> >
> > Michael Della Bitta
> >
> > 
> > Appinions
> > 18 East 41st Street, 2nd Floor
> > New York, NY 10017-6271
> >
> > www.appinions.com
> >
> > Where Influence Isn’t a Game
> >
> >
> > On Tue, Apr 30, 2013 at 8:06 AM, Furkan KAMACI  > >wrote:
> >
> > > Hi Folks;
> > >
> > > I can backup my indexes at SolrCloud via
> > > http://_master_host_:_port_/solr/replication?command=backup
> > > and it creates a file called snapshot. I know that I should pull that
> > > directory any other safe place (a backup store) However what should I
> do
> > to
> > > make a recovery from that backup file?
> > >
> >
>

RE: java.lang.NullPointerException. I am trying to use CachedSqlEntityProcessor

2013-04-30 Thread srinalluri 
Thanks James for your reply.

I have updated to 3.6.2. Now the NullPointerException is gone. But the
entities with CachedSqlEntityProcessor don't add anything to solr.

And entities without CachedSqlEntityProcessor, are working fine.

Why entities with CachedSqlEntityProcessor don't do anything? What is wrong
in my entity?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/java-lang-NullPointerException-I-am-trying-to-use-CachedSqlEntityProcessor-tp4059815p4060043.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: What Happens When I Unload A Core?

2013-04-30 Thread Shalin Shekhar Mangar 
By default, an unload action will only unregister the Solr core (locally
and from zookeeper if running in cloud mode) to stop it from taking
requests. It will not delete any files.

The UNLOAD action also accepts the following parameters:
1. deleteIndex=true -- will delete the solr index after the core is
unregistered (asynchronously)
2. deleteDataDir=true -- will delete the entire data directory
3. deleteInstanceDir=true -- will delete the entire instance directory
including configuration files


On Tue, Apr 30, 2013 at 8:43 PM, Furkan KAMACI wrote:

> I use Solr 4.2.1
>
> What happens if I unload a core, I mean what does Solr do? Because solr.xml
> didn't change and I think that Solr should writes something to somewhere or
> deletes something from somewhere?
>



-- 
Regards,
Shalin Shekhar Mangar.

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Furkan KAMACI 
However I am using SolrCloud with 5 shards. Every leader has a replica.
What do you mean with "followers"?

2013/4/30 Michael Della Bitta 

> Then there is no replication, and no slaves nor masters. There's a leader
> and followers. Documents themselves are sent from the leader to followers,
> not cores or segments. You should not expect the bits on the disk across
> leaders and followers to be the same because of the reasons I mentioned
> already.
>
>
> Michael Della Bitta
>
> 
> Appinions
> 18 East 41st Street, 2nd Floor
> New York, NY 10017-6271
>
> www.appinions.com
>
> Where Influence Isn’t a Game
>
>
> On Tue, Apr 30, 2013 at 11:04 AM, Furkan KAMACI  >wrote:
>
> > I use Solr 4.2.1 as SolrCloud
> >
> > 2013/4/30 Michael Della Bitta 
> >
> > > I'm a little confused. Are you using Solr Cloud, or ordinary
> replication?
> > >
> > >
> > > Michael Della Bitta
> > >
> > > 
> > > Appinions
> > > 18 East 41st Street, 2nd Floor
> > > New York, NY 10017-6271
> > >
> > > www.appinions.com
> > >
> > > Where Influence Isn’t a Game
> > >
> > >
> > > On Tue, Apr 30, 2013 at 10:33 AM, Furkan KAMACI <
> furkankam...@gmail.com
> > > >wrote:
> > >
> > > > I think that replication occurs after commit by default. It has been
> > long
> > > > time however there is still mismatch between leader and replica
> > > > (approximately 5 MB). I tried to pull indexes from leader but it is
> > still
> > > > same.
> > > >
> > > > 2013/4/30 Michael Della Bitta 
> > > >
> > > > > In Solr Cloud, commits can happen at different times across
> replicas.
> > > > Which
> > > > > means merges also may happen at different times. So there's no
> > > > expectation
> > > > > of the cores of different replicas being totally similar.
> > > > >
> > > > >
> > > > > Michael Della Bitta
> > > > >
> > > > > 
> > > > > Appinions
> > > > > 18 East 41st Street, 2nd Floor
> > > > > New York, NY 10017-6271
> > > > >
> > > > > www.appinions.com
> > > > >
> > > > > Where Influence Isn’t a Game
> > > > >
> > > > >
> > > > > On Tue, Apr 30, 2013 at 8:13 AM, Furkan KAMACI <
> > furkankam...@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > I use SolrCloud, 4.2.1 of Solr.
> > > > > >
> > > > > > Here is a detail from my admin page:
> > > > > >
> > > > > > Replication (Slave) Version Gen Size
> > > > > > Master: 1367309548534 84 779.87 MB
> > > > > > Slave: 1367307649512 82 784.44 MB
> > > > > >
> > > > > > When I use command=abortfetch still file size are not same. Any
> > idea?
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: What Happens When I Unload A Core?

2013-04-30 Thread Furkan KAMACI 
I have closed my application and restarted it.If it didn't change anything
could you tell me how admin page says:

"There are no SolrCores running.
 Using the Solr Admin UI currently requires at least one SolrCore."

I think it stored something to somewhere?


2013/4/30 Shalin Shekhar Mangar 

> By default, an unload action will only unregister the Solr core (locally
> and from zookeeper if running in cloud mode) to stop it from taking
> requests. It will not delete any files.
>
> The UNLOAD action also accepts the following parameters:
> 1. deleteIndex=true -- will delete the solr index after the core is
> unregistered (asynchronously)
> 2. deleteDataDir=true -- will delete the entire data directory
> 3. deleteInstanceDir=true -- will delete the entire instance directory
> including configuration files
>
>
> On Tue, Apr 30, 2013 at 8:43 PM, Furkan KAMACI  >wrote:
>
> > I use Solr 4.2.1
> >
> > What happens if I unload a core, I mean what does Solr do? Because
> solr.xml
> > didn't change and I think that Solr should writes something to somewhere
> or
> > deletes something from somewhere?
> >
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>

Re: What Happens When I Unload A Core?

2013-04-30 Thread Shalin Shekhar Mangar 
The UNLOAD command removes the core name from solr.xml


On Tue, Apr 30, 2013 at 11:28 PM, Furkan KAMACI wrote:

> I have closed my application and restarted it.If it didn't change anything
> could you tell me how admin page says:
>
> "There are no SolrCores running.
>  Using the Solr Admin UI currently requires at least one SolrCore."
>
> I think it stored something to somewhere?
>
>
> 2013/4/30 Shalin Shekhar Mangar 
>
> > By default, an unload action will only unregister the Solr core (locally
> > and from zookeeper if running in cloud mode) to stop it from taking
> > requests. It will not delete any files.
> >
> > The UNLOAD action also accepts the following parameters:
> > 1. deleteIndex=true -- will delete the solr index after the core is
> > unregistered (asynchronously)
> > 2. deleteDataDir=true -- will delete the entire data directory
> > 3. deleteInstanceDir=true -- will delete the entire instance directory
> > including configuration files
> >
> >
> > On Tue, Apr 30, 2013 at 8:43 PM, Furkan KAMACI  > >wrote:
> >
> > > I use Solr 4.2.1
> > >
> > > What happens if I unload a core, I mean what does Solr do? Because
> > solr.xml
> > > didn't change and I think that Solr should writes something to
> somewhere
> > or
> > > deletes something from somewhere?
> > >
> >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
>



-- 
Regards,
Shalin Shekhar Mangar.

Re: What Happens When I Unload A Core?

2013-04-30 Thread Furkan KAMACI 
Oops, it changes solr.xml, OK.

2013/4/30 Furkan KAMACI 

> I have closed my application and restarted it.If it didn't change anything
> could you tell me how admin page says:
>
> "There are no SolrCores running.
>  Using the Solr Admin UI currently requires at least one SolrCore."
>
> I think it stored something to somewhere?
>
>
> 2013/4/30 Shalin Shekhar Mangar 
>
>> By default, an unload action will only unregister the Solr core (locally
>> and from zookeeper if running in cloud mode) to stop it from taking
>> requests. It will not delete any files.
>>
>> The UNLOAD action also accepts the following parameters:
>> 1. deleteIndex=true -- will delete the solr index after the core is
>> unregistered (asynchronously)
>> 2. deleteDataDir=true -- will delete the entire data directory
>> 3. deleteInstanceDir=true -- will delete the entire instance directory
>> including configuration files
>>
>>
>> On Tue, Apr 30, 2013 at 8:43 PM, Furkan KAMACI > >wrote:
>>
>> > I use Solr 4.2.1
>> >
>> > What happens if I unload a core, I mean what does Solr do? Because
>> solr.xml
>> > didn't change and I think that Solr should writes something to
>> somewhere or
>> > deletes something from somewhere?
>> >
>>
>>
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>
>

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Michael Della Bitta 
I'd say a follower is a participant in a shard that's not the leader.


Michael Della Bitta


Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271

www.appinions.com

Where Influence Isn’t a Game


On Tue, Apr 30, 2013 at 1:27 PM, Furkan KAMACI wrote:

> However I am using SolrCloud with 5 shards. Every leader has a replica.
> What do you mean with "followers"?
>
> 2013/4/30 Michael Della Bitta 
>
> > Then there is no replication, and no slaves nor masters. There's a leader
> > and followers. Documents themselves are sent from the leader to
> followers,
> > not cores or segments. You should not expect the bits on the disk across
> > leaders and followers to be the same because of the reasons I mentioned
> > already.
> >
> >
> > Michael Della Bitta
> >
> > 
> > Appinions
> > 18 East 41st Street, 2nd Floor
> > New York, NY 10017-6271
> >
> > www.appinions.com
> >
> > Where Influence Isn’t a Game
> >
> >
> > On Tue, Apr 30, 2013 at 11:04 AM, Furkan KAMACI  > >wrote:
> >
> > > I use Solr 4.2.1 as SolrCloud
> > >
> > > 2013/4/30 Michael Della Bitta 
> > >
> > > > I'm a little confused. Are you using Solr Cloud, or ordinary
> > replication?
> > > >
> > > >
> > > > Michael Della Bitta
> > > >
> > > > 
> > > > Appinions
> > > > 18 East 41st Street, 2nd Floor
> > > > New York, NY 10017-6271
> > > >
> > > > www.appinions.com
> > > >
> > > > Where Influence Isn’t a Game
> > > >
> > > >
> > > > On Tue, Apr 30, 2013 at 10:33 AM, Furkan KAMACI <
> > furkankam...@gmail.com
> > > > >wrote:
> > > >
> > > > > I think that replication occurs after commit by default. It has
> been
> > > long
> > > > > time however there is still mismatch between leader and replica
> > > > > (approximately 5 MB). I tried to pull indexes from leader but it is
> > > still
> > > > > same.
> > > > >
> > > > > 2013/4/30 Michael Della Bitta 
> > > > >
> > > > > > In Solr Cloud, commits can happen at different times across
> > replicas.
> > > > > Which
> > > > > > means merges also may happen at different times. So there's no
> > > > > expectation
> > > > > > of the cores of different replicas being totally similar.
> > > > > >
> > > > > >
> > > > > > Michael Della Bitta
> > > > > >
> > > > > > 
> > > > > > Appinions
> > > > > > 18 East 41st Street, 2nd Floor
> > > > > > New York, NY 10017-6271
> > > > > >
> > > > > > www.appinions.com
> > > > > >
> > > > > > Where Influence Isn’t a Game
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 30, 2013 at 8:13 AM, Furkan KAMACI <
> > > furkankam...@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > I use SolrCloud, 4.2.1 of Solr.
> > > > > > >
> > > > > > > Here is a detail from my admin page:
> > > > > > >
> > > > > > > Replication (Slave) Version Gen Size
> > > > > > > Master: 1367309548534 84 779.87 MB
> > > > > > > Slave: 1367307649512 82 784.44 MB
> > > > > > >
> > > > > > > When I use command=abortfetch still file size are not same. Any
> > > idea?
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Furkan KAMACI 
If we talk about SolrCloud terminology does follower and replica means
same? Is there any documentation at wiki for that?

2013/4/30 Michael Della Bitta 

> I'd say a follower is a participant in a shard that's not the leader.
>
>
> Michael Della Bitta
>
> 
> Appinions
> 18 East 41st Street, 2nd Floor
> New York, NY 10017-6271
>
> www.appinions.com
>
> Where Influence Isn’t a Game
>
>
> On Tue, Apr 30, 2013 at 1:27 PM, Furkan KAMACI  >wrote:
>
> > However I am using SolrCloud with 5 shards. Every leader has a replica.
> > What do you mean with "followers"?
> >
> > 2013/4/30 Michael Della Bitta 
> >
> > > Then there is no replication, and no slaves nor masters. There's a
> leader
> > > and followers. Documents themselves are sent from the leader to
> > followers,
> > > not cores or segments. You should not expect the bits on the disk
> across
> > > leaders and followers to be the same because of the reasons I mentioned
> > > already.
> > >
> > >
> > > Michael Della Bitta
> > >
> > > 
> > > Appinions
> > > 18 East 41st Street, 2nd Floor
> > > New York, NY 10017-6271
> > >
> > > www.appinions.com
> > >
> > > Where Influence Isn’t a Game
> > >
> > >
> > > On Tue, Apr 30, 2013 at 11:04 AM, Furkan KAMACI <
> furkankam...@gmail.com
> > > >wrote:
> > >
> > > > I use Solr 4.2.1 as SolrCloud
> > > >
> > > > 2013/4/30 Michael Della Bitta 
> > > >
> > > > > I'm a little confused. Are you using Solr Cloud, or ordinary
> > > replication?
> > > > >
> > > > >
> > > > > Michael Della Bitta
> > > > >
> > > > > 
> > > > > Appinions
> > > > > 18 East 41st Street, 2nd Floor
> > > > > New York, NY 10017-6271
> > > > >
> > > > > www.appinions.com
> > > > >
> > > > > Where Influence Isn’t a Game
> > > > >
> > > > >
> > > > > On Tue, Apr 30, 2013 at 10:33 AM, Furkan KAMACI <
> > > furkankam...@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > I think that replication occurs after commit by default. It has
> > been
> > > > long
> > > > > > time however there is still mismatch between leader and replica
> > > > > > (approximately 5 MB). I tried to pull indexes from leader but it
> is
> > > > still
> > > > > > same.
> > > > > >
> > > > > > 2013/4/30 Michael Della Bitta  >
> > > > > >
> > > > > > > In Solr Cloud, commits can happen at different times across
> > > replicas.
> > > > > > Which
> > > > > > > means merges also may happen at different times. So there's no
> > > > > > expectation
> > > > > > > of the cores of different replicas being totally similar.
> > > > > > >
> > > > > > >
> > > > > > > Michael Della Bitta
> > > > > > >
> > > > > > > 
> > > > > > > Appinions
> > > > > > > 18 East 41st Street, 2nd Floor
> > > > > > > New York, NY 10017-6271
> > > > > > >
> > > > > > > www.appinions.com
> > > > > > >
> > > > > > > Where Influence Isn’t a Game
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Apr 30, 2013 at 8:13 AM, Furkan KAMACI <
> > > > furkankam...@gmail.com
> > > > > > > >wrote:
> > > > > > >
> > > > > > > > I use SolrCloud, 4.2.1 of Solr.
> > > > > > > >
> > > > > > > > Here is a detail from my admin page:
> > > > > > > >
> > > > > > > > Replication (Slave) Version Gen Size
> > > > > > > > Master: 1367309548534 84 779.87 MB
> > > > > > > > Slave: 1367307649512 82 784.44 MB
> > > > > > > >
> > > > > > > > When I use command=abortfetch still file size are not same.
> Any
> > > > idea?
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Michael Della Bitta 
I could be getting this wrong, and the wiki is down at the moment, but I
think a replica can be a leader, whereas a follower is definitely not.


Michael Della Bitta


Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271

www.appinions.com

Where Influence Isn’t a Game


On Tue, Apr 30, 2013 at 2:19 PM, Furkan KAMACI wrote:

> If we talk about SolrCloud terminology does follower and replica means
> same? Is there any documentation at wiki for that?
>
> 2013/4/30 Michael Della Bitta 
>
> > I'd say a follower is a participant in a shard that's not the leader.
> >
> >
> > Michael Della Bitta
> >
> > 
> > Appinions
> > 18 East 41st Street, 2nd Floor
> > New York, NY 10017-6271
> >
> > www.appinions.com
> >
> > Where Influence Isn’t a Game
> >
> >
> > On Tue, Apr 30, 2013 at 1:27 PM, Furkan KAMACI  > >wrote:
> >
> > > However I am using SolrCloud with 5 shards. Every leader has a replica.
> > > What do you mean with "followers"?
> > >
> > > 2013/4/30 Michael Della Bitta 
> > >
> > > > Then there is no replication, and no slaves nor masters. There's a
> > leader
> > > > and followers. Documents themselves are sent from the leader to
> > > followers,
> > > > not cores or segments. You should not expect the bits on the disk
> > across
> > > > leaders and followers to be the same because of the reasons I
> mentioned
> > > > already.
> > > >
> > > >
> > > > Michael Della Bitta
> > > >
> > > > 
> > > > Appinions
> > > > 18 East 41st Street, 2nd Floor
> > > > New York, NY 10017-6271
> > > >
> > > > www.appinions.com
> > > >
> > > > Where Influence Isn’t a Game
> > > >
> > > >
> > > > On Tue, Apr 30, 2013 at 11:04 AM, Furkan KAMACI <
> > furkankam...@gmail.com
> > > > >wrote:
> > > >
> > > > > I use Solr 4.2.1 as SolrCloud
> > > > >
> > > > > 2013/4/30 Michael Della Bitta 
> > > > >
> > > > > > I'm a little confused. Are you using Solr Cloud, or ordinary
> > > > replication?
> > > > > >
> > > > > >
> > > > > > Michael Della Bitta
> > > > > >
> > > > > > 
> > > > > > Appinions
> > > > > > 18 East 41st Street, 2nd Floor
> > > > > > New York, NY 10017-6271
> > > > > >
> > > > > > www.appinions.com
> > > > > >
> > > > > > Where Influence Isn’t a Game
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 30, 2013 at 10:33 AM, Furkan KAMACI <
> > > > furkankam...@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > I think that replication occurs after commit by default. It has
> > > been
> > > > > long
> > > > > > > time however there is still mismatch between leader and replica
> > > > > > > (approximately 5 MB). I tried to pull indexes from leader but
> it
> > is
> > > > > still
> > > > > > > same.
> > > > > > >
> > > > > > > 2013/4/30 Michael Della Bitta <
> michael.della.bi...@appinions.com
> > >
> > > > > > >
> > > > > > > > In Solr Cloud, commits can happen at different times across
> > > > replicas.
> > > > > > > Which
> > > > > > > > means merges also may happen at different times. So there's
> no
> > > > > > > expectation
> > > > > > > > of the cores of different replicas being totally similar.
> > > > > > > >
> > > > > > > >
> > > > > > > > Michael Della Bitta
> > > > > > > >
> > > > > > > > 
> > > > > > > > Appinions
> > > > > > > > 18 East 41st Street, 2nd Floor
> > > > > > > > New York, NY 10017-6271
> > > > > > > >
> > > > > > > > www.appinions.com
> > > > > > > >
> > > > > > > > Where Influence Isn’t a Game
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Apr 30, 2013 at 8:13 AM, Furkan KAMACI <
> > > > > furkankam...@gmail.com
> > > > > > > > >wrote:
> > > > > > > >
> > > > > > > > > I use SolrCloud, 4.2.1 of Solr.
> > > > > > > > >
> > > > > > > > > Here is a detail from my admin page:
> > > > > > > > >
> > > > > > > > > Replication (Slave) Version Gen Size
> > > > > > > > > Master: 1367309548534 84 779.87 MB
> > > > > > > > > Slave: 1367307649512 82 784.44 MB
> > > > > > > > >
> > > > > > > > > When I use command=abortfetch still file size are not same.
> > Any
> > > > > idea?
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: How to Recovery Backup Snapshot at SolrCloud?

2013-04-30 Thread Furkan KAMACI 
I had index and tlog folder under my data folder. I have a snapshot folder
too when I make backup. However what will I do next if I want to use
backup, will I remove index and tlog folders and put just my snapshot
folder? What folks do?

2013/4/30 Michael Della Bitta 

> Presumably you'd only be restoring a backup in the face of a catastrophe.
>
> Yes, you'd need to stop the node. And the transaction logs may not be
> useful in this case. You'd have trouble reconciling them with the version
> of the index in your backup I would think.
>
> Anybody who knows more about this want to chime in?
>
>
> Michael Della Bitta
>
> 
> Appinions
> 18 East 41st Street, 2nd Floor
> New York, NY 10017-6271
>
> www.appinions.com
>
> Where Influence Isn’t a Game
>
>
> On Tue, Apr 30, 2013 at 11:03 AM, Furkan KAMACI  >wrote:
>
> > Should I stop the node first? And what will happen to transaction logs?
> > Should I backup it too?
> >
> > 2013/4/30 Michael Della Bitta 
> >
> > > That directory is the data directory for the core... you'd just swap it
> > in.
> > >
> > >
> > > Michael Della Bitta
> > >
> > > 
> > > Appinions
> > > 18 East 41st Street, 2nd Floor
> > > New York, NY 10017-6271
> > >
> > > www.appinions.com
> > >
> > > Where Influence Isn’t a Game
> > >
> > >
> > > On Tue, Apr 30, 2013 at 8:06 AM, Furkan KAMACI  > > >wrote:
> > >
> > > > Hi Folks;
> > > >
> > > > I can backup my indexes at SolrCloud via
> > > > http://_master_host_:_port_/solr/replication?command=backup
> > > > and it creates a file called snapshot. I know that I should pull that
> > > > directory any other safe place (a backup store) However what should I
> > do
> > > to
> > > > make a recovery from that backup file?
> > > >
> > >
> >
>

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Furkan KAMACI 
It would be nice if I learn what is a follower means and how to define them
(I know the example of replica but didn't see an example of follower yet)

2013/4/30 Michael Della Bitta 

> I could be getting this wrong, and the wiki is down at the moment, but I
> think a replica can be a leader, whereas a follower is definitely not.
>
>
> Michael Della Bitta
>
> 
> Appinions
> 18 East 41st Street, 2nd Floor
> New York, NY 10017-6271
>
> www.appinions.com
>
> Where Influence Isn’t a Game
>
>
> On Tue, Apr 30, 2013 at 2:19 PM, Furkan KAMACI  >wrote:
>
> > If we talk about SolrCloud terminology does follower and replica means
> > same? Is there any documentation at wiki for that?
> >
> > 2013/4/30 Michael Della Bitta 
> >
> > > I'd say a follower is a participant in a shard that's not the leader.
> > >
> > >
> > > Michael Della Bitta
> > >
> > > 
> > > Appinions
> > > 18 East 41st Street, 2nd Floor
> > > New York, NY 10017-6271
> > >
> > > www.appinions.com
> > >
> > > Where Influence Isn’t a Game
> > >
> > >
> > > On Tue, Apr 30, 2013 at 1:27 PM, Furkan KAMACI  > > >wrote:
> > >
> > > > However I am using SolrCloud with 5 shards. Every leader has a
> replica.
> > > > What do you mean with "followers"?
> > > >
> > > > 2013/4/30 Michael Della Bitta 
> > > >
> > > > > Then there is no replication, and no slaves nor masters. There's a
> > > leader
> > > > > and followers. Documents themselves are sent from the leader to
> > > > followers,
> > > > > not cores or segments. You should not expect the bits on the disk
> > > across
> > > > > leaders and followers to be the same because of the reasons I
> > mentioned
> > > > > already.
> > > > >
> > > > >
> > > > > Michael Della Bitta
> > > > >
> > > > > 
> > > > > Appinions
> > > > > 18 East 41st Street, 2nd Floor
> > > > > New York, NY 10017-6271
> > > > >
> > > > > www.appinions.com
> > > > >
> > > > > Where Influence Isn’t a Game
> > > > >
> > > > >
> > > > > On Tue, Apr 30, 2013 at 11:04 AM, Furkan KAMACI <
> > > furkankam...@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > I use Solr 4.2.1 as SolrCloud
> > > > > >
> > > > > > 2013/4/30 Michael Della Bitta  >
> > > > > >
> > > > > > > I'm a little confused. Are you using Solr Cloud, or ordinary
> > > > > replication?
> > > > > > >
> > > > > > >
> > > > > > > Michael Della Bitta
> > > > > > >
> > > > > > > 
> > > > > > > Appinions
> > > > > > > 18 East 41st Street, 2nd Floor
> > > > > > > New York, NY 10017-6271
> > > > > > >
> > > > > > > www.appinions.com
> > > > > > >
> > > > > > > Where Influence Isn’t a Game
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Apr 30, 2013 at 10:33 AM, Furkan KAMACI <
> > > > > furkankam...@gmail.com
> > > > > > > >wrote:
> > > > > > >
> > > > > > > > I think that replication occurs after commit by default. It
> has
> > > > been
> > > > > > long
> > > > > > > > time however there is still mismatch between leader and
> replica
> > > > > > > > (approximately 5 MB). I tried to pull indexes from leader but
> > it
> > > is
> > > > > > still
> > > > > > > > same.
> > > > > > > >
> > > > > > > > 2013/4/30 Michael Della Bitta <
> > michael.della.bi...@appinions.com
> > > >
> > > > > > > >
> > > > > > > > > In Solr Cloud, commits can happen at different times across
> > > > > replicas.
> > > > > > > > Which
> > > > > > > > > means merges also may happen at different times. So there's
> > no
> > > > > > > > expectation
> > > > > > > > > of the cores of different replicas being totally similar.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Michael Della Bitta
> > > > > > > > >
> > > > > > > > > 
> > > > > > > > > Appinions
> > > > > > > > > 18 East 41st Street, 2nd Floor
> > > > > > > > > New York, NY 10017-6271
> > > > > > > > >
> > > > > > > > > www.appinions.com
> > > > > > > > >
> > > > > > > > > Where Influence Isn’t a Game
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, Apr 30, 2013 at 8:13 AM, Furkan KAMACI <
> > > > > > furkankam...@gmail.com
> > > > > > > > > >wrote:
> > > > > > > > >
> > > > > > > > > > I use SolrCloud, 4.2.1 of Solr.
> > > > > > > > > >
> > > > > > > > > > Here is a detail from my admin page:
> > > > > > > > > >
> > > > > > > > > > Replication (Slave) Version Gen Size
> > > > > > > > > > Master: 1367309548534 84 779.87 MB
> > > > > > > > > > Slave: 1367307649512 82 784.44 MB
> > > > > > > > > >
> > > > > > > > > > When I use command=abortfetch still file size are not
> same.
> > > Any
> > > > > > idea?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Solr Range Queries with Field value

2013-04-30 Thread Erick Erickson 
Could you define your use-case in some more detail? On the
surface, this query doesn't really make a lot of sense. How
would merchant_end_of_day_in_utc_epoch be determined?
Presumably there are zillions of values across your index for
this value, depending on the document. Which one should be
used?

Best
Erick

On Mon, Apr 29, 2013 at 8:34 PM, Indika Tantrigoda  wrote:
> Hi All,
>
> I'd like to know if its possible to use a field value in a Solr range query
> ? Something similar to start_time_utc_epoch:[1970-01-01T00:00:00Z TO
> merchant_end_of_day_in_utc_epoch])
>
> merchant_end_of_day_in_utc_epoch is an indexed field.
>
> I am using Solr 4.
>
> Thanks,
> Indika

How to recover from "Error opening new searcher" when machine crashed while indexing

2013-04-30 Thread Utkarsh Sengar 
Solr 4.0 was indexing data and the machine crashed.

Any suggestions on how to recover my index since I don't want to delete my
data directory?

When I try to start it again, I get this error:
ERROR 12:01:46,493 Failed to load Solr core: xyz.index1
ERROR 12:01:46,493 Cause:
ERROR 12:01:46,494 Error opening new searcher
org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.(SolrCore.java:701)
at org.apache.solr.core.SolrCore.(SolrCore.java:564)
at
org.apache.solr.core.CassandraCoreContainer.load(CassandraCoreContainer.java:213)
at
com.datastax.bdp.plugin.SolrCorePlugin.activateImpl(SolrCorePlugin.java:66)
at
com.datastax.bdp.plugin.PluginManager$PluginInitializer.call(PluginManager.java:161)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1290)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1402)
at org.apache.solr.core.SolrCore.(SolrCore.java:675)
... 9 more
Caused by: org.apache.lucene.index.IndexNotFoundException: no segments*
file found in 
NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/media/SSD/data/solr.data/rlcatalogks.prodinfo/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@d7581b;
maxCacheMB=48.0 maxMergeSizeMB=4.0): files: [_73ne_nrm.cfs,
_73ng_Lucene40_0.tip, _73nh_nrm.cfs, _73ng_Lucene40_0.tim, _73nf.fnm,
_73n5_Lucene40_0.frq, _73ne.fdt, _73nh.fdx, _73ne_nrm.cfe, _73ne.fdx,
_73ne_Lucene40_0.tim, _73ne.si, _73ni.fnm, _73nh_Lucene40_0.prx, _73ni.fdt,
_73n5.si, _73ne_Lucene40_0.tip, _73nf_Lucene40_0.frq, _73nf_Lucene40_0.prx,
_73nf_nrm.cfe, _73ne_Lucene40_0.frq, _73ng_Lucene40_0.prx,
_73nf_Lucene40_0.tip, _73n5.fdx, _73ng_Lucene40_0.frq, _73ng.fnm,
_73ni.fdx, _73n5.fnm, _73nf_Lucene40_0.tim, _73ni.si, _73n5.fdt,
_73nf_nrm.cfs, _73nh_nrm.cfe, _73ni_Lucene40_0.frq, _73ng.fdx,
_73ne_Lucene40_0.prx, _73nh.fnm, _73nh_Lucene40_0.tip,
_73nh_Lucene40_0.tim, _73nh.si, _73n5_Lucene40_0.tip, _73ni_Lucene40_0.prx,
_73n5_Lucene40_0.tim, _73nf.si, _73ng_nrm.cfe, _73n5_Lucene40_0.prx,
_392j_42f.del, _73ng.fdt, _73ng.si, _73ni_nrm.cfe, _73n5_nrm.cfe,
_73ni_nrm.cfs, _73nf.fdx, _73ni_Lucene40_0.tip, _73n5_nrm.cfs,
_73ni_Lucene40_0.tim, _73nf.fdt, _73ne.fnm, _73nh.fdt,
_73nh_Lucene40_0.frq, _73ng_nrm.cfs]


-- 
Thanks,
-Utkarsh

string field does not yield exact match result using qf parameter

2013-04-30 Thread kirpakaroji 
  I have a question regarding boosting the exact match queries to top,
followed by partial match and if there is no exact match then give me
partial match. The following 2 solutions have yielded different results, and
I was not clear on it why

   This is the schema I have

   
   
   
   
   f1


  






  


in my solrconfig.xml I have
   f1
   f1^10 f3^1
   f1^10 f3^1

now if I try to specify the query with these parameters in solrconfig.xml,
99% of the time exactmatch first and then partial match 1%of the time the
exact match result is in the index but does not show on the results and does
not give any partial matches for that query either.

But if I make it qf=f3&pf=f1^10 f3^1 yields the exactmatch result on top
100% of the time.

   Why I am seeing this behavior.

is there anyway to say qf=f1 on the interface and get only exact results if
present (in this case though f1 is string but the q parameter has spaces. do
I need to use pf field
   I am using dismax query parser.

Thanks



--
View this message in context: 
http://lucene.472066.n3.nabble.com/string-field-does-not-yield-exact-match-result-using-qf-parameter-tp4060096.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr Range Queries with Field value

2013-04-30 Thread Arun Rangarajan 
Erick,

I believe Indika wants to do this SQL WHERE clause in Solr:
WHERE start_time_utc_epoch >= '1970-01-01T00:00:00Z' AND start_time_utc_epoch
<= merchant_end_of_day_in_utc_epoch


On Tue, Apr 30, 2013 at 11:49 AM, Erick Erickson wrote:

> Could you define your use-case in some more detail? On the
> surface, this query doesn't really make a lot of sense. How
> would merchant_end_of_day_in_utc_epoch be determined?
> Presumably there are zillions of values across your index for
> this value, depending on the document. Which one should be
> used?
>
> Best
> Erick
>
> On Mon, Apr 29, 2013 at 8:34 PM, Indika Tantrigoda 
> wrote:
> > Hi All,
> >
> > I'd like to know if its possible to use a field value in a Solr range
> query
> > ? Something similar to start_time_utc_epoch:[1970-01-01T00:00:00Z TO
> > merchant_end_of_day_in_utc_epoch])
> >
> > merchant_end_of_day_in_utc_epoch is an indexed field.
> >
> > I am using Solr 4.
> >
> > Thanks,
> > Indika
>

RE: Bizarre Solr issue

2013-04-30 Thread Jack.Drysdale.ctr 
The logs stdout and stderr are blank.  The log request had info, but nothing
that is related.

Persistent flag is set to true in all environments.

After backing up the solr.xml file, the collections were manually erased
from the file allowing me to list collections without breaking!  Yay.

This allowed my script to create at least one of the collections.  I'm sure
we'll get the other, soon.

Thank you to everyone for your help!

Jack

___
-Original Message-
From: jack.drysdale@ustranscom.mil
[mailto:jack.drysdale@ustranscom.mil] 
Sent: Monday, April 29, 2013 12:27 PM
To: solr-user@lucene.apache.org
Subject: RE: Bizarre Solr issue

Hello, Shawn, and thanks for your reply.

I will look into this, ASAP.  I know that on one of the dev environments the
persistent flag is set to "true"; I'll check the others and the production.

I will also see if someone can get me a copy of the logs from the production
environment to see if any more detail is contained within.

Thanks,

Jack

-Original Message-
From: Shawn Heisey [mailto:s...@elyograg.org]
Sent: Monday, April 29, 2013 12:19 PM
To: solr-user@lucene.apache.org
Subject: Re: Bizarre Solr issue

On 4/29/2013 8:15 AM, jack.drysdale@ustranscom.mil wrote:
> Production environment is *nix running CF 9.0.0, with both Verity and 
> Solr collections.
>
> Trying to list collections is breaking - one collection in particular 
> is breaking the CFCOLLECTION action="list": Error message states that 
> the solrconfig.xml file cannot be found.
>
> I unregistered this collection via CFAdmin, then went into the file 
> system and deleted the folders for this collection and restarted both 
> Application and Solr services. Ran the script, again, and still 
> getting the same error message for the collection that we just 
> completely removed.  It's NOT being cached in the browser.
>
> This is working fine in development (Windows environment, CF9.0.1).

CFCOLLECTION and CFAdmin are not part of Solr.  We have no way of knowing
what happens when you do things in CFAdmin.  I do have one possible idea of
what might be going wrong here, though.

Here's how multi-core Solr works in all versions prior to 4.3: The directory
named with the solr.solr.home property (defaulting to ./solr) contains a
file called solr.xml.  This file describes the index cores that Solr knows
about and defines a few global settings.  Solr includes something called the
CoreAdmin API for manipulating cores and solr.xml, which is probably
utilized by CFAdmin.

If the solr.xml file is missing an attribute called persistent on the 
tag, or that attribute is set to false, then changes made using the
CoreAdmin API are not persisted in the solr.xml file on disk, so when Solr
restarts, it will use what it had before.

Note: SolrCloud (4.0 and later) does add the concept of collections - a
cluster-wide view of multiple cores.  SolrCloud is not required, and with
version 1.4.1, you won't have to worry about it.

Thanks,
Shawn



smime.p7s
Description: S/MIME cryptographic signature

Random IllegalStateExceptions on Solr slave (3.6.1)

2013-04-30 Thread Arun Rangarajan 
We have a master-slave Solr set up and run live queries only against the
slave. Full import (with optimize) happens on master every day at 2 a.m.
Delta imports happen every 10 min for one entity and every hour for another
entity.

The following exceptions occur a few times every day in our app logs:

...
org.apache.solr.client.solrj.SolrServerException: Error executing query
at
org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:95)
at
org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:118)
...

and this one:
...
org.apache.solr.client.solrj.SolrServerException: java.net.SocketException:
Connection reset
at
org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:478)
at
org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:244)
at
org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:89)
at
org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:118)
...

These happen on different queries at different times, so they are not
query-dependent.

If I inspect the localhost.log file in tomcat on the solr slave server, the
following exception occurs at the same time:

Apr 06, 2013 7:16:33 AM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet default threw exception
java.lang.IllegalStateException
at
org.apache.catalina.connector.ResponseFacade.sendError(ResponseFacade.java:407)
at
org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:389)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:291)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:722)
...


The replication set-up is as follows:


  
${enable.master:false}
startup
commit
optimize
solrconfig.xml,data-config.xml,schema.xml,stopwords.txt,synonyms.txt,elevate.xml
  
  
${enable.slave:false}
http://${master.ip}:${master.port}/solr/${
solr.core.name}/replication
00:01:00
  
  

Aside from the full and delta imports, we also have external file fields
which are loaded every 1 hour with reloadCache on both master and slave.

What is causing these exceptions and how do I fix it?

Thanks.

Re: Not In query

2013-04-30 Thread Jan Høydahl 
Hi,

How, practically would a user end up with 200.000 documents excluded? Is there 
some way in your application to exclude "categories" of documents with one 
click? If so, I would index those category IDs on all docs in that category, 
and then do &fq=-cat:123 instead of adding all the individual docids. Anyway, 
I'd start with the simple approach and then optimize once you (perhaps, perhaps 
not) bump into problems. Most likely it will work like a charm :)

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Solr Training - www.solrtraining.com

30. apr. 2013 kl. 16:21 skrev André Maldonado :

> Thank's Jan for your reply.
> 
> My application has thousands of users and I don't know yet how many of them
> will use this feature. They can exclude one document from their search
> results or can exclude 200.000 documents. It's much more natural that they
> exclude something like 50~300 documents. More than this will be strange.
> 
> However, I don't know how cache will work because we have a large number of
> users who can use this feature. Even that query for user 1 be cached, it
> won't work for other users.
> 
> Do you see another solution for this case?
> 
> Thank's
> 
> 
> 
> *
> --
> *
> *"E conhecereis a verdade, e a verdade vos libertará." (João 8:32)*
> 
> *andre.maldonado*@gmail.com 
> (11) 9112-4227
> 
> 
> 
> 
>   
> 
>  
> 
>  
> 
> 
> 
> On Fri, Apr 26, 2013 at 6:18 PM, Jan Høydahl  wrote:
> 
>> I would start with the way you propose, a negative filter
>> 
>> q=foo bar&fq=-id:(123 729 640 112...)
>> 
>> This will effectively hide those doc ids, and a benefit is that it is
>> cached so if the list of ids is long, you'll only take the performance hit
>> the first time. I don't know your application, but if it is highly likely
>> that a single user will add excludes for several thousand ids then you
>> should perhaps consider other options and benchmark up front.
>> 
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>> Solr Training - www.solrtraining.com
>> 
>> 26. apr. 2013 kl. 21:50 skrev André Maldonado :
>> 
>>> Hi all.
>>> 
>>> We have an index with 300.000 documents and a lot, a lot of fields.
>>> 
>>> We're planning a module where users will choose some documents to exclude
>>> from their search results. So, these documents will be excluded for UserA
>>> and visible for UserB.
>>> 
>>> So, we have some options to do this. The simplest way is to do a "Not In"
>>> query in document id. But we don't know the performance impact this will
>>> have. Is this an option?
>>> 
>>> There is another reasonable way to accomplish this?
>>> 
>>> Thank's
>>> 
>>> *
>>> 
>> --
>>> *
>>> *"E conhecereis a verdade, e a verdade vos libertará." (João 8:32)*
>>> 
>>> *andre.maldonado*@gmail.com 
>>> (11) 9112-4227
>>> 
>>> 
>>> 
>>> 
>>>  <
>> http://www.delicious.com/andre.maldonado>
>>> 
>>> 
>>> 
>> 
>>

Re: string field does not yield exact match result using qf parameter

2013-04-30 Thread Jan Høydahl 
Hi,

The "pf" feature will only kick in for phrases, i.e. multiple tokens. Per 
definition a "string" is one single token, so it will never kick in for strings.

A workaround can be found here: https://github.com/cominvent/exactmatch

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com
Solr Training - www.solrtraining.com

30. apr. 2013 kl. 20:52 skrev kirpakaroji :

>  I have a question regarding boosting the exact match queries to top,
> followed by partial match and if there is no exact match then give me
> partial match. The following 2 solutions have yielded different results, and
> I was not clear on it why
> 
>   This is the schema I have
> 
>   
>multiValued="true"/>
>   
>   
>   f1
> 
> positionIncrementGap="100">
>  
>
> generateWordParts="1" generateNumberParts="1" catenateWords="1"
> catenateNumbers="1" catenateAll="1" splitOnCaseChange="0"/>
> 
> words="./lang/stopwords_pt.txt" enablePositionIncrements="true"/>
>
> language="Portuguese"/>
>  
>
> 
> in my solrconfig.xml I have
>   f1
>   f1^10 f3^1
>   f1^10 f3^1
> 
> now if I try to specify the query with these parameters in solrconfig.xml,
> 99% of the time exactmatch first and then partial match 1%of the time the
> exact match result is in the index but does not show on the results and does
> not give any partial matches for that query either.
> 
>But if I make it qf=f3&pf=f1^10 f3^1 yields the exactmatch result on top
> 100% of the time.
> 
>   Why I am seeing this behavior.
> 
> is there anyway to say qf=f1 on the interface and get only exact results if
> present (in this case though f1 is string but the q parameter has spaces. do
> I need to use pf field
>   I am using dismax query parser.
> 
> Thanks
> 
> 
> 
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/string-field-does-not-yield-exact-match-result-using-qf-parameter-tp4060096.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: Master - Slave File Sizes are not Same even after "command=abortfetch"

2013-04-30 Thread Shawn Heisey 

On 4/30/2013 8:33 AM, Furkan KAMACI wrote:

I think that replication occurs after commit by default. It has been long
time however there is still mismatch between leader and replica
(approximately 5 MB). I tried to pull indexes from leader but it is still
same.


My mail server has been down most of the day, and the Apache mail 
infrastructure hasn't noticed yet that I'm back up.  I don't have copies 
of the newest messages on this thread.  I checked the web archive to see 
what else has been said.  I'll be repeating some of what has been said 
before.


On SolrCloud terminology: SolrCloud divides your index into one or more 
shards, each of which has a different piece of the index.  Each shard is 
made up of replicas.  One replica in each shard is designated leader. 
Note: a leader is still a replica, it is just the winner of the latest 
leader election.  Summary: shards, replicas, leader.


One term that you are using is "follower" ... this is not a valid 
SolrCloud term.  It might make sense to use this term for a replica that 
is not a leader, but I have never seen it used in anything official. 
Any replica can become leader, if the conditions are just right.


There are only two times that the leader replica has special 
significance - when you are indexing and when a replica starts 
operation, either as an existing replica that went down or as a new replica.


In SolrCloud, replication is *NOT* used when you index new data.  The 
*ONLY* time that replication happens in SolrCloud is when a replica is 
starts up, and even then it will only happen if the leader cannot figure 
out how to use its transaction log to sync the replica.


SolrCloud does distributed indexing.  This means that when an update 
comes in, SolrCloud determines which shard needs that update.  If the 
core that received the request is not the leader of that shard, the 
request is forwarded to the correct leader.  That leader will index the 
update and send it to all of the replicas for that shard, each of which 
will index the update independently.


Because each replica indexes independently, you can end up with 
different sizes.  The actual search results should be the same, although 
scoring can sometimes be a little bit different between replicas because 
deleted documents that exist in one replica but not another will 
contribute to the score.  SolrCloud does not attempt to keep the 
replicas absolutely identical, as long as they contain the same 
non-deleted documents.


Thanks,
Shawn

Re: How to Recovery Backup Snapshot at SolrCloud?

2013-04-30 Thread Timothy Potter 
I agree with Michael that you'll only ever need your backup if you
lose all nodes hosting a shard (leader + all other replicas), so the
tlog doesn't really factor in when recovering from backup.

The snapshot created by the replication handler is the index only and
it makes most sense in my mind to remove the tlog before firing up a
new node using the snapshot.

The way I see it, the backup would be needed if you lost all nodes
hosting a shard, then you need to 1) recover one node (the leader)
from backup, 2) re-index any documents that were indexed between the
time of your last snapshot and the time of the failure, then 3) bring
additional replicas online as needed. The additional replicas will
sync with the leader, snap pulling the entire index. You could do it
1,2,3 or 1,3,2.

Cheers,
Tim

On Tue, Apr 30, 2013 at 12:45 PM, Furkan KAMACI  wrote:
> I had index and tlog folder under my data folder. I have a snapshot folder
> too when I make backup. However what will I do next if I want to use
> backup, will I remove index and tlog folders and put just my snapshot
> folder? What folks do?
>
> 2013/4/30 Michael Della Bitta 
>
>> Presumably you'd only be restoring a backup in the face of a catastrophe.
>>
>> Yes, you'd need to stop the node. And the transaction logs may not be
>> useful in this case. You'd have trouble reconciling them with the version
>> of the index in your backup I would think.
>>
>> Anybody who knows more about this want to chime in?
>>
>>
>> Michael Della Bitta
>>
>> 
>> Appinions
>> 18 East 41st Street, 2nd Floor
>> New York, NY 10017-6271
>>
>> www.appinions.com
>>
>> Where Influence Isn’t a Game
>>
>>
>> On Tue, Apr 30, 2013 at 11:03 AM, Furkan KAMACI > >wrote:
>>
>> > Should I stop the node first? And what will happen to transaction logs?
>> > Should I backup it too?
>> >
>> > 2013/4/30 Michael Della Bitta 
>> >
>> > > That directory is the data directory for the core... you'd just swap it
>> > in.
>> > >
>> > >
>> > > Michael Della Bitta
>> > >
>> > > 
>> > > Appinions
>> > > 18 East 41st Street, 2nd Floor
>> > > New York, NY 10017-6271
>> > >
>> > > www.appinions.com
>> > >
>> > > Where Influence Isn’t a Game
>> > >
>> > >
>> > > On Tue, Apr 30, 2013 at 8:06 AM, Furkan KAMACI > > > >wrote:
>> > >
>> > > > Hi Folks;
>> > > >
>> > > > I can backup my indexes at SolrCloud via
>> > > > http://_master_host_:_port_/solr/replication?command=backup
>> > > > and it creates a file called snapshot. I know that I should pull that
>> > > > directory any other safe place (a backup store) However what should I
>> > do
>> > > to
>> > > > make a recovery from that backup file?
>> > > >
>> > >
>> >
>>

Re: How to recover from "Error opening new searcher" when machine crashed while indexing

2013-04-30 Thread Otis Gospodnetic 
Hi,

Try running the CheckIndex tool.

Otis
Solr & ElasticSearch Support
http://sematext.com/
On Apr 30, 2013 3:10 PM, "Utkarsh Sengar"  wrote:

> Solr 4.0 was indexing data and the machine crashed.
>
> Any suggestions on how to recover my index since I don't want to delete my
> data directory?
>
> When I try to start it again, I get this error:
> ERROR 12:01:46,493 Failed to load Solr core: xyz.index1
> ERROR 12:01:46,493 Cause:
> ERROR 12:01:46,494 Error opening new searcher
> org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.(SolrCore.java:701)
> at org.apache.solr.core.SolrCore.(SolrCore.java:564)
> at
>
> org.apache.solr.core.CassandraCoreContainer.load(CassandraCoreContainer.java:213)
> at
> com.datastax.bdp.plugin.SolrCorePlugin.activateImpl(SolrCorePlugin.java:66)
> at
>
> com.datastax.bdp.plugin.PluginManager$PluginInitializer.call(PluginManager.java:161)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1290)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1402)
> at org.apache.solr.core.SolrCore.(SolrCore.java:675)
> ... 9 more
> Caused by: org.apache.lucene.index.IndexNotFoundException: no segments*
> file found in NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@
> /media/SSD/data/solr.data/rlcatalogks.prodinfo/index
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@d7581b;
> maxCacheMB=48.0 maxMergeSizeMB=4.0): files: [_73ne_nrm.cfs,
> _73ng_Lucene40_0.tip, _73nh_nrm.cfs, _73ng_Lucene40_0.tim, _73nf.fnm,
> _73n5_Lucene40_0.frq, _73ne.fdt, _73nh.fdx, _73ne_nrm.cfe, _73ne.fdx,
> _73ne_Lucene40_0.tim, _73ne.si, _73ni.fnm, _73nh_Lucene40_0.prx,
> _73ni.fdt,
> _73n5.si, _73ne_Lucene40_0.tip, _73nf_Lucene40_0.frq,
> _73nf_Lucene40_0.prx,
> _73nf_nrm.cfe, _73ne_Lucene40_0.frq, _73ng_Lucene40_0.prx,
> _73nf_Lucene40_0.tip, _73n5.fdx, _73ng_Lucene40_0.frq, _73ng.fnm,
> _73ni.fdx, _73n5.fnm, _73nf_Lucene40_0.tim, _73ni.si, _73n5.fdt,
> _73nf_nrm.cfs, _73nh_nrm.cfe, _73ni_Lucene40_0.frq, _73ng.fdx,
> _73ne_Lucene40_0.prx, _73nh.fnm, _73nh_Lucene40_0.tip,
> _73nh_Lucene40_0.tim, _73nh.si, _73n5_Lucene40_0.tip,
> _73ni_Lucene40_0.prx,
> _73n5_Lucene40_0.tim, _73nf.si, _73ng_nrm.cfe, _73n5_Lucene40_0.prx,
> _392j_42f.del, _73ng.fdt, _73ng.si, _73ni_nrm.cfe, _73n5_nrm.cfe,
> _73ni_nrm.cfs, _73nf.fdx, _73ni_Lucene40_0.tip, _73n5_nrm.cfs,
> _73ni_Lucene40_0.tim, _73nf.fdt, _73ne.fnm, _73nh.fdt,
> _73nh_Lucene40_0.frq, _73ng_nrm.cfs]
>
>
> --
> Thanks,
> -Utkarsh
>

Re: Exact and Partial Matches

2013-04-30 Thread Otis Gospodnetic 
An alternative would be a custom SearchComponent that post-processes hits.

Otis
Solr & ElasticSearch Support
http://sematext.com/
On Apr 30, 2013 10:27 AM, "Sandeep Mestry"  wrote:

> Thanks Erick,
>
> I tried grouping and it appears to work okay. However, I will need to
> change the client to parse the output..
>
> &fq=title:(tree)&group=true&group.query=title:(trees) NOT
> title_ci:trees&group.query=title_ci:blair&group.sort=title_sort
> desc&sort=score desc,title_sort asc
>
> I used the actual query as the filter query so my scores will be 1 and then
> used 2 group queries - one which will give me exact matches and other that
> will give me partial minus exact matches.
> I have tried this with operators too and it seems to be doing the job I
> want, do you see any issue in this?
>
> Thanks again for your reply and by the way thanks for SOLR-4662.
>
> -S
>
>
> On 30 April 2013 15:06, Erick Erickson  wrote:
>
> > I don't think you can do that. You're essentially
> > trying to mix ordering of the result set. You
> > _might_ be able to kludge some of this with
> > grouping, but I doubt it.
> >
> > You'll need two queries I'd guess.
> >
> > Best
> > Erick
> >
> > On Mon, Apr 29, 2013 at 9:44 AM, Sandeep Mestry 
> > wrote:
> > > Dear Experts,
> > >
> > > I have a requirement for the exact matches and applying alphabetical
> > > sorting thereafter.
> > >
> > > To illustrate, the results should be sorted in exact matches and all
> > later
> > > alphabetical.
> > >
> > > So, if there are 5 documents as below
> > >
> > > Doc1
> > > title: trees
> > >
> > > Doc 2
> > > title: plum trees
> > >
> > > Doc 3
> > > title: Money Trees (Legendary Trees)
> > >
> > > Doc 4
> > > title: Cork Trees
> > >
> > > Doc 5
> > > title: Old Trees
> > >
> > > Then, if user searches with query term as 'trees', the results should
> be
> > in
> > > following order:
> > >
> > > Doc 1 trees - Highest Rank
> > > Doc 4 Cork Trees - Alphabetical afterwards..
> > > Doc 3 Money Trees (Legendary Trees)
> > > Doc 5 Old Trees
> > > Doc 2 plum trees
> > >
> > > I can achieve the alphabetical sorting by adding the title sort
> > > parameter, However,
> > > Solr relevancy is higher for Doc 3 (due to matches in 2 terms and so
> > > it arranges
> > > Doc 3 above Doc 4, 5 and 2).
> > > So, it looks like:
> > >
> > > Doc 1 trees - Highest Rank
> > > Doc 3 Money Trees (Legendary Trees)
> > > Doc 4 Cork Trees - Alphabetical afterwards..
> > > Doc 5 Old Trees
> > > Doc 2 plum trees
> > >
> > > Can you tell me an easy way to achieve this requirement please?
> > >
> > > I'm using Solr 4.0 and the *title *field is defined as follows:
> > >
> > >  > positionIncrementGap="100"
> > >>
> > > 
> > > 
> > >  > > stemEnglishPossessive="0" generateWordParts="1" generateNumberParts="1"
> > > catenateWords="1" catenateNumbers="1" catenateAll="1"
> > splitOnCaseChange="1"
> > > splitOnNumerics="0" preserveOriginal="1" />
> > > 
> > > 
> > > 
> > > 
> > >  > > stemEnglishPossessive="0" generateWordParts="1" generateNumberParts="1"
> > > catenateWords="1" catenateNumbers="1" catenateAll="1"
> > splitOnCaseChange="1"
> > > splitOnNumerics="0" preserveOriginal="1" />
> > > 
> > > 
> > > 
> > >
> > >
> > >
> > > Many Thanks in advance,
> > > Sandeep
> >
>

Re: Solr Range Queries with Field value

2013-04-30 Thread Indika Tantrigoda 
Yes, the SQL statement is what I am trying to achieve. As for the
merchant_end_of_day_in_utc_epoch, we map the time to start of epoch and
convert that to UTC, so that all the merchants are in the same timezone
which would make it easier to query for open ones.

For the use case when we need to determine if a merchant is currently open
now or in the future (within the same day). Therefore when converting the
start/end times to epoch and UTC a session (i.e. start time to end time)
might get spilled over past 12 midnight.

After some research the following syntax worked
start_time_utc_epoch:[1970-01-01T00:00:00Z TO
_val_:"merchant_end_of_day_in_utc_epoch"])

Thanks,
Indika



On 1 May 2013 00:52, Arun Rangarajan  wrote:

> Erick,
>
> I believe Indika wants to do this SQL WHERE clause in Solr:
> WHERE start_time_utc_epoch >= '1970-01-01T00:00:00Z' AND
> start_time_utc_epoch
> <= merchant_end_of_day_in_utc_epoch
>
>
> On Tue, Apr 30, 2013 at 11:49 AM, Erick Erickson  >wrote:
>
> > Could you define your use-case in some more detail? On the
> > surface, this query doesn't really make a lot of sense. How
> > would merchant_end_of_day_in_utc_epoch be determined?
> > Presumably there are zillions of values across your index for
> > this value, depending on the document. Which one should be
> > used?
> >
> > Best
> > Erick
> >
> > On Mon, Apr 29, 2013 at 8:34 PM, Indika Tantrigoda 
> > wrote:
> > > Hi All,
> > >
> > > I'd like to know if its possible to use a field value in a Solr range
> > query
> > > ? Something similar to start_time_utc_epoch:[1970-01-01T00:00:00Z TO
> > > merchant_end_of_day_in_utc_epoch])
> > >
> > > merchant_end_of_day_in_utc_epoch is an indexed field.
> > >
> > > I am using Solr 4.
> > >
> > > Thanks,
> > > Indika
> >
>

Re: Maximum number of facet query ina single query

2013-04-30 Thread Otis Gospodnetic 
FWIW, one of our current clients runs queries with 6000 facet queries...

Otis
Solr & ElasticSearch Support
http://sematext.com/


On Apr 30, 2013 5:22 AM, "vicky desai"  wrote:

> Hi,
>
> Is there any upper limit on the number of facet queries I can include in a
> single query. Also is there any performance hit if I include too many facet
> queries in a single query
>
> Any help would be appreciated
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Maximum-number-of-facet-query-ina-single-query-tp4059926.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Issue with fuzzy search in Distributed Search

2013-04-30 Thread meghana 
To ensure the all records exist in single node, i queried on specific
duration, so , for shards core and simple core query, results should be
similar. 

as you suggested, i analyzed the debugQuery for one specific search
*text:worde~1*, and I seen that the record which returns in shards core have
highlights like *word*, *words*, *word!n*. but when I look in debugQuery it
just processing for *word!n*, and was not processing  other highlights
(words, word), although it shows it in highlight for that record. and so,
shards core do not return other records , having text as *word* or *words* ,
but not *word!n* in it. 

on the other case, the simple core processing all *word*, *words*, *word!n*,
and return proper results.  this seems very weird behavior, any suggestion ? 



Jack Krupansky-2 wrote
> A fuzzy query itself does not know about distributed search - Lucene
> simply 
> scores the query results based on the local index. Then, Solr is merging
> the 
> merging the query results from different nodes.
> 
> Try the query locally for each node and set debugQuery=true and see how
> each 
> document gets scored.
> 
> I'm actually not sure what the specific "problem" (symptom) is that you
> are 
> seeing. I mean, maybe there is only 1 result on that node - how do you
> know 
> otherwise?? Or maybe one node has more exact matches.
> 
> -- Jack Krupansky
> 
> -Original Message- 
> From: meghana
> Sent: Tuesday, April 30, 2013 7:51 AM
> To: 

> solr-user@.apache

> Subject: Issue with fuzzy search in Distributed Search
> 
> I have created 2 versions of Solr core in different servers. one is simple
> core having all records in one core. And other is shards core, distributed
> over 3 cores on server.
> 
> Simple core :
> 
> http://localhost:8080/sorl/core0/select?q=text:hoers~1
> 
> Distributed core :
> 
> http://192.168.1.91:8080/core0/select?shards=http://192.168.1.91:8080/core0,http://192.168.1.91:8080/core1,http://192.168.1.91:8080/core2&q=text:hoers~1
> 
> data, schema and other configuration is similar in both the cores.
> 
> but while doing fuzzy search like hoers~1 one core returns many
> records(about 450), while other core return only 1 record.
> 
> While this issue does not seem related to Distributed Search, as Although
> i
> do not use distributed search, then also it do not return more rows.
> 
> as http://192.168.1.91:8080/core0/select?q=text:hoers~1
> 
> below is schema definition for my field.
>  positionIncrementGap="100" autoGeneratePhraseQueries="true">
>   
> 
>   
> 
> 
>  ignoreCase="true"
> words="stopwords.txt"
> enablePositionIncrements="false"
> />
> 
>  ignoreCase="true"
> words="stopwords_en.txt"
> enablePositionIncrements="true"
> />
> 
>  generateWordParts="1" generateNumberParts="1" catenateWords="1"
> catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"
> protected="protwords.txt" types="wdfftypes.txt"  />
> 
> 
> 
>  protected="protwords.txt"/>
> 
> 
>   
> 
>   
> 
> 
> 
> 
>  ignoreCase="true" expand="true"/>
> 
>  ignoreCase="true"
> words="stopwords_extra_query.txt"
> enablePositionIncrements="false"
> />
> 
>  ignoreCase="true"
> words="stopwords_en.txt"
> enablePositionIncrements="true"
> />
> 
>  generateWordParts="1" generateNumberParts="1" catenateWords="0"
> catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"
> protected="protwords.txt" types="wdfftypes.txt"  />
> 
> 
> 
>  protected="protwords.txt"/>
> 
> 
>   
> 
> 
> 
> Not sure, what is wrong with this. Can anybody help me on this??
> 
> 
> 
> 
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Issue-with-fuzzy-search-in-Distributed-Search-tp4060022.html
> Sent from the Solr - User mailing list archive at Nabble.com.





--
View this message in context: 
http://lucene.472066.n3.nabble.com/Solr-Results-differ-in-2-solr-cores-same-configuration-for-fuzzy-search-tp4060022p4060201.html
Sent from the Solr - User mailing list archive at Nabble.com.