Re: import solr source to eclipse
Configure eclipse with Jetty plugin. Create a Solr folder under your Solr-Java-Project and Run the project [Run as] on Jetty Server. This blog[1] may help you to configure Solr within eclipse. [1] http://hokiesuns.blogspot.in/2010/01/setting-up-apache-solr-in-eclipse.html On Tue, Oct 14, 2014 at 12:06 PM, Ali Nazemian wrote: > Thank you very much for your guides but how can I run solr server inside > eclipse? > Best regards. > > On Mon, Oct 13, 2014 at 8:02 PM, Rajani Maski > wrote: > > > Hi, > > > > The best tutorial for setting up Solr[solr 4.7] in eclipse/intellij is > > documented in Solr In Action book, Apendix A, *Working with the Solr > > codebase* > > > > > > On Mon, Oct 13, 2014 at 6:45 AM, Tomás Fernández Löbbe < > > tomasflo...@gmail.com> wrote: > > > > > The way I do this: > > > From a terminal: > > > svn checkout https://svn.apache.org/repos/asf/lucene/dev/trunk/ > > > lucene-solr-trunk > > > cd lucene-solr-trunk > > > ant eclipse > > > > > > ... And then, from your Eclipse "import existing java project", and > > select > > > the directory where you placed lucene-solr-trunk > > > > > > On Sun, Oct 12, 2014 at 7:09 AM, Ali Nazemian > > > wrote: > > > > > > > Hi, > > > > I am going to import solr source code to eclipse for some development > > > > purpose. Unfortunately every tutorial that I found for this purpose > is > > > > outdated and did not work. So would you please give me some hint > about > > > how > > > > can I import solr source code to eclipse? > > > > Thank you very much. > > > > > > > > -- > > > > A.Nazemian > > > > > > > > > > > > > -- > A.Nazemian >
Need to reindex when changing schema.xml?
Hello I hope this question has not been asked many times. I did some research but I never could find clearly answered anywhere. We have several multivalue fields on a instance with millions of documents which only contain a single value. I would like to change it to multivalue=false to be able to use grouping and stats on those fields. My question is: Can I do it without reindexing? Is there any document that says when rebuilding the index is needed versus when it is not needed? Thanks a lot
Re: import solr source to eclipse
Another alternative is launch the jetty server from outside and attach it remotely from eclipse. java -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=7666 -jar start.jar The above command waits until the application attach succeed. On Tue, Oct 14, 2014 at 12:56 PM, Rajani Maski wrote: > Configure eclipse with Jetty plugin. Create a Solr folder under your > Solr-Java-Project and Run the project [Run as] on Jetty Server. > > This blog[1] may help you to configure Solr within eclipse. > > > [1] > http://hokiesuns.blogspot.in/2010/01/setting-up-apache-solr-in-eclipse.html > > On Tue, Oct 14, 2014 at 12:06 PM, Ali Nazemian > wrote: > > > Thank you very much for your guides but how can I run solr server inside > > eclipse? > > Best regards. > > > > On Mon, Oct 13, 2014 at 8:02 PM, Rajani Maski > > wrote: > > > > > Hi, > > > > > > The best tutorial for setting up Solr[solr 4.7] in eclipse/intellij is > > > documented in Solr In Action book, Apendix A, *Working with the Solr > > > codebase* > > > > > > > > > On Mon, Oct 13, 2014 at 6:45 AM, Tomás Fernández Löbbe < > > > tomasflo...@gmail.com> wrote: > > > > > > > The way I do this: > > > > From a terminal: > > > > svn checkout https://svn.apache.org/repos/asf/lucene/dev/trunk/ > > > > lucene-solr-trunk > > > > cd lucene-solr-trunk > > > > ant eclipse > > > > > > > > ... And then, from your Eclipse "import existing java project", and > > > select > > > > the directory where you placed lucene-solr-trunk > > > > > > > > On Sun, Oct 12, 2014 at 7:09 AM, Ali Nazemian > > > > > wrote: > > > > > > > > > Hi, > > > > > I am going to import solr source code to eclipse for some > development > > > > > purpose. Unfortunately every tutorial that I found for this purpose > > is > > > > > outdated and did not work. So would you please give me some hint > > about > > > > how > > > > > can I import solr source code to eclipse? > > > > > Thank you very much. > > > > > > > > > > -- > > > > > A.Nazemian > > > > > > > > > > > > > > > > > > > > -- > > A.Nazemian > > >
mark solr documents as duplicates on hashing the combination of some fields
Dear all, Hi, I was wondering how can I mark some documents as duplicate (just marking for future usage not deleting) based on the hash combination of some fields? Suppose I have 2 fields name "url" and "title" I want to create hash based on url+title and send it to another field name "signature". If I do that using solr dedup, it will be resulted to deleting duplicate documents! So it is not applicable for my situation. Thank you very much. Best regards. -- A.Nazemian
Re: Need to reindex when changing schema.xml?
You should be able to change it without re-indexing, unless you've enabled docValues on that field. AFAIK docValues are the only persistent data structure that is different for single-valued versus multi-valued, everything else (UninvertedFields, etc) is built on the fly. I don't think there's any definitive reference on what requires a re-index, but that would be a nice thing to add to the Reference Guide Alan Woodward www.flax.co.uk On 14 Oct 2014, at 08:30, Roger Sindreu wrote: > Hello > > I hope this question has not been asked many times. I did some research but > I never could find clearly answered anywhere. > > We have several multivalue fields on a instance with millions of documents > which only contain a single value. I would like to change it to > multivalue=false to be able to use grouping and stats on those fields. > > My question is: Can I do it without reindexing? > > Is there any document that says when rebuilding the index is needed versus > when it is not needed? > > Thanks a lot
Re: best load balancer for solr cloud
Thanks Shawn, Amey, Any specific configuration needed for CloudSolrServer as I've seen increased latency on using it. Does ConcurrentUpdateSolrServer itself do discovery like CloudSolrServer. On Mon, Oct 13, 2014 at 7:53 PM, Shawn Heisey wrote: > On 10/13/2014 5:28 AM, Apoorva Gaurav wrote: > > Is it preferable to use CloudSolrServer or using an external load > balancer > > like haproxy. We're currently channeling all our requests via haproxy but > > want to get rid of management hassles as well as additional network call > > but saw a significant degradation in latency on switching to > > CloudSolrServer. Please suggest. > > If your client is Java, then there's no contest. Use CloudSolrServer. > It can react almost instantly to cluster changes. A load balancer will > need to do a health-check cycle before it knows about machines coming up > or going down. > > The other reply that you received mentioned ConcurrentUpdateSolrServer. > This is a high-performance option, but it comes at a cost -- your > application will never be informed about any indexing errors. Even if > the index requests all fail, your application will never know. > > Thanks, > Shawn > > -- Thanks & Regards, Apoorva
having Solr deduplication and partial update
Hi, I was wondering how can I have both solr deduplication and partial update. I found out that due to some reasons you can not rely on solr deduplication when you try to update a document partially! It seems that when you do partial update on some field- even if that field does not consider as duplication field- solr signature created by deduplication will be inapplicable! Is there anyway I can have both deduplication and partial update? Thank you very much. -- A.Nazemian
Re: SolrCloud: Meaning of SYNC state in ZkStateReader?
Ok, thanks for your response, Mark! Cheers, Martin On Tue, Oct 14, 2014 at 1:59 AM, Mark Miller wrote: > I think it's just cruft I left in and never ended up using anywhere. You > can ignore it. > > - Mark > > > On Oct 13, 2014, at 8:42 PM, Martin Grotzke < > martin.grot...@googlemail.com> wrote: > > > > Hi, > > > > can anybody tell me the meaning of ZkStateReader.SYNC? All other state > > related constants are clear to me, I'm only not sure about the semantics > > of SYNC. > > > > Background: I'm working on an async solr client > > (https://github.com/inoio/solrs) and want to add SolrCloud support - for > > this I'm reusing ZkStateReader. > > > > TIA && cheers, > > Martin > > > -- Martin Grotzke http://twitter.com/martin_grotzke
Re: SOLR Boolean clause impact on memory/Performance
Thanks Erick for responding. We have assigned 4GB memory for SOLR server and at high load where queries are having more than 10K boolean clauses, combination of cache and high boolean clauses are causing system to break. The system was working fine for last 8 months but ofcourse the boolean clauses has increased over time which I believe has caused the system to break and thats why I am looking for some numbers which can tell me how much memory will solr take to process say 1K boolean clauses in the query. The requirement at our end does required such huge number of boolean clauses. We need to present the search results to which user is entitled to. The entitlement is logic is dependent upon multiple packages. for example , user has entitlement to package A and B so we need to present search results in case the results have tag of package A or Package B. These packages have grown over time and seems to be causing issues. Thanks, Ankit On Mon, Oct 13, 2014 at 5:53 PM, Erick Erickson wrote: > Of course there will be performance and memory changes. The only > real question is whether your situation can tolerate them. The whole > point of maxBooleanClauses is exactly that going above that limit > should be a conscious decision because it has implications for > both memory and performance > > That said, that limit was put in there quite some time ago and > things are much faster now. I've seen installation where this limit is > raised over 10K. > > Are you sure this is the best approach though? Could joins > work here? Or reranking? (this last is doubtful, but...). > > This may well be an XY problem, you haven't explained _why_ > you need so many conditions which might enable other > suggestions. > > Best, > Erick > > On Mon, Oct 13, 2014 at 9:10 AM, ankit gupta > wrote: > > hi, > > > > Can we quantify the impact on SOLR memory usage/performance if we > increase > > the boolean clause. I am currently using lot of OR clauses in the query > > (close to 10K) and can see heap size growing. > > > > Thanks, > > Ankit >
Solr FilterCache size
I am planning to use FilterCache in my solr setup. Want to know few things regarding Filtercache 1. If my solrconfig.xml doesn't contain any cache configuration , basically I have removed all the cache properties from solrconfig.xml. In this case still caches will be used or not? if yes what is default value of it? 2. How to decide correct values size and initialSize of cache, what parameters needs to be considered in this case? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-FilterCache-size-tp4164161.html Sent from the Solr - User mailing list archive at Nabble.com.
Recovering from Out of Mem
I know there are some suggestions to avoid OOM issue e.g. setting appropriate Max Heap size etc. However, what's the best way to recover from it as it goes into non-responding state? We are using Tomcat on back end. The scenario is that once we face OOM issue it keeps on taking queries (doesn't give any error) but they just time out. So even though we have a fail over system implemented but we don't have a way to distinguish if these are real time out queries OR due to OOM. -- Regards, Salman Akram
numfound in solr
Hi I ran indexing and my total rows fetched is 1736789 but my numfound in solr indexed url 638169. What could be the possible reason?? -- View this message in context: http://lucene.472066.n3.nabble.com/numfound-in-solr-tp4164169.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr ExtractingRequestHandler - Internal server Error
Hi, I am trying to index rich documents with ExtractingRequestHandler. So for configuration I have in solrconfig.xml (I put all the jar of contrib/extraction/lib in solr/lib) And - text true ignored_ true links ignored_ But when i launch curl "http://localhost:8080/solr/update/extract?literal.id=doc1&commit=true"; -F "myfile:@file.html" I have the following error java.lang.AbstractMethodError at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBodyContentStreamHandlerBase The server encountered an internal error that prevent it from fulfilling this request. Do you have an idea why I get this error? Thanks ! -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-ExtractingRequestHandler-Internal-server-Error-tp4164174.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Recovering from Out of Mem
Best is to pass the Java cmd line option that kills the process on OOM and setup a supervisor on the process to restart it. You need a somewhat recent release for this to work properly though. - Mark > On Oct 14, 2014, at 9:06 AM, Salman Akram > wrote: > > I know there are some suggestions to avoid OOM issue e.g. setting > appropriate Max Heap size etc. However, what's the best way to recover from > it as it goes into non-responding state? We are using Tomcat on back end. > > The scenario is that once we face OOM issue it keeps on taking queries > (doesn't give any error) but they just time out. So even though we have a > fail over system implemented but we don't have a way to distinguish if > these are real time out queries OR due to OOM. > > -- > Regards, > > Salman Akram
Re: Recovering from Out of Mem
a really simple approach is to have the OOM generate an email e.g. 1) create a simple script (call it java_oom.sh) and drop it in your tomcat bin dir echo `date` | mail -s "Java Error: OutOfMemory - $HOSTNAME" not...@domain.com 2) configure your java options (in setenv.sh or similar) to trigger heap dump and the email script when OOM occurs # config error behaviors CATALINA_OPTS="$CATALINA_OPTS -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=$TOMCAT_DIR/temp/tomcat-dump.hprof -XX:OnError=$TOMCAT_DIR/bin/java_error.sh -XX:OnOutOfMemoryError=$TOMCAT_DIR/bin/java_oom.sh -XX:ErrorFile=$TOMCAT_DIR/temp/java_error%p.log" From: Mark Miller Sent: Tuesday, October 14, 2014 06:30 To: solr-user@lucene.apache.org Subject: Re: Recovering from Out of Mem Best is to pass the Java cmd line option that kills the process on OOM and setup a supervisor on the process to restart it. You need a somewhat recent release for this to work properly though. - Mark > On Oct 14, 2014, at 9:06 AM, Salman Akram > wrote: > > I know there are some suggestions to avoid OOM issue e.g. setting > appropriate Max Heap size etc. However, what's the best way to recover from > it as it goes into non-responding state? We are using Tomcat on back end. > > The scenario is that once we face OOM issue it keeps on taking queries > (doesn't give any error) but they just time out. So even though we have a > fail over system implemented but we don't have a way to distinguish if > these are real time out queries OR due to OOM. > > -- > Regards, > > Salman Akram
Re: Recovering from Out of Mem
And don't forget to set the proper permissions on the script, the tomcat or jetty user. Markus On Tuesday 14 October 2014 13:47:47 Boogie Shafer wrote: > a really simple approach is to have the OOM generate an email > > e.g. > > 1) create a simple script (call it java_oom.sh) and drop it in your tomcat > bin dir > > > echo `date` | mail -s "Java Error: OutOfMemory - $HOSTNAME" > not...@domain.com > > > 2) configure your java options (in setenv.sh or similar) to trigger heap > dump and the email script when OOM occurs > > # config error behaviors > CATALINA_OPTS="$CATALINA_OPTS -XX:+HeapDumpOnOutOfMemoryError > -XX:HeapDumpPath=$TOMCAT_DIR/temp/tomcat-dump.hprof > -XX:OnError=$TOMCAT_DIR/bin/java_error.sh > -XX:OnOutOfMemoryError=$TOMCAT_DIR/bin/java_oom.sh > -XX:ErrorFile=$TOMCAT_DIR/temp/java_error%p.log" > > > > > From: Mark Miller > Sent: Tuesday, October 14, 2014 06:30 > To: solr-user@lucene.apache.org > Subject: Re: Recovering from Out of Mem > > Best is to pass the Java cmd line option that kills the process on OOM and > setup a supervisor on the process to restart it. You need a somewhat > recent release for this to work properly though. > > - Mark > > > On Oct 14, 2014, at 9:06 AM, Salman Akram > > wrote: > > > > I know there are some suggestions to avoid OOM issue e.g. setting > > appropriate Max Heap size etc. However, what's the best way to recover > > from > > it as it goes into non-responding state? We are using Tomcat on back end. > > > > The scenario is that once we face OOM issue it keeps on taking queries > > (doesn't give any error) but they just time out. So even though we have a > > fail over system implemented but we don't have a way to distinguish if > > these are real time out queries OR due to OOM. > > > > -- > > Regards, > > > > Salman Akram
Re: numfound in solr
Your unique ID field is not unique most likely. Check what Solr thinks your unique ID field is in the schema.xml and then check whether that matches your expectations and your DB's unique criteria. The other option is if you use explicit commits only and did not commit at the end (less likely). Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 14 October 2014 08:08, neha sinha wrote: > Hi > > I ran indexing and my total rows fetched is 1736789 but my numfound in solr > indexed url 638169. > > > What could be the possible reason?? > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/numfound-in-solr-tp4164169.html > Sent from the Solr - User mailing list archive at Nabble.com.
Re: Recovering from Out of Mem
Boogie, Any example for java_error.sh script? — /Yago Riveiro On Tue, Oct 14, 2014 at 2:48 PM, Boogie Shafer wrote: > a really simple approach is to have the OOM generate an email > e.g. > 1) create a simple script (call it java_oom.sh) and drop it in your tomcat > bin dir > echo `date` | mail -s "Java Error: OutOfMemory - $HOSTNAME" not...@domain.com > 2) configure your java options (in setenv.sh or similar) to trigger heap dump > and the email script when OOM occurs > # config error behaviors > CATALINA_OPTS="$CATALINA_OPTS -XX:+HeapDumpOnOutOfMemoryError > -XX:HeapDumpPath=$TOMCAT_DIR/temp/tomcat-dump.hprof > -XX:OnError=$TOMCAT_DIR/bin/java_error.sh > -XX:OnOutOfMemoryError=$TOMCAT_DIR/bin/java_oom.sh > -XX:ErrorFile=$TOMCAT_DIR/temp/java_error%p.log" > > From: Mark Miller > Sent: Tuesday, October 14, 2014 06:30 > To: solr-user@lucene.apache.org > Subject: Re: Recovering from Out of Mem > Best is to pass the Java cmd line option that kills the process on OOM and > setup a supervisor on the process to restart it. You need a somewhat recent > release for this to work properly though. > - Mark >> On Oct 14, 2014, at 9:06 AM, Salman Akram >> wrote: >> >> I know there are some suggestions to avoid OOM issue e.g. setting >> appropriate Max Heap size etc. However, what's the best way to recover from >> it as it goes into non-responding state? We are using Tomcat on back end. >> >> The scenario is that once we face OOM issue it keeps on taking queries >> (doesn't give any error) but they just time out. So even though we have a >> fail over system implemented but we don't have a way to distinguish if >> these are real time out queries OR due to OOM. >> >> -- >> Regards, >> >> Salman Akram
Re: Need to reindex when changing schema.xml?
On 14 October 2014 04:40, Alan Woodward wrote: > I don't think there's any definitive reference on what requires a re-index, > but that would be a nice thing to add to the Reference Guide +1 That's a really good suggestion. Even a minimal page could be a good place we could add comments to as things are discovered. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853
Re: Recovering from Out of Mem
This will do:
kill -9 `ps aux | grep -v grep | grep tomcat6 | awk '{print $2}'`
pkill should also work
On Tuesday 14 October 2014 07:02:03 Yago Riveiro wrote:
> Boogie,
>
>
>
>
> Any example for java_error.sh script?
>
>
> —
> /Yago Riveiro
>
> On Tue, Oct 14, 2014 at 2:48 PM, Boogie Shafer
>
> wrote:
> > a really simple approach is to have the OOM generate an email
> > e.g.
> > 1) create a simple script (call it java_oom.sh) and drop it in your tomcat
> > bin dir echo `date` | mail -s "Java Error: OutOfMemory - $HOSTNAME"
> > not...@domain.com 2) configure your java options (in setenv.sh or
> > similar) to trigger heap dump and the email script when OOM occurs #
> > config error behaviors
> > CATALINA_OPTS="$CATALINA_OPTS -XX:+HeapDumpOnOutOfMemoryError
> > -XX:HeapDumpPath=$TOMCAT_DIR/temp/tomcat-dump.hprof
> > -XX:OnError=$TOMCAT_DIR/bin/java_error.sh
> > -XX:OnOutOfMemoryError=$TOMCAT_DIR/bin/java_oom.sh
> > -XX:ErrorFile=$TOMCAT_DIR/temp/java_error%p.log"
> >
> > From: Mark Miller
> > Sent: Tuesday, October 14, 2014 06:30
> > To: solr-user@lucene.apache.org
> > Subject: Re: Recovering from Out of Mem
> > Best is to pass the Java cmd line option that kills the process on OOM and
> > setup a supervisor on the process to restart it. You need a somewhat
> > recent release for this to work properly though. - Mark
> >
> >> On Oct 14, 2014, at 9:06 AM, Salman Akram
> >> wrote:
> >>
> >> I know there are some suggestions to avoid OOM issue e.g. setting
> >> appropriate Max Heap size etc. However, what's the best way to recover
> >> from
> >> it as it goes into non-responding state? We are using Tomcat on back end.
> >>
> >> The scenario is that once we face OOM issue it keeps on taking queries
> >> (doesn't give any error) but they just time out. So even though we have a
> >> fail over system implemented but we don't have a way to distinguish if
> >> these are real time out queries OR due to OOM.
> >>
> >> --
> >> Regards,
> >>
> >> Salman Akram
Re: SolrCloud 4.7 not doing distributed search when querying from a load balancer.
Try adding shards.info=true and debug=track to your queries ... these will
give more detailed information about what's going behind the scenes.
On Mon, Oct 13, 2014 at 11:11 PM, S.L wrote:
> Erick,
>
> I have upgraded to SolrCloud 4.10.1 with the same toplogy , 3 shards and 2
> replication factor with six cores altogether.
>
> Unfortunately , I still see the issue of intermittently no results being
> returned.I am not able to figure out whats going on here, I have included
> the logging information below.
>
> *Here's the query that I run.*
>
>
> http://server1.mydomain.com:8081/solr/dyCollection1/select/?q=*:*&fq=%28id:220a8dce-3b31-4d46-8386-da8405595c47%29&wt=json&distrib=true
>
>
>
> *Scenario 1: No result returned.*
>
> *Log Information for Scenario #1 .*
> 92860314 [http-bio-8081-exec-103] INFO
> org.apache.solr.handler.component.SpellCheckComponent –
>
> http://server1.mydomain.com:8082/solr/dyCollection1_shard3_replica2/|http://server2.mydomain.com:8082/solr/dyCollection1_shard3_replica1/
> null
> 92860315 [http-bio-8081-exec-103] INFO
> org.apache.solr.handler.component.SpellCheckComponent –
>
> http://server3.mydomain.com:8082/solr/dyCollection1_shard1_replica1/|http://server2.mydomain.com:8081/solr/dyCollection1_shard1_replica2/
> null
> 92860315 [http-bio-8081-exec-103] INFO
> org.apache.solr.handler.component.SpellCheckComponent –
>
> http://server1.mydomain.com:8081/solr/dyCollection1_shard2_replica1/|http://server3.mydomain.com:8081/solr/dyCollection1_shard2_replica2/
> null
> 92860315 [http-bio-8081-exec-103] INFO org.apache.solr.core.SolrCore –
> [dyCollection1_shard2_replica1] webapp=/solr path=/select/
>
> params={q=*:*&distrib=true&wt=json&fq=(id:220a8dce-3b31-4d46-8386-da8405595c47)}
> hits=0 status=0 QTime=5
>
> *Scenario #2 : I get result back*
>
>
>
> *Log information for scenario #2.*92881911 [http-bio-8081-exec-177] INFO
> org.apache.solr.core.SolrCore – [dyCollection1_shard2_replica1]
> webapp=/solr path=/select
>
> params={spellcheck=true&spellcheck.maxResultsForSuggest=5&spellcheck.extendedResults=true&spellcheck.collateExtendedResults=true&spellcheck.maxCollations=5&spellcheck.maxCollationTries=10&distrib=false&wt=javabin&spellcheck.collate=true&version=2&rows=10&NOW=1413251927427&shard.url=
>
> http://server1.mydomain.com:8081/solr/dyCollection1_shard2_replica1/|http://server3.mydomain.com:8081/solr/dyCollection1_shard2_replica2/&fl=productURL,score&df=suggestAggregate&start=0&q=*:*&spellcheck.dictionary=direct&spellcheck.dictionary=wordbreak&spellcheck.count=10&isShard=true&fsv=true&fq=(id:220a8dce-3b31-4d46-8386-da8405595c47)&spellcheck.alternativeTermCount=5
> }
> hits=1 status=0 QTime=1
> 92881913 [http-bio-8081-exec-177] INFO org.apache.solr.core.SolrCore –
> [dyCollection1_shard2_replica1] webapp=/solr path=/select
>
> params={spellcheck=false&spellcheck.maxResultsForSuggest=5&spellcheck.extendedResults=true&spellcheck.collateExtendedResults=true&ids=
>
> http://www.searcheddomain.com/p/ironwork-8-piece-comforter-set/-/A-15273248&spellcheck.maxCollations=5&spellcheck.maxCollationTries=10&distrib=false&wt=javabin&spellcheck.collate=true&version=2&rows=10&NOW=1413251927427&shard.url=http://server1.mydomain.com:8081/solr/dyCollection1_shard2_replica1/|http://server3.mydomain.com:8081/solr/dyCollection1_shard2_replica2/&df=suggestAggregate&q=*:*&spellcheck.dictionary=direct&spellcheck.dictionary=wordbreak&spellcheck.count=10&isShard=true&fq=(id:220a8dce-3b31-4d46-8386-da8405595c47)&spellcheck.alternativeTermCount=5
> }
> status=0 QTime=0
> 92881914 [http-bio-8081-exec-169] INFO
> org.apache.solr.handler.component.SpellCheckComponent –
>
> http://server1.mydomain.com:8082/solr/dyCollection1_shard3_replica2/|http://server2.mydomain.com:8082/solr/dyCollection1_shard3_replica1/
> null
> 92881914 [http-bio-8081-exec-169] INFO
> org.apache.solr.handler.component.SpellCheckComponent –
>
> http://server3.mydomain.com:8082/solr/dyCollection1_shard1_replica1/|http://server2.mydomain.com:8081/solr/dyCollection1_shard1_replica2/
> null
> 92881914 [http-bio-8081-exec-169] INFO
> org.apache.solr.handler.component.SpellCheckComponent –
>
> http://server1.mydomain.com:8081/solr/dyCollection1_shard2_replica1/|http://server3.mydomain.com:8081/solr/dyCollection1_shard2_replica2/
> null
> 92881914 [http-bio-8081-exec-169] INFO
> org.apache.solr.handler.component.SpellCheckComponent –
>
> http://server1.mydomain.com:8081/solr/dyCollection1_shard2_replica1/|http://server3.mydomain.com:8081/solr/dyCollection1_shard2_replica2/
> null
> 92881915 [http-bio-8081-exec-169] INFO org.apache.solr.core.SolrCore –
> [dyCollection1_shard2_replica1] webapp=/solr path=/select/
>
> params={q=*:*&distrib=true&wt=json&fq=(id:220a8dce-3b31-4d46-8386-da8405595c47)}
> hits=1 status=0 QTime=7
>
>
> *Autocommit and Soft commit settings.*
>
>
>${solr.autoSoftCommit.maxTime:-1}
>
>
>
>${solr.autoCommit.maxTime:15000}
>
>true
>
>
>
>
> On Tue, Oct 7, 2014 a
Re: numfound in solr
It would be nice to have a logging option to log updates vs. inserts, to help make it more obvious what is happening. And maybe even a way for a Solr update request to get back a summary of how many documents were inserted, updated, and deleted. -- Jack Krupansky -Original Message- From: Alexandre Rafalovitch Sent: Tuesday, October 14, 2014 9:59 AM To: solr-user Subject: Re: numfound in solr Your unique ID field is not unique most likely. Check what Solr thinks your unique ID field is in the schema.xml and then check whether that matches your expectations and your DB's unique criteria. The other option is if you use explicit commits only and did not commit at the end (less likely). Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 14 October 2014 08:08, neha sinha wrote: Hi I ran indexing and my total rows fetched is 1736789 but my numfound in solr indexed url 638169. What could be the possible reason?? -- View this message in context: http://lucene.472066.n3.nabble.com/numfound-in-solr-tp4164169.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Recovering from Out of Mem
jfyi - the bin/solr script does the following:
-XX:OnOutOfMemoryError="$SOLR_TIP/bin/oom_solr.sh $SOLR_PORT" where
$SOLR_PORT is the port Solr is bound to, e.g. 8983
The oom_solr.sh script looks like:
SOLR_PORT=$1
SOLR_PID=`ps waux | grep start.jar | grep $SOLR_PORT | grep -v grep | awk
'{print $2}' | sort -r`
if [ "$SOLR_PID" == "" ]; then
echo "Couldn't find Solr process running on port $SOLR_PORT!"
exit
fi
NOW=$(date +"%F%T")
(
echo "Running OOM killer script for process $SOLR_PID for Solr on port
$SOLR_PORT"
kill -9 $SOLR_PID
echo "Killed process $SOLR_PID"
) | tee solr_oom_killer-$SOLR_PORT-$NOW.log
I usually run Solr behind a supervisor type process (supervisord or
upstart) that will restart it if the process dies.
On Tue, Oct 14, 2014 at 8:09 AM, Markus Jelsma wrote:
> This will do:
> kill -9 `ps aux | grep -v grep | grep tomcat6 | awk '{print $2}'`
>
> pkill should also work
>
> On Tuesday 14 October 2014 07:02:03 Yago Riveiro wrote:
> > Boogie,
> >
> >
> >
> >
> > Any example for java_error.sh script?
> >
> >
> > —
> > /Yago Riveiro
> >
> > On Tue, Oct 14, 2014 at 2:48 PM, Boogie Shafer <
> boogie.sha...@proquest.com>
> >
> > wrote:
> > > a really simple approach is to have the OOM generate an email
> > > e.g.
> > > 1) create a simple script (call it java_oom.sh) and drop it in your
> tomcat
> > > bin dir echo `date` | mail -s "Java Error: OutOfMemory - $HOSTNAME"
> > > not...@domain.com 2) configure your java options (in setenv.sh or
> > > similar) to trigger heap dump and the email script when OOM occurs #
> > > config error behaviors
> > > CATALINA_OPTS="$CATALINA_OPTS -XX:+HeapDumpOnOutOfMemoryError
> > > -XX:HeapDumpPath=$TOMCAT_DIR/temp/tomcat-dump.hprof
> > > -XX:OnError=$TOMCAT_DIR/bin/java_error.sh
> > > -XX:OnOutOfMemoryError=$TOMCAT_DIR/bin/java_oom.sh
> > > -XX:ErrorFile=$TOMCAT_DIR/temp/java_error%p.log"
> > >
> > > From: Mark Miller
> > > Sent: Tuesday, October 14, 2014 06:30
> > > To: solr-user@lucene.apache.org
> > > Subject: Re: Recovering from Out of Mem
> > > Best is to pass the Java cmd line option that kills the process on OOM
> and
> > > setup a supervisor on the process to restart it. You need a somewhat
> > > recent release for this to work properly though. - Mark
> > >
> > >> On Oct 14, 2014, at 9:06 AM, Salman Akram
> > >> wrote:
> > >>
> > >> I know there are some suggestions to avoid OOM issue e.g. setting
> > >> appropriate Max Heap size etc. However, what's the best way to recover
> > >> from
> > >> it as it goes into non-responding state? We are using Tomcat on back
> end.
> > >>
> > >> The scenario is that once we face OOM issue it keeps on taking queries
> > >> (doesn't give any error) but they just time out. So even though we
> have a
> > >> fail over system implemented but we don't have a way to distinguish if
> > >> these are real time out queries OR due to OOM.
> > >>
> > >> --
> > >> Regards,
> > >>
> > >> Salman Akram
>
>
Re: import solr source to eclipse
I do exactly what Anurag mentioned, but _only_ when what I want to debug is, for some reason, not accessible via unit tests. It's very easy to do. It's usually much faster though to use unit tests, which you should be able to run from eclipse without starting a server at all. In IntelliJ, you just ctrl-click on the file and the menu gives you a choice of running or debugging the unit test, I'm sure Eclipse does something similar. There are zillions of units to choose from, and for new development it's a Good Thing to write the unit test first... Good luck! Erick On Tue, Oct 14, 2014 at 1:37 AM, Anurag Sharma wrote: > Another alternative is launch the jetty server from outside and attach it > remotely from eclipse. > > java -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=7666 > -jar start.jar > The above command waits until the application attach succeed. > > > On Tue, Oct 14, 2014 at 12:56 PM, Rajani Maski > wrote: > >> Configure eclipse with Jetty plugin. Create a Solr folder under your >> Solr-Java-Project and Run the project [Run as] on Jetty Server. >> >> This blog[1] may help you to configure Solr within eclipse. >> >> >> [1] >> http://hokiesuns.blogspot.in/2010/01/setting-up-apache-solr-in-eclipse.html >> >> On Tue, Oct 14, 2014 at 12:06 PM, Ali Nazemian >> wrote: >> >> > Thank you very much for your guides but how can I run solr server inside >> > eclipse? >> > Best regards. >> > >> > On Mon, Oct 13, 2014 at 8:02 PM, Rajani Maski >> > wrote: >> > >> > > Hi, >> > > >> > > The best tutorial for setting up Solr[solr 4.7] in eclipse/intellij is >> > > documented in Solr In Action book, Apendix A, *Working with the Solr >> > > codebase* >> > > >> > > >> > > On Mon, Oct 13, 2014 at 6:45 AM, Tomás Fernández Löbbe < >> > > tomasflo...@gmail.com> wrote: >> > > >> > > > The way I do this: >> > > > From a terminal: >> > > > svn checkout https://svn.apache.org/repos/asf/lucene/dev/trunk/ >> > > > lucene-solr-trunk >> > > > cd lucene-solr-trunk >> > > > ant eclipse >> > > > >> > > > ... And then, from your Eclipse "import existing java project", and >> > > select >> > > > the directory where you placed lucene-solr-trunk >> > > > >> > > > On Sun, Oct 12, 2014 at 7:09 AM, Ali Nazemian > > >> > > > wrote: >> > > > >> > > > > Hi, >> > > > > I am going to import solr source code to eclipse for some >> development >> > > > > purpose. Unfortunately every tutorial that I found for this purpose >> > is >> > > > > outdated and did not work. So would you please give me some hint >> > about >> > > > how >> > > > > can I import solr source code to eclipse? >> > > > > Thank you very much. >> > > > > >> > > > > -- >> > > > > A.Nazemian >> > > > > >> > > > >> > > >> > >> > >> > >> > -- >> > A.Nazemian >> > >>
Re: SOLR Boolean clause impact on memory/Performance
Then I predict they will continue to grow and whatever limit you put on maxBooleanClauses will be exceeded later. And so on, so I really think you need to re-think your model. One approach: 1> change your model so your users are assigned to a fixed number of groups. Then index group tokens with each document. You can index as many tokens in the _document_ as you want. Then your process looks like this: 1> user signs on, you go query the system-of-record for her groups. 2> each query from that user gets a filter query with their group tokens. The problem with this approach is if groups change, you have to re-index the affected documents. But it is fast. Essentially you exchange up-front work when indexing for _much_ less work at query time. Second approach: Use post-filters, see: http://lucidworks.com/blog/advanced-filter-caching-in-solr/ These were first created for the ACL problem. Best, Erick On Tue, Oct 14, 2014 at 4:31 AM, ankit gupta wrote: > Thanks Erick for responding. > > We have assigned 4GB memory for SOLR server and at high load where queries > are having more than 10K boolean clauses, combination of cache and high > boolean clauses are causing system to break. The system was working fine > for last 8 months but ofcourse the boolean clauses has increased over time > which I believe has caused the system to break and thats why I am looking > for some numbers which can tell me how much memory will solr take to > process say 1K boolean clauses in the query. > > The requirement at our end does required such huge number of boolean > clauses. We need to present the search results to which user is entitled > to. > > The entitlement is logic is dependent upon multiple packages. for example , > user has entitlement to package A and B so we need to present search > results in case the results have tag of package A or Package B. > > These packages have grown over time and seems to be causing issues. > > Thanks, > Ankit > > > > On Mon, Oct 13, 2014 at 5:53 PM, Erick Erickson > wrote: > >> Of course there will be performance and memory changes. The only >> real question is whether your situation can tolerate them. The whole >> point of maxBooleanClauses is exactly that going above that limit >> should be a conscious decision because it has implications for >> both memory and performance >> >> That said, that limit was put in there quite some time ago and >> things are much faster now. I've seen installation where this limit is >> raised over 10K. >> >> Are you sure this is the best approach though? Could joins >> work here? Or reranking? (this last is doubtful, but...). >> >> This may well be an XY problem, you haven't explained _why_ >> you need so many conditions which might enable other >> suggestions. >> >> Best, >> Erick >> >> On Mon, Oct 13, 2014 at 9:10 AM, ankit gupta >> wrote: >> > hi, >> > >> > Can we quantify the impact on SOLR memory usage/performance if we >> increase >> > the boolean clause. I am currently using lot of OR clauses in the query >> > (close to 10K) and can see heap size growing. >> > >> > Thanks, >> > Ankit >>
Re: Recovering from Out of Mem
yago,
you can put more complex restart logic as shown in the examples below or just
do something similar to the java_oom.sh i posted earlier where you just spit
out an email alert and deal with service restarts and troubleshooting manually
e.g. something like the following for a java_error.sh will drop an email with a
timestamp
echo `date` | mail -s "Java Error: General - $HOSTNAME" not...@domain.com
From: Tim Potter
Sent: Tuesday, October 14, 2014 07:35
To: solr-user@lucene.apache.org
Subject: Re: Recovering from Out of Mem
jfyi - the bin/solr script does the following:
-XX:OnOutOfMemoryError="$SOLR_TIP/bin/oom_solr.sh $SOLR_PORT" where
$SOLR_PORT is the port Solr is bound to, e.g. 8983
The oom_solr.sh script looks like:
SOLR_PORT=$1
SOLR_PID=`ps waux | grep start.jar | grep $SOLR_PORT | grep -v grep | awk
'{print $2}' | sort -r`
if [ "$SOLR_PID" == "" ]; then
echo "Couldn't find Solr process running on port $SOLR_PORT!"
exit
fi
NOW=$(date +"%F%T")
(
echo "Running OOM killer script for process $SOLR_PID for Solr on port
$SOLR_PORT"
kill -9 $SOLR_PID
echo "Killed process $SOLR_PID"
) | tee solr_oom_killer-$SOLR_PORT-$NOW.log
I usually run Solr behind a supervisor type process (supervisord or
upstart) that will restart it if the process dies.
On Tue, Oct 14, 2014 at 8:09 AM, Markus Jelsma wrote:
> This will do:
> kill -9 `ps aux | grep -v grep | grep tomcat6 | awk '{print $2}'`
>
> pkill should also work
>
> On Tuesday 14 October 2014 07:02:03 Yago Riveiro wrote:
> > Boogie,
> >
> >
> >
> >
> > Any example for java_error.sh script?
> >
> >
> > —
> > /Yago Riveiro
> >
> > On Tue, Oct 14, 2014 at 2:48 PM, Boogie Shafer <
> boogie.sha...@proquest.com>
> >
> > wrote:
> > > a really simple approach is to have the OOM generate an email
> > > e.g.
> > > 1) create a simple script (call it java_oom.sh) and drop it in your
> tomcat
> > > bin dir echo `date` | mail -s "Java Error: OutOfMemory - $HOSTNAME"
> > > not...@domain.com 2) configure your java options (in setenv.sh or
> > > similar) to trigger heap dump and the email script when OOM occurs #
> > > config error behaviors
> > > CATALINA_OPTS="$CATALINA_OPTS -XX:+HeapDumpOnOutOfMemoryError
> > > -XX:HeapDumpPath=$TOMCAT_DIR/temp/tomcat-dump.hprof
> > > -XX:OnError=$TOMCAT_DIR/bin/java_error.sh
> > > -XX:OnOutOfMemoryError=$TOMCAT_DIR/bin/java_oom.sh
> > > -XX:ErrorFile=$TOMCAT_DIR/temp/java_error%p.log"
> > >
> > > From: Mark Miller
> > > Sent: Tuesday, October 14, 2014 06:30
> > > To: solr-user@lucene.apache.org
> > > Subject: Re: Recovering from Out of Mem
> > > Best is to pass the Java cmd line option that kills the process on OOM
> and
> > > setup a supervisor on the process to restart it. You need a somewhat
> > > recent release for this to work properly though. - Mark
> > >
> > >> On Oct 14, 2014, at 9:06 AM, Salman Akram
> > >> wrote:
> > >>
> > >> I know there are some suggestions to avoid OOM issue e.g. setting
> > >> appropriate Max Heap size etc. However, what's the best way to recover
> > >> from
> > >> it as it goes into non-responding state? We are using Tomcat on back
> end.
> > >>
> > >> The scenario is that once we face OOM issue it keeps on taking queries
> > >> (doesn't give any error) but they just time out. So even though we
> have a
> > >> fail over system implemented but we don't have a way to distinguish if
> > >> these are real time out queries OR due to OOM.
> > >>
> > >> --
> > >> Regards,
> > >>
> > >> Salman Akram
>
>
Re: Solr FilterCache size
1> Uhmmm, why remove the settings then wonder what the defaults are? Just leave them in and you _know_. Otherwise please look at the code. 2> This is trial and error. The most important bit is the size parameter. There's little reason to make initialSize different from size, it's just pre-allocating what's essentially a map, takes up very little space anyway. Best, Erick On Tue, Oct 14, 2014 at 4:37 AM, nutchsolruser wrote: > I am planning to use FilterCache in my solr setup. Want to know few things > regarding Filtercache > 1. If my solrconfig.xml doesn't contain any cache configuration , basically > I have removed all the cache properties from solrconfig.xml. In this case > still caches will be used or not? if yes what is default value of it? > 2. How to decide correct values size and initialSize of cache, what > parameters needs to be considered in this case? > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/Solr-FilterCache-size-tp4164161.html > Sent from the Solr - User mailing list archive at Nabble.com.
Re: Need to reindex when changing schema.xml?
I'm going to inject a bit of caution here, since I've seen odd behaviors pop out in cases like this. But not, I admit, this particular case. I'd just go ahead and try it on a test index first. It's always possible there's a safety check somewhere that'll be tripped by this kind of change. Erick@ParanoiaRulesSoTest On Tue, Oct 14, 2014 at 7:02 AM, Alexandre Rafalovitch wrote: > On 14 October 2014 04:40, Alan Woodward wrote: >> I don't think there's any definitive reference on what requires a re-index, >> but that would be a nice thing to add to the Reference Guide > +1 > > That's a really good suggestion. Even a minimal page could be a good > place we could add comments to as things are discovered. > > Personal: http://www.outerthoughts.com/ and @arafalov > Solr resources and newsletter: http://www.solr-start.com/ and @solrstart > Solr popularizers community: https://www.linkedin.com/groups?gid=6713853
Re: numfound in solr
On 14 October 2014 10:35, Jack Krupansky wrote: > It would be nice to have a logging option to log updates vs. inserts, to > help make it more obvious what is happening. And maybe even a way for a Solr > update request to get back a summary of how many documents were inserted, > updated, and deleted. Is that something a custom UpdateRequestProcessor can easily calculate? Also, isn't there something in the Admin UI that will give that kind of statistics with the Freeze/Compare method? Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853
eDismax - boost function of multiple values
Hey everyone, I have a question about the boost function of solr. The documentation say about multiple function querys that I can write it seperated by whitespaces. Example: q=foo&bf=ord(popularity)^0.5 recip(rord(price),1,1000,1000)^0.3 Now I have two fields I like to boost. Inhabitants and importance. The field Inhabitants contain the inhabitants of citys. and the field importance contain a priority value - citys have the value 10, suburb the value 5 and streets the value 1. If I use the bf parameter I can boost inhabitants so that citys with the most inhabitants ranked up. Example: q=foo&bf=ord(inhabitants) The same happens if I boost importance. Example: q=foo&bf=ord(importance) But if I try to combine both so that importance and inhabitants ranked up only inhabitants will be ranked up and importance will be ignored. Example: q=foo&bf=ord(inhabitants) ord(importance) Knows anyone how I can fix this problem? greetings
Re: Facets for Child Documents?
Hi Joshua,
The functionality you are asking about is requested by
https://issues.apache.org/jira/browse/SOLR-5743.
I've prepared a patch with initial implementation and going to speak about
it on Lucene/Solr Revolution 2014 Conference, held in Washington, DC on
November 11-14, http://lucenerevolution.org/.
Please see 'Faceting with Lucene BlockJoinQuery' talk announcement in
Internals Sessions.
You are very welcome to join the conference and participate in discussion.
Your vote for https://issues.apache.org/jira/browse/SOLR-5743 would also be
match appreciated.
Thank you,
=
Dr Oleg Savrasov,
Community Coordinator,
Grid Dynamics Search team
2014-10-10 20:32 GMT+04:00 atawfik :
> Yes. One way is using a join query to link authors to books. The query will
> look like this:
>
> q={!join to=author_id_fk to=author_id} publication_date:[...]
>
>
> The other way is using grouping. Here, you first retrieved books based
> their
> publication then group them on their authors.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Facets-for-Child-Documents-tp4163592p4163751.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: best load balancer for solr cloud
On 10/14/2014 2:45 AM, Apoorva Gaurav wrote: > Any specific configuration needed for CloudSolrServer as I've seen > increased latency on using it. Does ConcurrentUpdateSolrServer itself do > discovery like CloudSolrServer. ConcurrentUpdateSolrServer is just a wrapper for HttpSolrServer that does simultaneous background indexing of multiple update requests with multiple threads. It has no idea what's happening in the SolrCloud clusterstate. CloudSolrServer is the only cloud-aware Solr client that I know about. I think someone is working on a cloud-aware client for another programming language besides Java, but I don't remember which language and I don't know what the status is. Thanks, Shawn
Re: eDismax - boost function of multiple values
Hi Jens, Where did you read that you can write it separated by white spaces? bq and bf are both can be defined multiple times. q=foo&bf=ord(inhabitants)bf=ord(importance) Ahmet On Tuesday, October 14, 2014 6:34 PM, Jens Mayer wrote: Hey everyone, I have a question about the boost function of solr. The documentation say about multiple function querys that I can write it seperated by whitespaces. Example: q=foo&bf=ord(popularity)^0.5 recip(rord(price),1,1000,1000)^0.3 Now I have two fields I like to boost. Inhabitants and importance. The field Inhabitants contain the inhabitants of citys. and the field importance contain a priority value - citys have the value 10, suburb the value 5 and streets the value 1. If I use the bf parameter I can boost inhabitants so that citys with the most inhabitants ranked up. Example: q=foo&bf=ord(inhabitants) The same happens if I boost importance. Example: q=foo&bf=ord(importance) But if I try to combine both so that importance and inhabitants ranked up only inhabitants will be ranked up and importance will be ignored. Example: q=foo&bf=ord(inhabitants) ord(importance) Knows anyone how I can fix this problem? greetings
Re: does one need to reindex when changing similarity class
thanks a lot for your answers! 2014-10-14 6:10 GMT+02:00 Jack Krupansky : > To correct myself, the selected Similarity class can have a computeNorm > method that calculates the "norm" value that will be stored in the index > when the document is indexed, so changing the Similarity class will require > reindexing if the implementation of the computeNorm method is different. > > -- Jack Krupansky > > -Original Message- From: Markus Jelsma > Sent: Monday, October 13, 2014 5:06 PM > > To: solr-user@lucene.apache.org > Subject: RE: does one need to reindex when changing similarity class > > Yes, if the replacing similarity has a different implementation on norms, > you should reindex or gradually update all documents within decent time. > > > > -Original message- > >> From:Ahmet Arslan >> Sent: Thursday 9th October 2014 18:27 >> To: solr-user@lucene.apache.org >> Subject: Re: does one need to reindex when changing similarity class >> >> How about SweetSpotSimilarity? Length norm is saved at index time? >> >> >> >> On Thursday, October 9, 2014 5:44 PM, Jack Krupansky < >> j...@basetechnology.com> wrote: >> The similarity class is only invoked at query time, so it doesn't >> participate in indexing. >> >> -- Jack Krupansky >> >> >> >> >> -Original Message- From: Markus Jelsma >> Sent: Thursday, October 9, 2014 6:59 AM >> To: solr-user@lucene.apache.org >> Subject: RE: does one need to reindex when changing similarity class >> >> Hi - no you don't have to, although maybe if you changed on how norms are >> encoded. >> Markus >> >> >> >> -Original message- >> > From:elisabeth benoit >> > Sent: Thursday 9th October 2014 12:26 >> > To: solr-user@lucene.apache.org >> > Subject: does one need to reindex when changing similarity class >> > >> > I've read somewhere that we do have to reindex when changing similarity >> > class. Is that right? >> > >> > Thanks again, >> > Elisabeth >> > >> >> >
solr trunk update.. build issues
Just wanted to check if anyone faced below issues and how they fixed: Solr launch issue --- >From command line ant compile, ant dist gives BUILD SUCCESSFUL generating \solr\dist\solr-6.0.0-SNAPSHOT.war. After copying and renaming the same file to example/start.jar on run shows below error $ java –jar start.jar Error: Could not find or load main class ▒jar Problems with eclipse project --- Also shows below problems in eclipse project: Description Resource Path Location Type Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/protobuf-java-2.5.0.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/solr-core-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/org.restlet.ext.servlet-2.1.1.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/org.restlet-2.1.1.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/wstx-asl-3.2.7.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/zookeeper-3.4.6.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/solr-solrj-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/spatial4j-0.4.1.jar' trunk Build path Build Path Problem The project cannot be built until build path errors are resolved trunk Unknown Java Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-analyzers-phonetic-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-analyzers-kuromoji-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-codecs-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-backward-codecs-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-expressions-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-core-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-highlighter-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-grouping-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-memory-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-join-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-queries-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-misc-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-spatial-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-queryparser-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/noggit-0.6.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-suggest-5.0.0-SNAPSHOT.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/commons-fileupload-1.2.1.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/commons-io-2.3.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/commons-lang-2.6.jar' trunk Build path Build Path Problem Project 'trunk' is missing required library: 'solr/example/solr-webapp/webapp/WEB-INF/lib/concurrentlinkedhashmap-lru-
Re: solr trunk update.. build issues
On Wed, Oct 15, 2014 at 12:38 AM, Anurag Sharma wrote: > Just wanted to check if anyone faced below issues and how they fixed: > > Solr launch issue > --- > From command line ant compile, ant dist gives BUILD SUCCESSFUL > generating \solr\dist\solr-6.0.0-SNAPSHOT.war. After copying and renaming > the same file to example/start.jar on run shows below error > $ java –jar start.jar > Error: Could not find or load main class ▒jar > > The start.jar contains the jetty bootstrap code. It is different from the war file and you shouldn't rename it to start.jar. You can use "ant example" from inside the solr directory to create a runnable example. You can also run "ant example" followed by the using the bin scripts to start Solr. See https://cwiki.apache.org/confluence/display/solr/Running+Solr > Problems with eclipse project > --- > Also shows below problems in eclipse project: > Description Resource Path Location Type > Project 'trunk' is missing required library: > 'solr/example/solr-webapp/webapp/WEB-INF/lib/protobuf-java-2.5.0.jar' > trunk Build > path Build Path Problem > Project 'trunk' is missing required library: > 'solr/example/solr-webapp/webapp/WEB-INF/lib/solr-core-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/org.restlet.ext.servlet-2.1.1.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > 'solr/example/solr-webapp/webapp/WEB-INF/lib/org.restlet-2.1.1.jar' trunk > Build > path Build Path Problem > Project 'trunk' is missing required library: > 'solr/example/solr-webapp/webapp/WEB-INF/lib/wstx-asl-3.2.7.jar' trunk > Build > path Build Path Problem > Project 'trunk' is missing required library: > 'solr/example/solr-webapp/webapp/WEB-INF/lib/zookeeper-3.4.6.jar' trunk > Build > path Build Path Problem > Project 'trunk' is missing required library: > 'solr/example/solr-webapp/webapp/WEB-INF/lib/solr-solrj-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > 'solr/example/solr-webapp/webapp/WEB-INF/lib/spatial4j-0.4.1.jar' trunk > Build > path Build Path Problem > The project cannot be built until build path errors are resolved trunk > Unknown Java Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-analyzers-phonetic-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-analyzers-kuromoji-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-codecs-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-backward-codecs-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-expressions-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-core-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-highlighter-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-grouping-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-memory-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-join-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-queries-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-misc-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-spatial-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-queryparser-5.0.0-SNAPSHOT.jar' > trunk Build path Build Path Problem > Project 'trunk' is missing required library: > 'solr/example/solr-webapp/webapp/WEB-INF/lib/noggit-0.6.jar' trunk Build > path Build Path Problem > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/luc
Re: solr trunk update.. build issues
Hi Shalin, Sorry about the typo, actually it's renaming the war file( \solr\dist\solr-6.0.0-SNAPSHOT.war) placed to \trunk\solr\example\webapps\solr.war as referred in last point of step7 in http://wiki.apache.org/solr/HowToCompileSolr I'll also refer and try https://cwiki.apache.org/confluence/display/solr/Running+Solr Regarding problem#2, on running "ant eclipse" at /trunk (top level directory for lucene & solr) and closing/opening the project removes all the problems in the project but eclipse is not able to show the modified file though it still exist. I need to explore more to find the root cause. Thanks for the quick response. Anurag On Wed, Oct 15, 2014 at 12:51 AM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > On Wed, Oct 15, 2014 at 12:38 AM, Anurag Sharma > wrote: > > > Just wanted to check if anyone faced below issues and how they fixed: > > > > Solr launch issue > > --- > > From command line ant compile, ant dist gives BUILD SUCCESSFUL > > generating \solr\dist\solr-6.0.0-SNAPSHOT.war. After copying and renaming > > the same file to example/start.jar on run shows below error > > $ java –jar start.jar > > Error: Could not find or load main class ▒jar > > > > > The start.jar contains the jetty bootstrap code. It is different from the > war file and you shouldn't rename it to start.jar. You can use "ant > example" from inside the solr directory to create a runnable example. You > can also run "ant example" followed by the using the bin scripts to start > Solr. See https://cwiki.apache.org/confluence/display/solr/Running+Solr > > > > Problems with eclipse project > > --- > > Also shows below problems in eclipse project: > > Description Resource Path Location Type > > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/protobuf-java-2.5.0.jar' > > trunk Build > > path Build Path Problem > > Project 'trunk' is missing required library: > > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/solr-core-5.0.0-SNAPSHOT.jar' > > trunk Build path Build Path Problem > > Project 'trunk' is missing required library: > > > > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/org.restlet.ext.servlet-2.1.1.jar' > > trunk Build path Build Path Problem > > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/org.restlet-2.1.1.jar' trunk > > Build > > path Build Path Problem > > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/wstx-asl-3.2.7.jar' trunk > > Build > > path Build Path Problem > > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/zookeeper-3.4.6.jar' trunk > > Build > > path Build Path Problem > > Project 'trunk' is missing required library: > > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/solr-solrj-5.0.0-SNAPSHOT.jar' > > trunk Build path Build Path Problem > > Project 'trunk' is missing required library: > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/spatial4j-0.4.1.jar' trunk > > Build > > path Build Path Problem > > The project cannot be built until build path errors are resolved trunk > > Unknown Java Problem > > Project 'trunk' is missing required library: > > > > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-analyzers-phonetic-5.0.0-SNAPSHOT.jar' > > trunk Build path Build Path Problem > > Project 'trunk' is missing required library: > > > > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-analyzers-kuromoji-5.0.0-SNAPSHOT.jar' > > trunk Build path Build Path Problem > > Project 'trunk' is missing required library: > > > > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-codecs-5.0.0-SNAPSHOT.jar' > > trunk Build path Build Path Problem > > Project 'trunk' is missing required library: > > > > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-backward-codecs-5.0.0-SNAPSHOT.jar' > > trunk Build path Build Path Problem > > Project 'trunk' is missing required library: > > > > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-expressions-5.0.0-SNAPSHOT.jar' > > trunk Build path Build Path Problem > > Project 'trunk' is missing required library: > > > > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-core-5.0.0-SNAPSHOT.jar' > > trunk Build path Build Path Problem > > Project 'trunk' is missing required library: > > > > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-highlighter-5.0.0-SNAPSHOT.jar' > > trunk Build path Build Path Problem > > Project 'trunk' is missing required library: > > > > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-grouping-5.0.0-SNAPSHOT.jar' > > trunk Build path Build Path Problem > > Project 'trunk' is missing required library: > > > > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-memory-5.0.0-SNAPSHOT.jar' > > trunk Build path Build Path Problem > > Project 'trunk' is missing required library: > > > > > 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-join-5.0.0-SNAPSHOT.jar' > >
RE: Result grouping using externalfilefield
Sudhaker,
Not sure if this has anything to do with your problem, but I had an issue with
grouping on non-string fields (in my case it was an integer) in SolrCloud
setup (4.7). But I was using internal fields. We worked around it by defining
the field as a string instead.
-Original Message-
From: Sudhakar Maddineni [mailto:maddineni...@gmail.com]
Sent: Friday, September 19, 2014 2:10 PM
To: solr-user@lucene.apache.org
Subject: Result grouping using externalfilefield
Hi,
Just trying to understand grouping feature with solrcloud 4.2 and I have 4
node/2 shard cluster setup.I am trying to group my search results based on an
externalfilefield that I defined.
Here is my configuration:
core1:
==
schema.xml:
solrconfig.xml:
And, created empty file with name external_core.txt in index dir.
core2:
===
same configuration as above except the default value=2
http://localhost:8983/solr/core1/select?q=te*&wt=json&fl=*,field(core)&indent=true&collection=core1,core2&df=default&group=true&group.field=core&group.limit=10
query results:
{
"responseHeader":{
"status":0,
"QTime":19,
"params":{
"df":"default",
"fl":"*,field(core)",
"indent":"true",
"q":"te*",
"group.limit":"10",
"group.field":"core",
"collection":"core1,core2",
"group":"true",
"wt":"json"}},
"grouped":[
"core",{
"matches":3,
"groups":[{
*"groupValue":null,*
"doclist":{"numFound":3,"start":0,"maxScore":0.20959431,"docs":[
{
"document_id":"_1",
*"field(core)":1.0},*
{
"document_id":"_1",
...
*"field(core)":0.0},*
{
"document_id":"_1",
..
*"field(core)":0.0}*]
}}]}]}
Issue:
Not getting query results grouped by "core" field which is an externalfilefield.
Can't we use EFF as a grouping field?If not, Is there an alternative using EFF
when grouping results.
Appreciate your help.
Thanks,Sudhakar.
Re: solr trunk update.. build issues
I am able to launch solr example start.jar using: java –jar start.jar The eclipse issue is also resolved now, looks like eclipse was taking time to process all the files and was an intermittent issue. Now am able to run the selected JUnits as well. On Wed, Oct 15, 2014 at 1:06 AM, Anurag Sharma wrote: > Hi Shalin, > > Sorry about the typo, actually it's renaming the war file( > \solr\dist\solr-6.0.0-SNAPSHOT.war) placed to > \trunk\solr\example\webapps\solr.war as referred in last point of step7 in > http://wiki.apache.org/solr/HowToCompileSolr > I'll also refer and try > https://cwiki.apache.org/confluence/display/solr/Running+Solr > > Regarding problem#2, on running "ant eclipse" at /trunk (top level > directory for lucene & solr) and closing/opening the project removes all > the problems in the project but eclipse is not able to show the modified > file though it still exist. I need to explore more to find the root cause. > > Thanks for the quick response. > Anurag > > > On Wed, Oct 15, 2014 at 12:51 AM, Shalin Shekhar Mangar < > shalinman...@gmail.com> wrote: > >> On Wed, Oct 15, 2014 at 12:38 AM, Anurag Sharma >> wrote: >> >> > Just wanted to check if anyone faced below issues and how they fixed: >> > >> > Solr launch issue >> > --- >> > From command line ant compile, ant dist gives BUILD SUCCESSFUL >> > generating \solr\dist\solr-6.0.0-SNAPSHOT.war. After copying and >> renaming >> > the same file to example/start.jar on run shows below error >> > $ java –jar start.jar >> > Error: Could not find or load main class ▒jar >> > >> > >> The start.jar contains the jetty bootstrap code. It is different from the >> war file and you shouldn't rename it to start.jar. You can use "ant >> example" from inside the solr directory to create a runnable example. You >> can also run "ant example" followed by the using the bin scripts to start >> Solr. See https://cwiki.apache.org/confluence/display/solr/Running+Solr >> >> >> > Problems with eclipse project >> > --- >> > Also shows below problems in eclipse project: >> > Description Resource Path Location Type >> > Project 'trunk' is missing required library: >> > 'solr/example/solr-webapp/webapp/WEB-INF/lib/protobuf-java-2.5.0.jar' >> > trunk Build >> > path Build Path Problem >> > Project 'trunk' is missing required library: >> > >> 'solr/example/solr-webapp/webapp/WEB-INF/lib/solr-core-5.0.0-SNAPSHOT.jar' >> > trunk Build path Build Path Problem >> > Project 'trunk' is missing required library: >> > >> > >> 'solr/example/solr-webapp/webapp/WEB-INF/lib/org.restlet.ext.servlet-2.1.1.jar' >> > trunk Build path Build Path Problem >> > Project 'trunk' is missing required library: >> > 'solr/example/solr-webapp/webapp/WEB-INF/lib/org.restlet-2.1.1.jar' >> trunk >> > Build >> > path Build Path Problem >> > Project 'trunk' is missing required library: >> > 'solr/example/solr-webapp/webapp/WEB-INF/lib/wstx-asl-3.2.7.jar' trunk >> > Build >> > path Build Path Problem >> > Project 'trunk' is missing required library: >> > 'solr/example/solr-webapp/webapp/WEB-INF/lib/zookeeper-3.4.6.jar' trunk >> > Build >> > path Build Path Problem >> > Project 'trunk' is missing required library: >> > >> 'solr/example/solr-webapp/webapp/WEB-INF/lib/solr-solrj-5.0.0-SNAPSHOT.jar' >> > trunk Build path Build Path Problem >> > Project 'trunk' is missing required library: >> > 'solr/example/solr-webapp/webapp/WEB-INF/lib/spatial4j-0.4.1.jar' trunk >> > Build >> > path Build Path Problem >> > The project cannot be built until build path errors are resolved trunk >> > Unknown Java Problem >> > Project 'trunk' is missing required library: >> > >> > >> 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-analyzers-phonetic-5.0.0-SNAPSHOT.jar' >> > trunk Build path Build Path Problem >> > Project 'trunk' is missing required library: >> > >> > >> 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-analyzers-kuromoji-5.0.0-SNAPSHOT.jar' >> > trunk Build path Build Path Problem >> > Project 'trunk' is missing required library: >> > >> > >> 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-codecs-5.0.0-SNAPSHOT.jar' >> > trunk Build path Build Path Problem >> > Project 'trunk' is missing required library: >> > >> > >> 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-backward-codecs-5.0.0-SNAPSHOT.jar' >> > trunk Build path Build Path Problem >> > Project 'trunk' is missing required library: >> > >> > >> 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-expressions-5.0.0-SNAPSHOT.jar' >> > trunk Build path Build Path Problem >> > Project 'trunk' is missing required library: >> > >> > >> 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-core-5.0.0-SNAPSHOT.jar' >> > trunk Build path Build Path Problem >> > Project 'trunk' is missing required library: >> > >> > >> 'solr/example/solr-webapp/webapp/WEB-INF/lib/lucene-highlighter-5.0.0-SNAPSHOT.jar' >> > trunk Build path Build Path Problem >> > Project 'trunk' is missing required library: >> >
Re: SOLR Boolean clause impact on memory/Performance
A terms query will be better than a boolean query here (assuming you don't care about scoring those terms): http://heliosearch.org/solr-terms-query/ But you need a recent version of Solr or Heliosearch. -Yonik http://heliosearch.org - native code faceting, facet functions, sub-facets, off-heap data On Mon, Oct 13, 2014 at 12:10 PM, ankit gupta wrote: > hi, > > Can we quantify the impact on SOLR memory usage/performance if we increase > the boolean clause. I am currently using lot of OR clauses in the query > (close to 10K) and can see heap size growing. > > Thanks, > Ankit
Need help on this AND query, solr returning incorrect results?
Hi,
I've indexed two rows with two columns:
title: Google
subject: Java Interface
and another
title: FaceBook
subject: Not Java
Now I use this query:
title:"facebook" and subject:"java"
It returns both of the rows above, weird. It looks like an "OR" query
to me, hmm.
Attached the original PHP code (replace the `if (0)` if you're
inserting data for the first time)
'127.0.0.1'));
/// INSERT
if (0)
{
$data = array (
array (
'id' => 100,
'title'=> 'Google',
'subject' => 'Java Interface'
),
array (
'id' => 101,
'title'=> 'FaceBook',
'subject' => 'Not Java'
)
);
foreach ($data as $input)
{
$doc = new SolrInputDocument();
foreach ($input as $key => $value)
{
$doc->addField ($key, $value);
}
$solr->addDocument ($doc, false, 1000);
}
sleep (3);
}
/// QUERY
$query = new SolrQuery();
$query->setQuery('title:"facebook" and subject:"java"');
$query->addField('id')->addField('title')->addField('subject');
$resp = $solr->query($query)->getResponse ();
print_r ($resp->response->docs);
?>
--
Best Regards,
Aaron Lewis - PGP: 0x13714D33 - http://pgp.mit.edu/
Finger Print: 9F67 391B B770 8FF6 99DC D92D 87F6 2602 1371 4D33
Re: ArrayIndexOutOfBoundsException in ToParentBlockJoinQuery
Thanks a lot Mikhail, moving parent filters to fq solved the problem.
Thanks for the wt=csv technique that also was very helpful.
Thanks.
Faisal
On Mon, Oct 13, 2014 at 11:08 PM, Mikhail Khludnev <
mkhlud...@griddynamics.com> wrote:
> Hello Fasial,
>
> It's convenient to use wt=csv to verify block layout (make sue that rows=
> is high enough). Note, every span of children should be followed by parent.
> see
>
> http://blog.griddynamics.com/2013/12/grandchildren-and-siblings-with-block.html
> Putting BRAND_s: clause into the {! which= } is absolutely wrong. That
> param should contain only parent docs filter which differentiate them from
> parents. Any parent level filters should be passed at the top level of
> query or like fq=.
>
>
> On Tue, Oct 14, 2014 at 8:33 AM, Faisal Mansoor
> wrote:
>
> > Thanks Erick,
> >
> > I am using the 4.10.0 solr. To update any field, I first delete all
> > existing documents matching _root_:id before inserting the updated
> version.
> > So, I don't think that's causing the problem.
> > Is there a way to validate the sanity of the index, are there any tools
> > which can analyze the index and list problems or inconsistencies in the
> > index?
> >
> > Thanks,
> > Faisal
> >
> > On Mon, Oct 13, 2014 at 6:55 PM, Erick Erickson >
> > wrote:
> >
> > > What version of Solr? For Block Join you need to index all the docs
> > > (parent/child) docs at once, and when they're updated they _all_ need
> > > to be updated AFAIK.
> > >
> > > Best,
> > > Erick
> > >
> > > On Mon, Oct 13, 2014 at 3:51 PM, Faisal Mansoor
> > > wrote:
> > > > Hi,
> > > >
> > > > I am having problems running Parent Block Join Query in Solr. For
> some
> > > > reasons the following query throws ArrayIndexOutOfBoundsException.
> > > >
> > > > {!type=parent which='type_s:(shirt) AND BRAND_s:(Puma)'}
> > > > (type_s:(sku) AND SIZE_s:(XL))
> > > >
> > > >
> > > > The index I am using is not that big, only has a few hundred entries.
> > But
> > > > it is frequently updated. Any clues what can cause this problem?
> > > >
> > > > Thanks.
> > > > Faisal
> > > >
> > > > Here is the complete stack trace for the exception:
> > > >
> > > >
> > > > "trace": "java.lang.ArrayIndexOutOfBoundsException: -1\n\tat
> > > >
> > org.apache.lucene.codecs.lucene40.BitVector.get(BitVector.java:149)\n\tat
> > > >
> > >
> >
> org.apache.lucene.search.join.ToParentBlockJoinQuery$BlockJoinScorer.nextDoc(ToParentBlockJoinQuery.java:293)\n\tat
> > > >
> > >
> >
> org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:192)\n\tat
> > > >
> > >
> >
> org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:163)\n\tat
> > > > org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)\n\tat
> > > >
> > >
> >
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:621)\n\tat
> > > >
> > >
> >
> org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:297)\n\tat
> > > >
> > >
> >
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:209)\n\tat
> > > >
> > >
> >
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1619)\n\tat
> > > >
> > >
> >
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1433)\n\tat
> > > >
> > >
> >
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:514)\n\tat
> > > >
> > >
> >
> org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:485)\n\tat
> > > >
> > >
> >
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:218)\n\tat
> > > >
> > >
> >
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)\n\tat
> > > > org.apache.solr.core.SolrCore.execute(SolrCore.java:1967)\n\tat
> > > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)\n\tat
> > > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)\n\tat
> > > >
> > >
> >
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)\n\tat
> > > >
> > >
> >
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)\n\tat
> > > >
> > >
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)\n\tat
> > > >
> > >
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)\n\tat
> > > >
> > >
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)\n\tat
> > > >
> > >
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)\n\tat
> > > >
> > >
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)\n\tat
> > > >
> > >
> >
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)\n\tat
> > > >
> > >
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)\n\tat
> > > >
> > >
> >
> org.eclipse.jetty.server.handler.Cont
Re: Need help on this AND query, solr returning incorrect results?
and is case sensitive, have you tried it with AND?
So this query is probably parsed as
title:facebook OR defaultsearchfield:and OR subject:java
assuming your default operator is "OR"
Try it on the URL with &debug=query for a quick check of what the actual
query is after it's made it through all the parsing...
Best,
Erick
On Tue, Oct 14, 2014 at 7:46 PM, Aaron Lewis wrote:
> Hi,
>
> I've indexed two rows with two columns:
>
> title: Google
> subject: Java Interface
>
> and another
>
> title: FaceBook
> subject: Not Java
>
> Now I use this query:
>
> title:"facebook" and subject:"java"
>
> It returns both of the rows above, weird. It looks like an "OR" query
> to me, hmm.
>
> Attached the original PHP code (replace the `if (0)` if you're
> inserting data for the first time)
>
>
> $solr = new SolrClient (array ('hostname' => '127.0.0.1'));
>
>/// INSERT
>if (0)
>{
> $data = array (
> array (
> 'id' => 100,
> 'title'=> 'Google',
> 'subject' => 'Java Interface'
> ),
> array (
> 'id' => 101,
> 'title'=> 'FaceBook',
> 'subject' => 'Not Java'
> )
> );
>
> foreach ($data as $input)
> {
> $doc = new SolrInputDocument();
> foreach ($input as $key => $value)
> {
> $doc->addField ($key, $value);
> }
>
> $solr->addDocument ($doc, false, 1000);
> }
>
> sleep (3);
>}
>
>/// QUERY
>$query = new SolrQuery();
>
>$query->setQuery('title:"facebook" and subject:"java"');
>$query->addField('id')->addField('title')->addField('subject');
>
>$resp = $solr->query($query)->getResponse ();
>print_r ($resp->response->docs);
>
> ?>
>
>
>
> --
> Best Regards,
> Aaron Lewis - PGP: 0x13714D33 - http://pgp.mit.edu/
> Finger Print: 9F67 391B B770 8FF6 99DC D92D 87F6 2602 1371 4D33
[RESOLVED] Re: Need help on this AND query, solr returning incorrect results?
Thanks Erick, the uppercase worked, I just didn't know that ...
On Wed, Oct 15, 2014 at 1:00 PM, Erick Erickson wrote:
> and is case sensitive, have you tried it with AND?
>
> So this query is probably parsed as
>
> title:facebook OR defaultsearchfield:and OR subject:java
>
> assuming your default operator is "OR"
>
> Try it on the URL with &debug=query for a quick check of what the actual
> query is after it's made it through all the parsing...
>
> Best,
> Erick
>
> On Tue, Oct 14, 2014 at 7:46 PM, Aaron Lewis
> wrote:
>> Hi,
>>
>> I've indexed two rows with two columns:
>>
>> title: Google
>> subject: Java Interface
>>
>> and another
>>
>> title: FaceBook
>> subject: Not Java
>>
>> Now I use this query:
>>
>> title:"facebook" and subject:"java"
>>
>> It returns both of the rows above, weird. It looks like an "OR" query
>> to me, hmm.
>>
>> Attached the original PHP code (replace the `if (0)` if you're
>> inserting data for the first time)
>>
>>
>> >$solr = new SolrClient (array ('hostname' => '127.0.0.1'));
>>
>>/// INSERT
>>if (0)
>>{
>> $data = array (
>> array (
>> 'id' => 100,
>> 'title'=> 'Google',
>> 'subject' => 'Java Interface'
>> ),
>> array (
>> 'id' => 101,
>> 'title'=> 'FaceBook',
>> 'subject' => 'Not Java'
>> )
>> );
>>
>> foreach ($data as $input)
>> {
>> $doc = new SolrInputDocument();
>> foreach ($input as $key => $value)
>> {
>> $doc->addField ($key, $value);
>> }
>>
>> $solr->addDocument ($doc, false, 1000);
>> }
>>
>> sleep (3);
>>}
>>
>>/// QUERY
>>$query = new SolrQuery();
>>
>>$query->setQuery('title:"facebook" and subject:"java"');
>>$query->addField('id')->addField('title')->addField('subject');
>>
>>$resp = $solr->query($query)->getResponse ();
>>print_r ($resp->response->docs);
>>
>> ?>
>>
>>
>>
>> --
>> Best Regards,
>> Aaron Lewis - PGP: 0x13714D33 - http://pgp.mit.edu/
>> Finger Print: 9F67 391B B770 8FF6 99DC D92D 87F6 2602 1371 4D33
--
Best Regards,
Aaron Lewis - PGP: 0x13714D33 - http://pgp.mit.edu/
Finger Print: 9F67 391B B770 8FF6 99DC D92D 87F6 2602 1371 4D33
Re: [RESOLVED] Re: Need help on this AND query, solr returning incorrect results?
Glad it worked. That's one of Solr's little quirks that _everybody_ finds out,
usually painfully.
Erick
On Tue, Oct 14, 2014 at 10:04 PM, Aaron Lewis
wrote:
> Thanks Erick, the uppercase worked, I just didn't know that ...
>
> On Wed, Oct 15, 2014 at 1:00 PM, Erick Erickson
> wrote:
>> and is case sensitive, have you tried it with AND?
>>
>> So this query is probably parsed as
>>
>> title:facebook OR defaultsearchfield:and OR subject:java
>>
>> assuming your default operator is "OR"
>>
>> Try it on the URL with &debug=query for a quick check of what the actual
>> query is after it's made it through all the parsing...
>>
>> Best,
>> Erick
>>
>> On Tue, Oct 14, 2014 at 7:46 PM, Aaron Lewis
>> wrote:
>>> Hi,
>>>
>>> I've indexed two rows with two columns:
>>>
>>> title: Google
>>> subject: Java Interface
>>>
>>> and another
>>>
>>> title: FaceBook
>>> subject: Not Java
>>>
>>> Now I use this query:
>>>
>>> title:"facebook" and subject:"java"
>>>
>>> It returns both of the rows above, weird. It looks like an "OR" query
>>> to me, hmm.
>>>
>>> Attached the original PHP code (replace the `if (0)` if you're
>>> inserting data for the first time)
>>>
>>>
>>> >>$solr = new SolrClient (array ('hostname' => '127.0.0.1'));
>>>
>>>/// INSERT
>>>if (0)
>>>{
>>> $data = array (
>>> array (
>>> 'id' => 100,
>>> 'title'=> 'Google',
>>> 'subject' => 'Java Interface'
>>> ),
>>> array (
>>> 'id' => 101,
>>> 'title'=> 'FaceBook',
>>> 'subject' => 'Not Java'
>>> )
>>> );
>>>
>>> foreach ($data as $input)
>>> {
>>> $doc = new SolrInputDocument();
>>> foreach ($input as $key => $value)
>>> {
>>> $doc->addField ($key, $value);
>>> }
>>>
>>> $solr->addDocument ($doc, false, 1000);
>>> }
>>>
>>> sleep (3);
>>>}
>>>
>>>/// QUERY
>>>$query = new SolrQuery();
>>>
>>>$query->setQuery('title:"facebook" and subject:"java"');
>>>$query->addField('id')->addField('title')->addField('subject');
>>>
>>>$resp = $solr->query($query)->getResponse ();
>>>print_r ($resp->response->docs);
>>>
>>> ?>
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Aaron Lewis - PGP: 0x13714D33 - http://pgp.mit.edu/
>>> Finger Print: 9F67 391B B770 8FF6 99DC D92D 87F6 2602 1371 4D33
>
>
>
> --
> Best Regards,
> Aaron Lewis - PGP: 0x13714D33 - http://pgp.mit.edu/
> Finger Print: 9F67 391B B770 8FF6 99DC D92D 87F6 2602 1371 4D33
