Re: import solr source to eclipse

2014-10-14 Thread Rajani Maski
Configure eclipse with Jetty plugin. Create a Solr folder under your Solr-Java-Project and Run the project [Run as] on Jetty Server. This blog[1] may help you to configure Solr within eclipse. [1] http://hokiesuns.blogspot.in/2010/01/setting-up-apache-solr-in-eclipse.html On Tue, Oct 14, 2014 a

Need to reindex when changing schema.xml?

2014-10-14 Thread Roger Sindreu
Hello I hope this question has not been asked many times. I did some research but I never could find clearly answered anywhere. We have several multivalue fields on a instance with millions of documents which only contain a single value. I would like to change it to multivalue=false to be able to

Re: import solr source to eclipse

2014-10-14 Thread Anurag Sharma
Another alternative is launch the jetty server from outside and attach it remotely from eclipse. java -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=7666 -jar start.jar The above command waits until the application attach succeed. On Tue, Oct 14, 2014 at 12:56 PM, Rajani Maski

mark solr documents as duplicates on hashing the combination of some fields

2014-10-14 Thread Ali Nazemian
Dear all, Hi, I was wondering how can I mark some documents as duplicate (just marking for future usage not deleting) based on the hash combination of some fields? Suppose I have 2 fields name "url" and "title" I want to create hash based on url+title and send it to another field name "signature".

Re: Need to reindex when changing schema.xml?

2014-10-14 Thread Alan Woodward
You should be able to change it without re-indexing, unless you've enabled docValues on that field. AFAIK docValues are the only persistent data structure that is different for single-valued versus multi-valued, everything else (UninvertedFields, etc) is built on the fly. I don't think there's

Re: best load balancer for solr cloud

2014-10-14 Thread Apoorva Gaurav
Thanks Shawn, Amey, Any specific configuration needed for CloudSolrServer as I've seen increased latency on using it. Does ConcurrentUpdateSolrServer itself do discovery like CloudSolrServer. On Mon, Oct 13, 2014 at 7:53 PM, Shawn Heisey wrote: > On 10/13/2014 5:28 AM, Apoorva Gaurav wrote: > >

having Solr deduplication and partial update

2014-10-14 Thread Ali Nazemian
Hi, I was wondering how can I have both solr deduplication and partial update. I found out that due to some reasons you can not rely on solr deduplication when you try to update a document partially! It seems that when you do partial update on some field- even if that field does not consider as dup

Re: SolrCloud: Meaning of SYNC state in ZkStateReader?

2014-10-14 Thread Martin Grotzke
Ok, thanks for your response, Mark! Cheers, Martin On Tue, Oct 14, 2014 at 1:59 AM, Mark Miller wrote: > I think it's just cruft I left in and never ended up using anywhere. You > can ignore it. > > - Mark > > > On Oct 13, 2014, at 8:42 PM, Martin Grotzke < > martin.grot...@googlemail.com> wro

Re: SOLR Boolean clause impact on memory/Performance

2014-10-14 Thread ankit gupta
Thanks Erick for responding. We have assigned 4GB memory for SOLR server and at high load where queries are having more than 10K boolean clauses, combination of cache and high boolean clauses are causing system to break. The system was working fine for last 8 months but ofcourse the boolean clause

Solr FilterCache size

2014-10-14 Thread nutchsolruser
I am planning to use FilterCache in my solr setup. Want to know few things regarding Filtercache 1. If my solrconfig.xml doesn't contain any cache configuration , basically I have removed all the cache properties from solrconfig.xml. In this case still caches will be used or not? if yes what is def

Recovering from Out of Mem

2014-10-14 Thread Salman Akram
I know there are some suggestions to avoid OOM issue e.g. setting appropriate Max Heap size etc. However, what's the best way to recover from it as it goes into non-responding state? We are using Tomcat on back end. The scenario is that once we face OOM issue it keeps on taking queries (doesn't gi

numfound in solr

2014-10-14 Thread neha sinha
Hi I ran indexing and my total rows fetched is 1736789 but my numfound in solr indexed url 638169. What could be the possible reason?? -- View this message in context: http://lucene.472066.n3.nabble.com/numfound-in-solr-tp4164169.html Sent from the Solr - User mailing list archive at Nabble

Solr ExtractingRequestHandler - Internal server Error

2014-10-14 Thread dev09
Hi, I am trying to index rich documents with ExtractingRequestHandler. So for configuration I have in solrconfig.xml (I put all the jar of contrib/extraction/lib in solr/lib) And - text true ignored_ true links ignored_ But when i launch curl "http://localhost

Re: Recovering from Out of Mem

2014-10-14 Thread Mark Miller
Best is to pass the Java cmd line option that kills the process on OOM and setup a supervisor on the process to restart it. You need a somewhat recent release for this to work properly though. - Mark > On Oct 14, 2014, at 9:06 AM, Salman Akram > wrote: > > I know there are some suggestions

Re: Recovering from Out of Mem

2014-10-14 Thread Boogie Shafer
a really simple approach is to have the OOM generate an email e.g. 1) create a simple script (call it java_oom.sh) and drop it in your tomcat bin dir echo `date` | mail -s "Java Error: OutOfMemory - $HOSTNAME" not...@domain.com 2) configure your java options (in setenv.sh or similar) to tr

Re: Recovering from Out of Mem

2014-10-14 Thread Markus Jelsma
And don't forget to set the proper permissions on the script, the tomcat or jetty user. Markus On Tuesday 14 October 2014 13:47:47 Boogie Shafer wrote: > a really simple approach is to have the OOM generate an email > > e.g. > > 1) create a simple script (call it java_oom.sh) and drop it in yo

Re: numfound in solr

2014-10-14 Thread Alexandre Rafalovitch
Your unique ID field is not unique most likely. Check what Solr thinks your unique ID field is in the schema.xml and then check whether that matches your expectations and your DB's unique criteria. The other option is if you use explicit commits only and did not commit at the end (less likely). R

Re: Recovering from Out of Mem

2014-10-14 Thread Yago Riveiro
Boogie, Any example for java_error.sh script? — /Yago Riveiro On Tue, Oct 14, 2014 at 2:48 PM, Boogie Shafer wrote: > a really simple approach is to have the OOM generate an email > e.g. > 1) create a simple script (call it java_oom.sh) and drop it in your tomcat > bin dir > echo `date`

Re: Need to reindex when changing schema.xml?

2014-10-14 Thread Alexandre Rafalovitch
On 14 October 2014 04:40, Alan Woodward wrote: > I don't think there's any definitive reference on what requires a re-index, > but that would be a nice thing to add to the Reference Guide +1 That's a really good suggestion. Even a minimal page could be a good place we could add comments to as th

Re: Recovering from Out of Mem

2014-10-14 Thread Markus Jelsma
This will do: kill -9 `ps aux | grep -v grep | grep tomcat6 | awk '{print $2}'` pkill should also work On Tuesday 14 October 2014 07:02:03 Yago Riveiro wrote: > Boogie, > > > > > Any example for java_error.sh script? > > > — > /Yago Riveiro > > On Tue, Oct 14, 2014 at 2:48 PM, Boogie Shafe

Re: SolrCloud 4.7 not doing distributed search when querying from a load balancer.

2014-10-14 Thread Tim Potter
Try adding shards.info=true and debug=track to your queries ... these will give more detailed information about what's going behind the scenes. On Mon, Oct 13, 2014 at 11:11 PM, S.L wrote: > Erick, > > I have upgraded to SolrCloud 4.10.1 with the same toplogy , 3 shards and 2 > replication facto

Re: numfound in solr

2014-10-14 Thread Jack Krupansky
It would be nice to have a logging option to log updates vs. inserts, to help make it more obvious what is happening. And maybe even a way for a Solr update request to get back a summary of how many documents were inserted, updated, and deleted. -- Jack Krupansky -Original Message- F

Re: Recovering from Out of Mem

2014-10-14 Thread Tim Potter
jfyi - the bin/solr script does the following: -XX:OnOutOfMemoryError="$SOLR_TIP/bin/oom_solr.sh $SOLR_PORT" where $SOLR_PORT is the port Solr is bound to, e.g. 8983 The oom_solr.sh script looks like: SOLR_PORT=$1 SOLR_PID=`ps waux | grep start.jar | grep $SOLR_PORT | grep -v grep | awk '{print

Re: import solr source to eclipse

2014-10-14 Thread Erick Erickson
I do exactly what Anurag mentioned, but _only_ when what I want to debug is, for some reason, not accessible via unit tests. It's very easy to do. It's usually much faster though to use unit tests, which you should be able to run from eclipse without starting a server at all. In IntelliJ, you just

Re: SOLR Boolean clause impact on memory/Performance

2014-10-14 Thread Erick Erickson
Then I predict they will continue to grow and whatever limit you put on maxBooleanClauses will be exceeded later. And so on, so I really think you need to re-think your model. One approach: 1> change your model so your users are assigned to a fixed number of groups. Then index group tokens with ea

Re: Recovering from Out of Mem

2014-10-14 Thread Boogie Shafer
yago, you can put more complex restart logic as shown in the examples below or just do something similar to the java_oom.sh i posted earlier where you just spit out an email alert and deal with service restarts and troubleshooting manually e.g. something like the following for a java_error.sh

Re: Solr FilterCache size

2014-10-14 Thread Erick Erickson
1> Uhmmm, why remove the settings then wonder what the defaults are? Just leave them in and you _know_. Otherwise please look at the code. 2> This is trial and error. The most important bit is the size parameter. There's little reason to make initialSize different from size, it's just pre-allocati

Re: Need to reindex when changing schema.xml?

2014-10-14 Thread Erick Erickson
I'm going to inject a bit of caution here, since I've seen odd behaviors pop out in cases like this. But not, I admit, this particular case. I'd just go ahead and try it on a test index first. It's always possible there's a safety check somewhere that'll be tripped by this kind of change. Erick@P

Re: numfound in solr

2014-10-14 Thread Alexandre Rafalovitch
On 14 October 2014 10:35, Jack Krupansky wrote: > It would be nice to have a logging option to log updates vs. inserts, to > help make it more obvious what is happening. And maybe even a way for a Solr > update request to get back a summary of how many documents were inserted, > updated, and delet

eDismax - boost function of multiple values

2014-10-14 Thread Jens Mayer
Hey everyone, I have a question about the boost function of solr. The documentation say about multiple function querys that I can write it seperated by whitespaces. Example: q=foo&bf=ord(popularity)^0.5 recip(rord(price),1,1000,1000)^0.3 Now I have two fields I like to boost. Inhabitants and im

Re: Facets for Child Documents?

2014-10-14 Thread Oleg Savrasov
Hi Joshua, The functionality you are asking about is requested by https://issues.apache.org/jira/browse/SOLR-5743. I've prepared a patch with initial implementation and going to speak about it on Lucene/Solr Revolution 2014 Conference, held in Washington, DC on November 11-14, http://lucenerevolut

Re: best load balancer for solr cloud

2014-10-14 Thread Shawn Heisey
On 10/14/2014 2:45 AM, Apoorva Gaurav wrote: > Any specific configuration needed for CloudSolrServer as I've seen > increased latency on using it. Does ConcurrentUpdateSolrServer itself do > discovery like CloudSolrServer. ConcurrentUpdateSolrServer is just a wrapper for HttpSolrServer that does s

Re: eDismax - boost function of multiple values

2014-10-14 Thread Ahmet Arslan
Hi Jens, Where did you read that you can write it separated by white spaces? bq and bf are both can be defined multiple times. q=foo&bf=ord(inhabitants)bf=ord(importance) Ahmet On Tuesday, October 14, 2014 6:34 PM, Jens Mayer wrote: Hey everyone, I have a question about the boost function

Re: does one need to reindex when changing similarity class

2014-10-14 Thread elisabeth benoit
thanks a lot for your answers! 2014-10-14 6:10 GMT+02:00 Jack Krupansky : > To correct myself, the selected Similarity class can have a computeNorm > method that calculates the "norm" value that will be stored in the index > when the document is indexed, so changing the Similarity class will requ

solr trunk update.. build issues

2014-10-14 Thread Anurag Sharma
Just wanted to check if anyone faced below issues and how they fixed: Solr launch issue --- >From command line ant compile, ant dist gives BUILD SUCCESSFUL generating \solr\dist\solr-6.0.0-SNAPSHOT.war. After copying and renaming the same file to example/start.jar on run shows

Re: solr trunk update.. build issues

2014-10-14 Thread Shalin Shekhar Mangar
On Wed, Oct 15, 2014 at 12:38 AM, Anurag Sharma wrote: > Just wanted to check if anyone faced below issues and how they fixed: > > Solr launch issue > --- > From command line ant compile, ant dist gives BUILD SUCCESSFUL > generating \solr\dist\solr-6.0.0-SNAPSHOT.war. After co

Re: solr trunk update.. build issues

2014-10-14 Thread Anurag Sharma
Hi Shalin, Sorry about the typo, actually it's renaming the war file( \solr\dist\solr-6.0.0-SNAPSHOT.war) placed to \trunk\solr\example\webapps\solr.war as referred in last point of step7 in http://wiki.apache.org/solr/HowToCompileSolr I'll also refer and try https://cwiki.apache.org/confluence/di

RE: Result grouping using externalfilefield

2014-10-14 Thread Cario, Elaine
Sudhaker, Not sure if this has anything to do with your problem, but I had an issue with grouping on non-string fields (in my case it was an integer) in SolrCloud setup (4.7). But I was using internal fields. We worked around it by defining the field as a string instead. -Original Messa

Re: solr trunk update.. build issues

2014-10-14 Thread Anurag Sharma
I am able to launch solr example start.jar using: java –jar start.jar The eclipse issue is also resolved now, looks like eclipse was taking time to process all the files and was an intermittent issue. Now am able to run the selected JUnits as well. On Wed, Oct 15, 2014 at 1:06 AM, Anurag Sharma

Re: SOLR Boolean clause impact on memory/Performance

2014-10-14 Thread Yonik Seeley
A terms query will be better than a boolean query here (assuming you don't care about scoring those terms): http://heliosearch.org/solr-terms-query/ But you need a recent version of Solr or Heliosearch. -Yonik http://heliosearch.org - native code faceting, facet functions, sub-facets, off-heap da

Need help on this AND query, solr returning incorrect results?

2014-10-14 Thread Aaron Lewis
Hi, I've indexed two rows with two columns: title: Google subject: Java Interface and another title: FaceBook subject: Not Java Now I use this query: title:"facebook" and subject:"java" It returns both of the rows above, weird. It looks like an "OR" query to me, hmm. Attached the original P

Re: ArrayIndexOutOfBoundsException in ToParentBlockJoinQuery

2014-10-14 Thread Faisal Mansoor
Thanks a lot Mikhail, moving parent filters to fq solved the problem. Thanks for the wt=csv technique that also was very helpful. Thanks. Faisal On Mon, Oct 13, 2014 at 11:08 PM, Mikhail Khludnev < mkhlud...@griddynamics.com> wrote: > Hello Fasial, > > It's convenient to use wt=csv to verify blo

Re: Need help on this AND query, solr returning incorrect results?

2014-10-14 Thread Erick Erickson
and is case sensitive, have you tried it with AND? So this query is probably parsed as title:facebook OR defaultsearchfield:and OR subject:java assuming your default operator is "OR" Try it on the URL with &debug=query for a quick check of what the actual query is after it's made it through

[RESOLVED] Re: Need help on this AND query, solr returning incorrect results?

2014-10-14 Thread Aaron Lewis
Thanks Erick, the uppercase worked, I just didn't know that ... On Wed, Oct 15, 2014 at 1:00 PM, Erick Erickson wrote: > and is case sensitive, have you tried it with AND? > > So this query is probably parsed as > > title:facebook OR defaultsearchfield:and OR subject:java > > assuming your defaul

Re: [RESOLVED] Re: Need help on this AND query, solr returning incorrect results?

2014-10-14 Thread Erick Erickson
Glad it worked. That's one of Solr's little quirks that _everybody_ finds out, usually painfully. Erick On Tue, Oct 14, 2014 at 10:04 PM, Aaron Lewis wrote: > Thanks Erick, the uppercase worked, I just didn't know that ... > > On Wed, Oct 15, 2014 at 1:00 PM, Erick Erickson > wrote: >> and