Re: Solr went on recovery multiple time.

2015-03-30 Thread sthita
Hi Bill, My index size is around 48GB and contains around 8 million documents. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-went-on-recovery-multiple-time-tp4196249p4196504.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrJ commit with openSearcher=false

2015-03-30 Thread Shawn Heisey
On 3/31/2015 12:25 AM, vidit.asthana wrote: > How can I issue a hard commit through SolrJ such that openSearcher=false? > > Also how can I issue same request through http? Will this work - > > curl > "http://localhost:8983/solr/collection1/update?commit=true&openSearcher=false"; This SolrJ code

Re: Restart solr failed after applied the patch in https://issues.apache.org/jira/browse/SOLR-6359

2015-03-30 Thread Ramkumar R. Aiyengar
It shouldn't be any different without the patch, or with the patch and (100,10) as parameters. Which is why I wanted you to check with 100,10.. If you see the same issue with that, then the patch is probably not an issue, may be it is with the patched build in general.. On 30 Mar 2015 13:01, "fores

Spark-Solr in python

2015-03-30 Thread Chaushu, Shani
Hi, I saw there is a tool for reading solr into Spark RDD in JAVA I want to do something like this in python, is there any package in python for reading solr into spark RDD? Thanks , Shani - Intel Electronics Ltd. This e-mail

SolrJ commit with openSearcher=false

2015-03-30 Thread vidit.asthana
How can I issue a hard commit through SolrJ such that openSearcher=false? Also how can I issue same request through http? Will this work - curl "http://localhost:8983/solr/collection1/update?commit=true&openSearcher=false"; -- View this message in context: http://lucene.472066.n3.nabble.c

Re: Collapse and Expand behaviour on result with 1 document.

2015-03-30 Thread Derek Poh
If I want to group the results (by a certain field) even if there is only 1 document, I should use the group parameter instead? The requirement is to group the result of product documents by their supplier id. "&group=true&group.field=P_SupplierId&group.limit=5" Is it true that the performance

Re: Unable to perform search query after changing uniqueKey

2015-03-30 Thread Erick Erickson
I would never put spaces in my field names! Frankly I have no clue what Solr does with that, but it can't be good. Solr explicitly supports Java naming conventions, camel case, underscores and numbers. Special symbols are frowned upon, I never use anything but upper case, lower case and underscores

Re: Same schema.xml is loaded for different cores in SolrCloud

2015-03-30 Thread Erick Erickson
By now, I wouldn't particularly trust my setup. I'd blow it away and start over. bootstrapping is _only_ required to get the configs up to Zookeeper the first time. In fact I suggest you don't use it at all. Just start SolrCloud, and use zkcli to push the configs up. Thereafter, create your collec

Re: Solr went on recovery multiple time.

2015-03-30 Thread William Bell
How big is your index? -Xmx6g is pretty small in general. On Mon, Mar 30, 2015 at 7:20 PM, sthita wrote: > Thanks Erick for the reply. These are the configurations. > > 1.Solr servers count : 4 (1 master , 3 slaves) > 2.My initial and maximum memory allocations to each solr servers are : > -Xms5

RE: Korean script conversion

2015-03-30 Thread Eyal Naamati
We only want the conversion Hanja->Hangul, for each Hanja character there exists only one Hangul character that can replace it in a Korean text. The other way around is not convertible. We want to allow searching in both scripts and find matches in both scripts. Thanks Eyal Naamati Alma Develop

Re: Unable to perform search query after changing uniqueKey

2015-03-30 Thread Zheng Lin Edwin Yeo
Latest information that I've found for this is that the error only occurs for shard2. If I do a search for just shard1, those records that are assigned to shard1 will be able to be displayed. Only when I search for shard2 will the NullPointerException error occurs. Previously I was doing a search

Re: Collapse and Expand behaviour on result with 1 document.

2015-03-30 Thread Joel Bernstein
The expanded section will only include groups that have expanded documents. So, if the document that in the main result set has no documents to expand, then this is working as expected. Joel Bernstein http://joelsolr.blogspot.com/ On Mon, Mar 30, 2015 at 8:43 PM, Derek Poh wrote: > Hi > > I

Re: Same schema.xml is loaded for different cores in SolrCloud

2015-03-30 Thread Zheng Lin Edwin Yeo
Hi Erick, I've started shard2 with the following command instead, but it's still the same problem. java -DzkHost=localhost:9983 -Djetty.port=8984 -jar start.jar But you mean for shard1 we do not have to include " -Dbootstrap_confdir=./solr/logmill/conf" for subsequent startup? Regards, Edwin O

Re: Unable to perform search query after changing uniqueKey

2015-03-30 Thread Zheng Lin Edwin Yeo
Hi Erick, I've changed the uniqueKey from id to Item No. Item No Below are my definitions for both the id and Item No. Regards, Edwin On 30 March 2015 at 23:05, Erick Erickson wrote: > Well, let's see the definition of your ID field, 'cause I'm puzzled. > > It's definitely A Bad Thing t

Re: Solr went on recovery multiple time.

2015-03-30 Thread sthita
Thanks Erick for the reply. These are the configurations. 1.Solr servers count : 4 (1 master , 3 slaves) 2.My initial and maximum memory allocations to each solr servers are : -Xms5g -Xmx6g 3. 5 running zookeepers 4. My index size in each solr server around: 48GB 5. Every sec merger add/delete 5

RE: Solr Unexpected Query Parser Exception

2015-03-30 Thread Reitzel, Charles
Saw that one. Can't remember for certain, but recall the actual syntax error was in a filter query. It could have been a quoting error or a date math error in a range expression. But, either way, the issue was in the fq. Using edismax. hth -Original Message- From: Jack Krupansky [

RE: NoNode for /clusterstate.json in solr5.0.0 cloud

2015-03-30 Thread Purohit, Sumit
One development difference is that i am trying to use it in a scala project. I compile "spark-solr" as jar and use it in my scala project. Do you think this might be a reason. thanks sumit From: Purohit, Sumit Sent: Monday, March 30, 2015 3:19 PM To: solr-

Collapse and Expand behaviour on result with 1 document.

2015-03-30 Thread Derek Poh
Hi I have a query which return 1 document. When I add the collapse and expand parameters to it, "&expand=true&expand.rows=5&fq={!collapse%20field=P_SupplierId}", the expanded section is empty (). Is this the behaviour of collapse and expand parameters on result which contain only 1 document?

Re: how do you replicate solr-cloud between datacenters?

2015-03-30 Thread Jack Krupansky
That's an open issue. See: https://issues.apache.org/jira/browse/SOLR-6273 -- Jack Krupansky On Mon, Mar 30, 2015 at 5:45 PM, Timothy Ehlers wrote: > Can you use /replication ??? How would you do this between datacenters? > > -- > Tim Ehlers >

RE: NoNode for /clusterstate.json in solr5.0.0 cloud

2015-03-30 Thread Purohit, Sumit
Yes i am using default localhost:9983 and i think my program does connect with zookeeper as per logs : 2015-03-30 15:00:51,512 [Executor task launch worker-0] INFO SolrZkClient - Using default ZkCredentialsProvider 2015-03-30 15:00:51,530 [Executor task launch worker-0] INFO ConnectionManage

Re: NoNode for /clusterstate.json in solr5.0.0 cloud

2015-03-30 Thread Timothy Potter
I upgraded the spark-solr project to solrj-5.0.0 and was able to index into the gettingstarted collection using Solr 5.0.0, so seems like it may be environmental. Almost seems like the spark project is looking at the wrong ZooKeeper? Are you using the default -zkHost localhost:9983 On Mon, Mar 30,

how do you replicate solr-cloud between datacenters?

2015-03-30 Thread Timothy Ehlers
Can you use /replication ??? How would you do this between datacenters? -- Tim Ehlers

Re: Optimize SolrCloud without downtime

2015-03-30 Thread Erick Erickson
Hmmm, are you indexing during the time you see the deleted docs not changing? Because this is very strange. Theoretically, if you reindex everything, that should result in segments that have _no_ live docs in them and they should really disappear ASAP. One way to work around this if we determine t

RE: NoNode for /clusterstate.json in solr5.0.0 cloud

2015-03-30 Thread Purohit, Sumit
Thanks Tim, i had to make some changes in my local spark-solr clone to build it for sorl5. If its ok, i can commit these to github. thanks sumit From: Timothy Potter [thelabd...@gmail.com] Sent: Monday, March 30, 2015 2:27 PM To: solr-user@lucene.apache.o

Re: NoNode for /clusterstate.json in solr5.0.0 cloud

2015-03-30 Thread Timothy Potter
Ok, let me upgrade my version of spark-solr to 5 to see what I get ... On Mon, Mar 30, 2015 at 2:26 PM, Purohit, Sumit wrote: > yes there is getting started collection.. > and on admin webpage console-->cloud--->tree--->/clusterstate.json shows > me this table > > version =1 > aversion=0 > chi

RE: NoNode for /clusterstate.json in solr5.0.0 cloud

2015-03-30 Thread Purohit, Sumit
yes there is getting started collection.. and on admin webpage console-->cloud--->tree--->/clusterstate.json shows me this table version =1 aversion=0 children_count=0 ctimeFri= Mar 27 19:20:21 UTC 2015 (1427484021901) cversion=0 czxid=32 ephemeralOwner=0 mtime=Fri Mar 27 19:20:36 UTC 2015 (142

Re: NoNode for /clusterstate.json in solr5.0.0 cloud

2015-03-30 Thread Timothy Potter
Anything in the server-side Solr logs? Also, if you go to the Solr admin console at http://localhost:8983/solr, do you see the gettingstarted collection in the cloud panel? On Mon, Mar 30, 2015 at 1:12 PM, Purohit, Sumit wrote: > I have a basic Solr 5.0.0 cloud setup after following > http://l

NoNode for /clusterstate.json in solr5.0.0 cloud

2015-03-30 Thread Purohit, Sumit
I have a basic Solr 5.0.0 cloud setup after following http://lucene.apache.org/solr/quickstart.html I am trying to read data from spark and index it into solr using following lib: https://github.com/LucidWorks/spark-solr I am getting following error when my code try to make request to solr Exc

RE: Structured and Unstructured data indexing in SolrCloud

2015-03-30 Thread Reitzel, Charles
Hi Vijay, The short answer is yes, you can combine almost anything you want into a single collection. But, in addition to working out your queries, you might want work out your data life cycle. In our application, we have comingled the structured and unstructured documents into a single col

Re: protected phrases - possible?

2015-03-30 Thread Ahmet Arslan
Hi Jing, You can boost phrases by pf (phrase fields) parameter. If you don't like this solution, you can modify search query at client side. E.g. surround certain phrases with quotes. This will force proximity search without interfering with tokenisation. Ahmet On Monday, March 30, 2015 8:49

protected phrases - possible?

2015-03-30 Thread Tao, Jing
Hi, The way our collection is setup, searches for "breast cancer" are returning results for ovarian cancer, or anything that contains either "breast" or "cancer". The reason is, we are searching across multiple fields. Even though I have set a "mm" value so that if less than 3 terms, ALL term

Re: Date & Time datatypes?

2015-03-30 Thread Erick Erickson
Hmmm, not OOB. You could index two tdate fields though and in the second one only index seconds into the day and index the other with date math (i.e. /DAY) to normalize them. Or a tdate field with just a long field for the seconds. You could then use the range faceting to, say, put the docs into h

Re: Same schema.xml is loaded for different cores in SolrCloud

2015-03-30 Thread Erick Erickson
OK, this is a bit confused: 1> You're starting two embedded Zookeepers but they don't know about each other. So looking for the configsets is a bit confused. 2> There's no need to do the bootstrap thing after the first time. The _very_ first time you do this it pushes the configs up to Zookeeper,

Re: SOLR Index in shared/Network folder

2015-03-30 Thread Walter Underwood
I suggest that you do not try to save money on disk space. Disk is cheap. You will spend weeks of expensive engineering time trying to make this work. Once you make it work, it will be slow an unreliable. 300GB Amazon EBS volumes are $180/year, $360/year for SSD. Just spend the money. wunder Wa

Re: Unable to perform search query after changing uniqueKey

2015-03-30 Thread Erick Erickson
Well, let's see the definition of your ID field, 'cause I'm puzzled. It's definitely A Bad Thing to have it be any kind of tokenized field though, but that's a shot in the dark. Best, Erick On Mon, Mar 30, 2015 at 2:17 AM, Zheng Lin Edwin Yeo wrote: > Hi Mostafa, > > Yes, I've defined all the f

Re: Solr went on recovery multiple time.

2015-03-30 Thread Erick Erickson
Sometimes happens when your zookeeper timeout is short (the 15 second timeout has been too short) and you hit a GC pause that is longer. Try lengthening your ZK timeout to 45 or 60 seconds. It would help if you told us about your setup, memory allocated to Solr, all that stuff, see: http://wiki.a

Re: SOLR Index in shared/Network folder

2015-03-30 Thread Erick Erickson
First examine whether you can reduce the amount of data you keep around, field norms, stored fields, etc. Here's a place to start: http://stackoverflow.com/questions/10080881/solr-index-size-reduction I have heard of people doing what you suggest, but be _very_ careful that you don't accidentally

Re: Restart solr failed after applied the patch in https://issues.apache.org/jira/browse/SOLR-6359

2015-03-30 Thread forest_soup
But if the value can only be 100,10, is there any difference with no that patch? Can we enlarge those 2 values? Thanks! -- View this message in context: http://lucene.472066.n3.nabble.com/Restart-solr-failed-after-applied-the-patch-in-https-issues-apache-org-jira-browse-SOLR-6359-tp4196251p4196

Re: Korean script conversion

2015-03-30 Thread Benson Margulies
Why do you think that this is a good idea? Hanja are used for special purposes; they are not trivally convertable to Hanjul due to ambiguity, and it's not at all clear that a typical search user wants to treat them as equivalent. On Sun, Mar 29, 2015 at 1:52 AM, Eyal Naamati < eyal.naam...@exlibri

Re: Restart solr failed after applied the patch in https://issues.apache.org/jira/browse/SOLR-6359

2015-03-30 Thread forest_soup
Yes, I also doubt the patch. I restore the patch with original .jar file, there is no that issue. -- View this message in context: http://lucene.472066.n3.nabble.com/Restart-solr-failed-after-applied-the-patch-in-https-issues-apache-org-jira-browse-SOLR-6359-tp4196251p4196278.html Sent from the

Re: Optimize SolrCloud without downtime

2015-03-30 Thread Pavel Hladik
Hi, thanks for reply. We have a lot of deleted docs cause we have to reindex all records from time to time, changing some important parameters.. When we do update, it means create and delete. Our deleted docs do not disappear by merging segments. I see our deleted docs are almost the same number

Date & Time datatypes?

2015-03-30 Thread Bram Van Dam
Howdy folks, Is there any way index only the date and time portions of a datetime field? A Date is really a period of 24hrs, starting at 00:00 in said date's time zone. It would be useful if there was a way to search for documents of a certain date with these semantics. As for times, I'd lik

Re: Restart solr failed after applied the patch in https://issues.apache.org/jira/browse/SOLR-6359

2015-03-30 Thread Ramkumar R. Aiyengar
I doubt this has anything to do with the patch. Do you observe the same behaviour if you reduce the values for the config to defaults? (100, 10) On 30 Mar 2015 09:51, "forest_soup" wrote: > https://issues.apache.org/jira/browse/SOLR-6359 > > I also posted the questions to the JIRA ticket. > > We

Re: Same schema.xml is loaded for different cores in SolrCloud

2015-03-30 Thread Zheng Lin Edwin Yeo
I've roughly know what is the problem from here. http://stackoverflow.com/questions/23338324/zookeeper-multiple-collection-different-schema However, I couldn't find the zoo_data directory in all of my solr folder. What could be the problem or where is the directory supposed to be located? Regards

Re: Unable to perform search query after changing uniqueKey

2015-03-30 Thread Zheng Lin Edwin Yeo
Hi Mostafa, Yes, I've defined all the fields in schema.xml. It is able to work on the version without SolrCloud, but it is not working for the one with SolrCloud. Both of them are using the same schema.xml. Regards, Edwin On 30 March 2015 at 14:34, Mostafa Gomaa wrote: > Hi Zheng, > > It's p

Restart solr failed after applied the patch in https://issues.apache.org/jira/browse/SOLR-6359

2015-03-30 Thread forest_soup
https://issues.apache.org/jira/browse/SOLR-6359 I also posted the questions to the JIRA ticket. We have a SolrCloud with 5 solr servers of Solr 4.7.0. There are one collection with 80 shards(2 replicas per shard) on those 5 servers. And we made a patch by merge the patch (https://issues.apache.or

Solr went on recovery multiple time.

2015-03-30 Thread sthita
Solr server crashes 2-3 times a day . When i looked into log i can see some recovery failure. I got these errors on my solr log WARN - 2015-03-30 16:03:51.137; org.apache.solr.cloud.RecoveryStrategy; Stopping recovery for zkNodeName=searcher:8980_solr_rn0core=rn0 ERROR - 2015-03-30 16:03:56.12

Re: Installing the auto-phrase-tokenfilter

2015-03-30 Thread luismart
Many thanks Andrea, I followed your guidance: the following site helped me to install and configure Ant in Windows http://madhukaudantha.blogspot.com.es/2010/06/installing-ant-for-windows-7.html Then after cloning the repository https://github.com/LucidWorks/auto-phrase-tokenfilter I just run th