Re: Solr Search Inconsistent result

2014-12-22 Thread Ankit Jain
Hi Ahmet, Thanks for the response. Document ID is unique because we are using *UUID* to generate the document ID. Thanks, Ankit On Tue, Dec 23, 2014 at 12:16 AM, Ahmet Arslan wrote: > Hi, > > Do you happen to have documents with with unique id in different shards? > When unique ids are not uni

How to define Json list in schema in xml

2014-12-22 Thread Xin Cai
hi guys I am looking to parse a json file that contains fields that has a list of schools So for example I would have {"Schools":[ name: "Seirra High School", name: "Walnut elementary School"]} So if I want to be able to index all the different schools so i can fast look up with people that wen

Re: Pointing solr cloud to multiple index directories.

2014-12-22 Thread Shawn Heisey
On 12/22/2014 2:10 PM, Nishanth S wrote: > I have 5 drives in my machine which are mounted to 5 different > locations(/d/1 ,/d/2,/d/3).How can I point solr to write to all these > directories?. Erick has asked a relevant question. I assume that you're trying to take advantage of the extra I/O ba

Solr unit tests intermittently fail with error: java.lang.NoClassDefFoundError: org/eclipse/jetty/util/security/CertificateUtils

2014-12-22 Thread brian4
I'm trying to run a unit test for a custom request handler component with Solr 4.10.0. I followed the pattern of existing "unit tests", extending "SolrTestCaseJ4". I first ran "ant eclipse" on the 4.10 source, then included all lib files generated (as well as all the solr and lucene core lib file

Re: Pointing solr cloud to multiple index directories.

2014-12-22 Thread Erick Erickson
Not at all sure what you're asking If you're creating cores/replicas, you can specify a dataDir. But you haven't really told us anything at all about what you're trying to do here. or _why_ you want to write to them all. Best Erick On Mon, Dec 22, 2014 at 1:10 PM, Nishanth S wrote: > H

Pointing solr cloud to multiple index directories.

2014-12-22 Thread Nishanth S
Hey folks, I have 5 drives in my machine which are mounted to 5 different locations(/d/1 ,/d/2,/d/3).How can I point solr to write to all these directories?. Thanks, Nishanth

Re: IOException occured when talking to solr server

2014-12-22 Thread Shawn Heisey
On 12/22/2014 5:27 AM, Aditya wrote: > I am getting following error. Could anyone throw me some light on it. I am > accessing Solr via Solrj, when there is more load on the server i am > getting this error. Is there any way to overcome this situitation. > > org.apache.solr.client.solrj.SolrServerEx

Re: Solr Search Inconsistent result

2014-12-22 Thread Ahmet Arslan
Hi, Do you happen to have documents with with unique id in different shards? When unique ids are not unique across shards, people see inconsistent results. Please see : http://find.searchhub.org/document/2814183511b5a52 Ahmet On Monday, December 22, 2014 8:06 PM, Ankit Jain wrote: Hi Ahmet,

Re: Solr Search Inconsistent result

2014-12-22 Thread Ankit Jain
Hi Ahmet, Thanks for the response. I am running this query from Solr Search UI. The number of shards for a collection is two. Thanks, Ankit On Mon, Dec 22, 2014 at 8:34 PM, Ahmet Arslan wrote: > Hi, > > Is this sharded query? > > Ahmet > > > On Monday, December 22, 2014 4:47 PM, Ankit Jain >

Re: Old facet value doesn't go away after index update

2014-12-22 Thread Tang, Rebecca
Thank you for the explanation! Rebecca Tang Applications Developer, UCSF CKM Industry Documents Digital Libraries E: rebecca.t...@ucsf.edu On 12/19/14 12:37 PM, "Shawn Heisey" wrote: >On 12/19/2014 11:22 AM, Tang, Rebecca wrote: >> I have an index that has a field called collection_facet. >

Re: Parallel Indexing

2014-12-22 Thread Peri Subrahmanya
Thanks guys for the quick responses. I need to take the suggestions, incorporate them, figure out how is that we are doing the fetching etc and reply back on this post. The suggestions have been very helpful in taking this forward for us here. Thanks -Peri.S > On Dec 22, 2014, at 10:32 AM, Er

Re: Parallel Indexing

2014-12-22 Thread Erick Erickson
Just to pile on _very_ frequently in my experience the problem is not Solr at all, but acquiring the data in the first place, i.e. often executing the DB query. A very simple test is (in the SolrJ world) just comment out the server.add(doclist). Assuming you're using SolrJ, you _are_ indexin

Re: SolrCloud & Paging on large indexes

2014-12-22 Thread Erick Erickson
Have you read Hossman's blog here? https://lucidworks.com/blog/coming-soon-to-solr-efficient-cursor-based-iteration-of-large-result-sets/#referrer=solr.pl And how to use it here? http://wiki.apache.org/solr/CommonQueryParameters#Deep_paging_with_cursorMark Because if you're trying this and _still

Re: Parallel Indexing

2014-12-22 Thread Mikhail Khludnev
What your indexer is build on? Do you use SolrJ, just REST, or DataImportHandler? What's you DB schema is briefly? Frankly speaking, there are few approaches to handle indexing concurrently, details depends on the details mentioned above. On Mon, Dec 22, 2014 at 5:54 PM, Peri Subrahmanya < peri.su

Re: Endless 100% CPU usage on searcherExecutor thread

2014-12-22 Thread Erick Erickson
Milliseconds. The thing to track here is your cumulative_hitratio. 0.7 isn't bad, but it's not great either. I'd be really curious what kinds of fq clauses you're entering, anything that mentions NOW is potentially a waste unless you round with "date math" FWIW, Erick On Mon, Dec 22, 2014 at

Re: Solr Search Inconsistent result

2014-12-22 Thread Ahmet Arslan
Hi, Is this sharded query? Ahmet On Monday, December 22, 2014 4:47 PM, Ankit Jain wrote: Hi All, We are getting inconsistent search result on searching on *multivalued* field: *Input Query:* ( t : [ 0 TO 1419245069253 ] )AND(_all:"impetus-i0111.impetus.co.in") The "_all" field is multiva

Re: Parallel Indexing

2014-12-22 Thread Ahmet Arslan
Hi Peri, You can always send concurrent update requests to solr. Usually data acquisition takes more time than indexing time. You can dump your db record into several csv files and you can feed them to solr in parallel. Ahmet On Monday, December 22, 2014 4:55 PM, Peri Subrahmanya wrote: H

Re: Querying parent with multiple child documents

2014-12-22 Thread Rajesh
Thanks for your reply Mikhail. It's working as expected. -- View this message in context: http://lucene.472066.n3.nabble.com/Querying-parent-with-multiple-child-documents-tp4175546p4175579.html Sent from the Solr - User mailing list archive at Nabble.com.

Parallel Indexing

2014-12-22 Thread Peri Subrahmanya
Hi, We have millions of records in our db that we do a complete re-index of every fortnight or so. It takes around 11 hours or so and I was wondering if there was a way to fetch the records in batches parallel and issue the solr http command with the solr docs in parallel. Please let me know.

Solr Search Inconsistent result

2014-12-22 Thread Ankit Jain
Hi All, We are getting inconsistent search result on searching on *multivalued* field: *Input Query:* ( t : [ 0 TO 1419245069253 ] )AND(_all:"impetus-i0111.impetus.co.in") The "_all" field is multivalued field. The above query is returning sometimes 11 records and sometimes 12471 records. Plea

Re: IOException occured when talking to solr server

2014-12-22 Thread Alexandre Rafalovitch
Could be the size of pool of listeners if this happens only under load. That's a pure Tomcat setting, look for it there. But look for the exception in the logs first, it may give better clues. Regards, Alex. Sign up for my Solr resources newsletter at http://www.solr-start.com/ On 22 De

Re: SolrCloud & Paging on large indexes

2014-12-22 Thread Bram Van Dam
On 12/22/2014 12:47 PM, heaven wrote: I have a very bad experience with pagination on collections larger than a few millions of documents. Pagination becomes very and very slow. Just tried to switch to page 76662 and it took almost 30 seconds. Yeah that's pretty much my experience, and I think

Re: IOException occured when talking to solr server

2014-12-22 Thread Tomoko Uchida
Hi, > org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://localhost/solr Clearly it is server side problem, so client SolrJ logs are not helpful. You should check Tomcat and Solr error logs and look for cause of the load. Best, Tomoko 2014-12-22 21:27 GMT+09:0

IOException occured when talking to solr server

2014-12-22 Thread Aditya
Hello all I am getting following error. Could anyone throw me some light on it. I am accessing Solr via Solrj, when there is more load on the server i am getting this error. Is there any way to overcome this situitation. org.apache.solr.client.solrj.SolrServerException: IOException occured when t

Re: Endless 100% CPU usage on searcherExecutor thread

2014-12-22 Thread heaven
It is getting better now with smaller caches like this: filterCache class:org.apache.solr.search.FastLRUCache version:1.0 description:Concurrent LRU Cache(maxSize=4096, initialSize=512, minSize=3686, acceptableSize=3891, cleanupThread=false, autowarmCount=256, regenerator=org.apache.solr.search.Sol

Re: SolrCloud & Paging on large indexes

2014-12-22 Thread heaven
I have a very bad experience with pagination on collections larger than a few millions of documents. Pagination becomes very and very slow. Just tried to switch to page 76662 and it took almost 30 seconds. Solr now supports cursors which work fast and are useful for exports and some data processin

Re: Querying parent with multiple child documents

2014-12-22 Thread Mikhail Khludnev
On Mon, Dec 22, 2014 at 2:16 PM, Rajesh wrote: > > OR query between child docs. > fq = {!parent which=\"type:parent\" v=\"productname:childproduct OR > orderDetail:childproduct2\"} > fl = *,[child parentFilter=\"type:parent\" > childFilter=\"productname:childproduct OR orderDetail:childproduct2\"]

Querying parent with multiple child documents

2014-12-22 Thread Rajesh
Hi, I've a document which has multiple child documents associated with it and child documents are from different table(Both contain different fields) mentioned below. I can query the parent and child document with a OR condition between these two child records. Is there a way how I can specify AND

Re: SolrCloud & Paging on large indexes

2014-12-22 Thread Mikhail Khludnev
Hello Bram, make sure you checked the doc https://cwiki.apache.org/confluence/display/solr/Pagination+of+Results On Mon, Dec 22, 2014 at 12:59 PM, Bram Van Dam wrote: > > Hi folks, > > If I understand things correctly, you can use paging & sorting in a > SolrCloud environment. However, if I requ

Re: set keepword file to be used based on a field value

2014-12-22 Thread Tomoko Uchida
Hi Leo, Yes, my image is similar to yours. > If the value ends with "_CAT1" ==> use > as keepword file "keepwords1.txt" and so on? But my second option is not about configurations, but "customizing" Solr. Utilizing customizability of Lucene/Solr, you can write your own TokenFilter class. Maybe y

SolrCloud & Paging on large indexes

2014-12-22 Thread Bram Van Dam
Hi folks, If I understand things correctly, you can use paging & sorting in a SolrCloud environment. However, if I request the first 10 documents, a distributed query will be launched to all shards requesting the top 10, and then (Shards * 10) documents will then be sorted so that only the to

Re: set keepword file to be used based on a field value

2014-12-22 Thread leostro
Hi Tomoko, I understand you first reply and the first hint (one field for each categoryid). I thought this was a relatively "common" scenario. I'm interested in understanding the option you are talking about in the second reply. > you can tell "which keepwords set (file) shoud be used" to custom