Re: Performance if there is a large number of field

2018-05-11 Thread Erick Erickson
Deepak: I would strongly urge you to consider changing your problem solution to _not_ need 35,000 fields. What that usually indicates is that there are much better ways of tackling the problem. As Shawn says, 35,000 fields won't make much difference for an individual search. But 35,000 fields _do_

Re: Performance if there is a large number of field

2018-05-11 Thread Shawn Heisey
On 5/11/2018 9:26 AM, Andy C wrote: > Why are range searches more efficient than wildcard searches? I guess I > would have expected that they just provide different mechanism for defining > the range of unique terms that are of interest, and that the merge > processing would be identical. I hope I

Re: Performance if there is a large number of field

2018-05-11 Thread Deepak Goel
Deepak "The greatness of a nation can be judged by the way its animals are treated. Please stop cruelty to Animals, become a Vegan" +91 73500 12833 deic...@gmail.com Facebook: https://www.facebook.com/deicool LinkedIn: www.linkedin.com/in/deicool "Plant a Tree, Go Green" Make In India : http://

Re: Performance if there is a large number of field

2018-05-11 Thread Andy C
Shawn, Why are range searches more efficient than wildcard searches? I guess I would have expected that they just provide different mechanism for defining the range of unique terms that are of interest, and that the merge processing would be identical. Would a search such as: field:c* be more e

Re: Performance if there is a large number of field

2018-05-11 Thread Shawn Heisey
On 5/10/2018 2:22 PM, Deepak Goel wrote: Are there any benchmarks for this approach? If not, I can give it a spin. Also wondering if there are any alternative approach (i guess lucene stores data in a inverted field format) Here is the only other query I know of that can find documents missing

Re: Performance if there is a large number of field

2018-05-10 Thread Deepak Goel
On Fri, 11 May 2018, 01:15 Shawn Heisey, wrote: > On 5/10/2018 11:49 AM, Deepak Goel wrote: > > Sorry but I am unclear about - "What if there is no default value and the > > field does not contain anything"? What does Solr pass on to Lucene? Or is > > the field itself omitted from the document? >

Re: Performance if there is a large number of field

2018-05-10 Thread Shawn Heisey
On 5/10/2018 11:49 AM, Deepak Goel wrote: Sorry but I am unclear about - "What if there is no default value and the field does not contain anything"? What does Solr pass on to Lucene? Or is the field itself omitted from the document? If there is no default value and the field doesn't exist in w

Re: Performance if there is a large number of field

2018-05-10 Thread Deepak Goel
Deepak "The greatness of a nation can be judged by the way its animals are treated. Please stop cruelty to Animals, become a Vegan" +91 73500 12833 deic...@gmail.com Facebook: https://www.facebook.com/deicool LinkedIn: www.linkedin.com/in/deicool "Plant a Tree, Go Green" Make In India : http://

Re: Performance if there is a large number of field

2018-05-10 Thread Shawn Heisey
On 5/10/2018 10:58 AM, Deepak Goel wrote: I wonder what does Solr stores in the document for fields which are not being used. And if the queries have a performance difference https://lucene.apache.org/solr/guide/6_6/defining-fields.html (A default value that will be added automatically to any doc

Re: Performance if there is a large number of field

2018-05-10 Thread Deepak Goel
I wonder what does Solr stores in the document for fields which are not being used. And if the queries have a performance difference https://lucene.apache.org/solr/guide/6_6/defining-fields.html (A default value that will be added automatically to any document that does not have a value in this fie

Re: Performance if there is a large number of field

2018-05-10 Thread Shawn Heisey
On 5/10/2018 7:51 AM, Issei Nishigata wrote: I am designing a schema. I calculated the number of the necessary field as trial, and found that I need at least more than 35000. I do not use all these fields in 1 document. I use 300 field each document at maximum, and do not use remaining 34700 fie

Performance if there is a large number of field

2018-05-10 Thread Issei Nishigata
Hi, all I am designing a schema. I calculated the number of the necessary field as trial, and found that I need at least more than 35000. I do not use all these fields in 1 document. I use 300 field each document at maximum, and do not use remaining 34700 fields. Does this way of using it affec