Re: Too many fields to Sort in Solr

2013-03-29 Thread adityab
Joel, thanks for your excellent idea using docValues. its working exactly as you described. So far my unit test case has no issues and i see low memory foot print. Will be sending the build for performance that should give comparable numbers. Now i see another replication issue in 4.2. there is

Re: Too many fields to Sort in Solr

2013-03-29 Thread Joel Bernstein
OK, that makes sense. How are DocValues working for you? On Fri, Mar 29, 2013 at 9:02 AM, adityab wrote: > Hi Joel, > Might have an answer for this. Initially my servers were on 3.5 and then i > moved to Solr 4.0. at this time i use the solrconfig.xml that was in the > example and updated is wi

Re: Too many fields to Sort in Solr

2013-03-29 Thread adityab
Hi Joel, Might have an answer for this. Initially my servers were on 3.5 and then i moved to Solr 4.0. at this time i use the solrconfig.xml that was in the example and updated is with parameters i changed in 3.5 for the environment. there was no "" in the 4.0 example solrconfig.xml file. We conti

Re: Too many fields to Sort in Solr

2013-03-28 Thread Joel Bernstein
Not, sure that making changes to the solrconfig.xml is going down the right path here. There might something else with your setup that's causing this issue. I'm not sure what it would be though. On Thu, Mar 28, 2013 at 1:38 PM, adityab wrote: > Wo. that's strange. > > I tried toggling with the

Re: Too many fields to Sort in Solr

2013-03-28 Thread adityab
Wo. that's strange. I tried toggling with the code factory line in solrconfig.xml (attached in this post) commenting gives me error where as un-commenting works. can you please take a look into config and let me know if anything wrong there? thanks Aditya solrconfig.xml

Re: Too many fields to Sort in Solr

2013-03-28 Thread Joel Bernstein
I didn't have to do anything with the codecs to make it work. Checked my solrconfig.xml and the codecFactory element is not present. I'm running the out of the box jetty setup. On Thu, Mar 28, 2013 at 11:58 AM, adityab wrote: > Update --- > > I was able to fix the exception by adding following

Re: Too many fields to Sort in Solr

2013-03-28 Thread adityab
Update --- I was able to fix the exception by adding following line in solrconfig.xml Not sure if its mentioned in any document to have this declared in config file. I am now re-indexing and data on the master and will perform test to see if it works as expected. thanks for your support. A

Re: Too many fields to Sort in Solr

2013-03-28 Thread adityab
still no luck Performed. 1. Stop the Application Server (JBoss) 2. Deleted everything under data 3. Star the server 4. Observe exception in log (i have uploaded the file) on a side note. do i need to have any additional jar files in the solr home lib folder. currently its empty. docValueExce

Re: Too many fields to Sort in Solr

2013-03-28 Thread Joel Bernstein
OK, you'll need to re-index. Shutdown, delete the data, re-index. On Thu, Mar 28, 2013 at 9:12 AM, adityab wrote: > Here is the field type definition. same as what you posted yesterday just a > different name. > > docValuesFormat="Disk" positionIncrementGap="0"/> > > And Field Definition > st

Re: Too many fields to Sort in Solr

2013-03-28 Thread adityab
Here is the field type definition. same as what you posted yesterday just a different name. And Field Definition as soon as i restart the server i see the exception in log. removing the *docValuesFormat="Disk"* from the field type i don't see this exception. 01:49:37,177 ERROR [org.apache

Re: Too many fields to Sort in Solr

2013-03-28 Thread Joel Bernstein
Hi, I tested this config on Solr 4.2 this morning and it worked: I also loaded data and ran a sort and looked at the heap with jvisualvm and the longs were not loaded into the jvm's heap. The sort was also very fast, although only on 600,000 records. Possibly you are not on Solr 4.2? Can you

Re: Too many fields to Sort in Solr

2013-03-27 Thread adityab
Hi Joel, you are correct, boost function populates the field cache. Well i am not aware of docValue, so while trying the example you provided i see the error when i define the field type Caused by: org.apache.solr.common.SolrException: FieldType 'dvLong' is configured with a docValues format, bu

Re: Too many fields to Sort in Solr

2013-03-26 Thread Joel Bernstein
I pretty sure that the boost function is going load up the fieldCache for that field, which will have the same memory footprint as sorting. You may want to try out disk based doc values (new 4.2). Documentation is sparse on this but you can specify a field as being docValues="true" and set it to a

Re: Too many fields to Sort in Solr

2013-03-25 Thread Erick Erickson
Certainly that will be true for the bare q=*:*, I meant with the boosting clause added. Best Erick On Sun, Mar 24, 2013 at 7:01 PM, adityab wrote: > thanks Eric. in this query "q=*:*" the Lucene score is always 1 > > > > -- > View this message in context: > http://lucene.472066.n3.nabble.com/T

Re: Too many fields to Sort in Solr

2013-03-24 Thread adityab
thanks Eric. in this query "q=*:*" the Lucene score is always 1 -- View this message in context: http://lucene.472066.n3.nabble.com/Too-many-fields-to-Sort-in-Solr-tp4049139p4050944.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Too many fields to Sort in Solr

2013-03-24 Thread Erick Erickson
Seems like a reasonable thing to do. Examine the debug output to insure that there's no short-circuiting being done as far as ConstantScoreQuery... Best Erick On Tue, Mar 19, 2013 at 7:05 PM, adityab wrote: > Hi All, > > I want to validate my approach by the experts, just to make sure i am on