How can I optimize Sorting on multiple text fields

2012-06-21 Thread Alok Bhandari
Hello, the requirement which I have is that on solr side we have indexed data of multiple customers and each customer we have at least a million documents. After executing search end user want to sort on some fields on datagrid lets say subject, title, date etc. Now as the sorting on text fields

RE: Exception using distributed field-collapsing

2012-06-21 Thread Bryan Loofbourrow
> Does a *:* query with no sorting work? Well, this is interesting. Leaving q= as it was, but removing the sort, makes the whole thing work. And if you were thinking of asking whether the sort field is a date, the answer is yes, it's an indexed and stored DateField. It's also on the list of field

Re: solr java.lang.NullPointerException on select queries

2012-06-21 Thread Erick Erickson
Right, if you optimize, at the end maxDocs should == numDocs. Usually the document reclamation stuff is done when segments merge, but that won't happen in this case since this index is becoming static, so a manual optimize is probably indicated. Something like this should also work, either way: ht

RE: Exception using distributed field-collapsing

2012-06-21 Thread Young, Cody
No, I believe it was a different exception, just brainstorming. (it was a null reference iirc) Does a *:* query with no sorting work? Cody -Original Message- From: Bryan Loofbourrow [mailto:bloofbour...@knowledgemosaic.com] Sent: Thursday, June 21, 2012 10:33 AM To: solr-user@lucene.ap

RE: Exception using distributed field-collapsing

2012-06-21 Thread Bryan Loofbourrow
Cody, > Does it work in the non distributed case? Yes. > > Is the field you're grouping on stored? What is the type on the uniqueKey > field? Is it stored and indexed? The field I'm grouping on is a string, stored and indexed. The unique key field is a string, stored and indexed. > I've had a

RE: Exception using distributed field-collapsing

2012-06-21 Thread Young, Cody
Does it work in the non distributed case? Is the field you're grouping on stored? What is the type on the uniqueKey field? Is it stored and indexed? I've had a problem with distributed not working when the uniqueKey field was indexed but not stored. Also, in distributed searches, the uniqueKey

Re: solr java.lang.NullPointerException on select queries

2012-06-21 Thread avenka
Erick, much thanks for detailing these options. I am currently trying the second one as that seems a little easier and quicker to me. I successfully deleted documents with IDs after the problem time that I do know to an accuracy of a couple hours. Now, the stats are: numDocs : 2132454075 maxDo

Solr 4.0 with Near Real Time and Faceted Search in Replicated topology

2012-06-21 Thread Niran Fajemisin
Hi all, We're thinking of moving forward with Solr 4.0 and we plan to have a master index server and at least two slaves servers. The Master server will be used primarily for indexing and the queries will be load balanced across to the replicated slave servers. I would like to know if, with the

Re: Commit when a segment is written

2012-06-21 Thread Erick Erickson
I don't think autocommit is deprecated, it's just commented out of the config and using commitWithin (assuming you're working from SolrJ) is preferred if possible. But what governs "a particular set of docs"? What are the criteria that determine when you want to commit? Flushes and commits are ort

Re: suggester/autocomplete locks file preventing replication

2012-06-21 Thread tom
pocking into the code i think the FileDictionary class is the culprit: It takes an InputStream as a ctor argument but never releases the stream. what puzzles me is that the class seems to allow a one-time iteration and then the stream is useless, unless i'm missing smth. here. is there a good

Re: suggester/autocomplete locks file preventing replication

2012-06-21 Thread tom
BTW: a core unload doesnt release the lock either ;( On 21.06.2012 14:39, tom wrote: hi, i'm using the suggester with a file like so: suggest name="classname">org.apache.solr.spelling.suggest.Suggester name="lookupImpl">org.apache.solr.spelling.suggest.fst.FSTLookup

suggester/autocomplete locks file preventing replication

2012-06-21 Thread tom
hi, i'm using the suggester with a file like so: suggest name="classname">org.apache.solr.spelling.suggest.Suggester name="lookupImpl">org.apache.solr.spelling.suggest.fst.FSTLookup content 0.05 true 100 autocomplete.d

Re: parameters to decide solr memory consumption

2012-06-21 Thread Erick Erickson
No, that's 255 bytes/record. Also, any time you store a field, the raw data is preserved in the *.fdt and *.fdx files. If you're thinking about RAM requirements, you must subtract the amount of data in those files from the total, as a start. This might help: http://lucene.apache.org/core/old_versi

Re: solrj and replication

2012-06-21 Thread tom
ok tested it myself and a slave runnning embedded works, just not within my application -- yet... On 20.06.2012 18:14, tom wrote: hi, i was just wondering if i need to do smth special if i want to have an embedded slave to get replication working ? my setup is like so: - in my clustered app

Re: write.lock

2012-06-21 Thread Dmitry Kan
Hi, We are running exactly same solr version and have these issues relatively frequently. The main cause in our case has usually been the out of memory exceptions, as some of our shards are pretty fat. Allocating more RAM usually helps for a while. The lock file needs to be manually removed still,

Re: Editing solr update handler sub class

2012-06-21 Thread Erick Erickson
H. I think you would have a _far_ easier time of this just getting all the source code, modifying the relevant source, and then just issuing an "ant dist" (or "ant example" if you wanted to try it). There are other targets that will package up the whole thing just like you would get it from the

LeaderElection

2012-06-21 Thread Trym R. Møller
Hi Messing with behaviour when Solr looses its ZooKeeper connection I'm trying to reproduce how a replica slice gets leader. I have made the below unit test in the LeaderElectionTest class which fails. I don't know if this simulates how Solr uses the LeaderElection class but please comment on

Re: Solr with Tomcat on VPS

2012-06-21 Thread mcfly04
Thanks for clarifying! I have the solr/home configured in the server.xml of Tomcat. When I had not set it properly there were errors in the log. It is configured correctly now as no errors are in the log regarding sorl/home. The issue is that I cannot access any of the Servlets. I can access a

how to import product of entity date with DIH

2012-06-21 Thread jueljust
i need to import data from sql server and cassandra first, get user ids from sql server then get one user's characters from cassandra by user id last, save the users characters doc into solr one user have multi charcters and i need to save the doc like 1040.txt 19734880.txt field un

Re: solr java.lang.NullPointerException on select queries

2012-06-21 Thread Erick Erickson
Ah, OK, I misunderstood. OK, here's a couple of off-the-top-of-my-head ideas. make a backup of your index before anything else ... Split up your current index into two parts by segments. That is, copy the whole directory to another place, and remove some of the segments from each. I.e. when you'r

Re: Solr with Tomcat on VPS

2012-06-21 Thread irshad siddiqui
Hi, you can also refer below url for solr configuration http://tek-manthan.blogspot.in/ Regards, Irshad On Thu, Jun 21, 2012 at 4:24 PM, mcfly04 wrote: > Thank you for your response. > > Are you referring to the SolrRequestFilter path-prefix? > Here is a copy of my web.xml: > > - > > >

Re: Solr with Tomcat on VPS

2012-06-21 Thread irshad siddiqui
HI, in this web.xml file you need to add below line solr/home your solr core path here java.lang.String On Thu, Jun 21, 2012 at 4:24 PM, mcfly04 wrote: > Thank you for your response. > > Are you referring to the SolrRequestFilter path-prefix? > Here is a copy o

Re: Solr with Tomcat on VPS

2012-06-21 Thread mcfly04
Thank you for your response. Are you referring to the SolrRequestFilter path-prefix? Here is a copy of my web.xml: - http://java.sun.com/dtd/web-app_2_3.dtd";> SolrRequestFilter org.apache.solr.servlet.SolrDispatchFilter SolrRequestF

Re: Solr with Tomcat on VPS

2012-06-21 Thread irshad siddiqui
Hi, You have to check installation configuration. Inside tomcat webapps folder you have putted solr.war file. and within this WEB-INF folder web.xml you have to check your solr core folder url On Thu, Jun 21, 2012 at 3:34 PM, mcfly04 wrote: > Can anyone help? > > -- > View this message i

Re: Solr with Tomcat on VPS

2012-06-21 Thread mcfly04
Can anyone help? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-with-Tomcat-on-VPS-tp3990397p3990677.html Sent from the Solr - User mailing list archive at Nabble.com.