Re: Question: index performance

2007-04-19 Thread Chris Hostetter
: solr, 100 documents at a time. I was doing a commit after each of those : but after what Yonik says I will remove it and commit only after each : batch of 25k. do the commit only when you think it's neccessary to expose those docs to your search clients, one of which may be "you" checking on th

Re: Question: index performance

2007-04-13 Thread galo
Hi there, I'm building an index to which I'm sending a few hundred thousand entries. I pull them off the database in batches of 25k and send them to solr, 100 documents at a time. I was doing a commit after each of those but after what Yonik says I will remove it and commit only after each ba

Re: Question: index performance

2007-04-13 Thread Yonik Seeley
On 4/13/07, James liu <[EMAIL PROTECTED]> wrote: i find it will be OutOfMemory when i get more that 10k records. so now i index 10k records( 5k / record) In one request? There's really no reason to put more than hundreds of documents in a single add request. If you are indexing using multipl