On 2/23/07, Yonik Seeley <[EMAIL PROTECTED]> wrote:
On 2/23/07, Brian Whitman <[EMAIL PROTECTED]> wrote:
> I'm trying to add lots of documents at once (hundreds of thousands)
> in a loop. I don't need these docs to appear as results until I'm
> done, though.
>
> For a simple test, I call the post.sh script in a loop with the same
> moderately sized xml file. This adds a 20K doc and then commits.
> Repeat hundreds of thousands of times.
Try not committing so often (perhaps until you are done).
Don't use post.sh, or modify it to remove the commit.
Part of the problem might be repeatedly inserting the same doc over
and over again-- that is an odd pattern of deletes, which might be
triggering a bad performance case on the lucene or solr side (I'm
assuming the doc has a unique key).
-Mike
