I don't know who you are replying to here, but...
There's nothing to stop you doing:
* import 2m docs
* sleep 2 days
* import 2m docs
* sleep 2 days
* repeat above until done
* commit
There's no reason why you should commit regularly. If you need to slow
down for your DB, do, but that does
okay.
the query kills the database, because no index of modified is set ...
--
View this message in context:
http://lucene.472066.n3.nabble.com/Large-Hdd-Space-using-during-commit-optimize-tp1985807p1993750.html
Sent from the Solr - User mailing list archive at Nabble.com.
Solr doesn't lock anything as far as I know, it just executes the
query you specify. The query you specify may well do bad things
to your database, but that's not Solr's fault. What happens if you
simply try executing the query outside Solr? Do you see the
same "locking" behavior?
You might want t
aha aha :D
hm i dont know. we import in 2MillionSteps because we think that solr locks
our database and we want a better controll of the import ...
--
View this message in context:
http://lucene.472066.n3.nabble.com/Large-Hdd-Space-using-during-commit-optimize-tp1985807p1991392.html
Sent from
On Mon, 29 Nov 2010 08:43 -0800, "stockii" wrote:
>
> aha okay. thx
>
> i dont know that solr copys the complete index for optimize. can i solr
> say,
> that he start an optimize, but wihtout copy ?
No.
The copy is to keep an index available for searches while the optimise
is happening.
Also
aha okay. thx
i dont know that solr copys the complete index for optimize. can i solr say,
that he start an optimize, but wihtout copy ?
--
View this message in context:
http://lucene.472066.n3.nabble.com/Large-Hdd-Space-using-during-commit-optimize-tp1985807p1987477.html
Sent from the Solr -
First, don't optimize after every chunk, it's just making extra work for
your system.
If you're using a 3.x or trunk build, optimizing doesn't do much for you
anyway, but
if you must, just optimize after your entire import is done.
Optimizing will pretty much copy the old index into a new set of f
On Mon, 29 Nov 2010 03:07 -0800, "stockii" wrote:
>
> Hello.
>
> i have ~37 Million Docs that i want to index.
>
> when i starte a full-import i importing only every 2 Million Docs,
> because
> of better controll over solr and space/heap
>
> so when i import 2 million docs and solr sta
