Re: Improve indexing time

2009-07-13 Thread Noble Paul നോബിള്‍ नोब्ळ्
considering the fact that there are only 20 to 30 docs changed the indexing is not the bottleneck. Bottleneck is probably the db and the time taken for the query to run. Are there deltaQueries in the sub-entities? if you can create a 'VIEW' in DB to identify the delta it could be faster On Tue, Ju

Re: Improve indexing time

2009-07-13 Thread Glen Newton
Try using LuSql to create the index. It is 4-10 times faster on a multicore machine, and can run in 1/20th the heap size Solr needs. See slides 22-25 in this presentation comparing Solr DIH with LuSql: http://code4lib.org/files/glen_newton_LuSql.pdf LuSql: http://lab.cisti-icist.nrc-cnrc.gc.ca/ci

Improve indexing time

2009-07-13 Thread Gurjot Singh
Hi, We have a solr index of size 626 MB and number of douments indexed are 141810. We have configured index based spellchecker with buildOnCommit option set to true. Spellcheck index is of size 8.67 MB. We use data import handler to create the index from scratch and also to update the index period