Nice. Will port it onto an SSD.
A have a few questions about optimize. Is the search index fully searchable after a commit? How much time does one have to wait in case of a hard commit for the index to be available? I have an index of 180G. Do I need to hit the optimize on this chunk. This is a single core. Say I cannot get in a cloud env because of cost but this is a fairly large amazon machine where I have given SOLR 12G of memory. In context of my index if I had say 20G more data per month onto it. how much time before it is fully available for search? And when should I hit the optimize button? Thanks Sid. On Tue, Oct 6, 2015 at 6:55 AM, Toke Eskildsen <t...@statsbiblioteket.dk> wrote: > On Mon, 2015-10-05 at 17:26 -0400, Siddhartha Singh Sandhu wrote: > > Following up on that: Would having an SSD make considerable difference in > > speed? > > Yes, but only to a point. > > The UK Web Archive has done some tests on optimizing indexes on both > spinning drives and SSDs: > https://github.com/ukwa/shine/tree/master/python/test-logs > > With spinning drives, their machines maxed out on IOWait. With SSD, the > machine maxed out on CPU. That might sound great, but the problem is > that optimizing on a single shard is single threaded (at least for Solr > 4.10.x), so if there is only a single shard on the machine, only 1 CPU > is running at full tilt. There is always a bottleneck. > > What might help is that the SSD (probably) does not get bogged down by > the process, so it should be much better at handling other requests > while the optimization is running. > > - Toke Eskildsen, State and University Library, Denmark > > >
