In a word, "no". I once doubled the JVM requirements by changing just the query. You have to prototype. Here's a blog on the subject:
https://lucidworks.com/blog/2012/07/23/sizing-hardware-in-the-abstract-why-we-dont-have-a-definitive-answer/ On Wed, Oct 28, 2015 at 11:06 AM, Salman Ansari <salman.rah...@gmail.com> wrote: > I have already indexed all the documents in Solr and not indexing anymore. > So the problem I am running in is after all the documents are indexed. I am > using Solr cloud with two shards and two replicas for each shard but on the > same machine. Is there anywhere I can look at the relation between index > size and machine specs and its effect on Solr query performance? > > Regards, > Salman > > On Mon, Oct 26, 2015 at 5:55 PM, Upayavira <u...@odoko.co.uk> wrote: > >> >> >> On Sun, Oct 25, 2015, at 05:43 PM, Salman Ansari wrote: >> > Thanks guys for your responses. >> > >> > That's a very very large cache size. It is likely to use a VERY large >> > amount of heap, and autowarming up to 4096 entries at commit time might >> > take many *minutes*. Each filterCache entry is maxDoc/8 bytes. On an >> > index core with 70 million documents, each filterCache entry is at least >> > 8.75 million bytes. Multiply that by 16384, and a completely full cache >> > would need about 140GB of heap memory. 4096 entries will require 35GB. >> > I don't think this cache is actually storing that many entries, or you >> > would most certainly be running into OutOfMemoryError exceptions. >> > >> > True, however, I have tried with the default filtercache at the beginning >> > but the problem was still there. So, I don't think that is how I should >> > increase the performance of my Solr. Moreover, as you mentioned, when I >> > change the configuration, I should be running out of memory but that did >> > not happen. Do you think my Solr has not taken the latest configs? I have >> > restarted the Solr btw. >> > >> > Lately I have been trying different ways to improve this and I have >> > created >> > a brand new index on the same machine using 2 shards and it had few >> > entries >> > (about 5) and the performance was booming, I got the results back in 42 >> > ms >> > sometimes. What concerns me is that may be I am loading too much into one >> > index so that is why this is killing the performance. Is there a >> > recommended index size/document number and size that I should be looking >> > at >> > to tune this? Any other ideas other than increasing the memory size as I >> > have already tried this? >> >> The optimal index size is down to the size of segments on disk. New >> segments are created when hard commits occur, and existing on-disk >> segments may get merged in the background when the segment count gets >> too high. Now, if those on-disk segments get too large, copying them >> around at merge time can get prohibitive, especially if your index is >> changing frequently. >> >> Splitting such an index into shards is one approach to dealing with this >> issue. >> >> Upayavira >>
