Using a second machines , you will dispose of fresh memory, disk and CPUs. So assuming you succeeded in saturating the first machine indexing power, of course it is normal that you improve your indexing time giving an additional node serving the indexing process. Are you balancing 50:50 or with some compositeId strategy(which anyway will try to balance if possible) ?
Cheers 2015-09-23 15:33 GMT+01:00 Zheng Lin Edwin Yeo <edwinye...@gmail.com>: > I've tried to run different shards from different machine, and there is a > slight improvement in the performance (about 3 mins faster for 1GB worth of > data, from 22 mins to 19 mins). > > Is this a normal scenario? Both of my machine are running on Intel i7 core. > > > Regards, > Edwin > > > On 21 September 2015 at 16:24, Zheng Lin Edwin Yeo <edwinye...@gmail.com> > wrote: > > > I'm not sure if that is because currently my machine is a normal PC and > > not a server, but my CPU specification for each of the core is Intel(R) > > Core(TM) i7-4910MQ CPU @ 2.90GHz. > > > > It should probably be better when the real server which has a much better > > specification comes, and I should be able to do the indexing in a lesser > > time using the knowledge that I've learnt here. > > > > > > Regards, > > Edwin > > > > > > > > On 21 September 2015 at 16:00, Toke Eskildsen <t...@statsbiblioteket.dk> > > wrote: > > > >> On Mon, 2015-09-21 at 10:13 +0800, Zheng Lin Edwin Yeo wrote: > >> > I didn't find any increase in indexing throughput by adding shards in > >> the > >> > same machine. > >> > > >> > However, I've managed to feed the index to Solr from more than one > >> thread > >> > at a time. It can take up to 3 threads without affecting the indexing > >> > speed. Anything more than that, the CPU will hit 100%, and the > indexing > >> > speed in all the threads will be reduced. > >> > >> It is a bit surprising that the limit is 3 Threads on an 8 core machine, > >> but I am happy to hear that your findings fit the overall theory. > >> > >> > >> Thank you for the verification, > >> Toke Eskildsen, State and University Library, Denmark > >> > >> > >> > > > -- -------------------------- Benedetti Alessandro Visiting card - http://about.me/alessandro_benedetti Blog - http://alexbenedetti.blogspot.co.uk "Tyger, tyger burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry?" William Blake - Songs of Experience -1794 England
