If cpu is just 50% and adding a shard does increase indexing throughput then check for disk bottleneck. On Sep 17, 2015 18:19, "Zheng Lin Edwin Yeo" <edwinye...@gmail.com> wrote:
> Thank you everyone for your reply. > > > How many CPUs on that machine? How many other requests using the server? > > A) There's 8 CPU on the machine, and there is no other requests that's > using the server. Only the indexing script is running. > > > A simple metric is to look at CPU usage on the machine: If it is near > 100% when you index, you will need extra hardware to get more speed. > If it is substantially less than 100%, then feed Solr from more than one > thread at a time. > > A) So far from what I observe, the CPU usage is usually around 50% to 70%. > It haven't go up to 100% yet. But I'll probably try to do sharing on a > different machine, as that is probably the case for the real production > server. > > > Regards, > Edwin > > > On 17 September 2015 at 19:55, Toke Eskildsen <t...@statsbiblioteket.dk> > wrote: > > > On Thu, 2015-09-17 at 16:58 +0800, Zheng Lin Edwin Yeo wrote: > > > > > I was trying with 2 shards and 4 shards but all on the same machine, > > > and they have the same performance (no improvement in performance) as > > > the one with 1 shard. My machine has a 32GB RAM. > > > > As you are testing indexing speed, Shalin's post is spot-on: Sharding on > > the same machine won't help you. I just added my comment on search to > > help build a complete picture. > > > > A simple metric is to look at CPU usage on the machine: If it is near > > 100% when you index, you will need extra hardware to get more speed. > > If it is substantially less than 100%, then feed Solr from more than one > > thread at a time. > > > > - Toke Eskildsen, State and University Library, Denmark > > > > > > > > >
