On 10/29/2014 1:05 PM, Toke Eskildsen wrote: > We did have some problems on a 256GB machine churning terabytes of data > through 40 concurrent Tika processes and into Solr. After some days, > performance got really bad. When we did a top, we noticed that most of the > time was used in the kernel (the 'sy' on the '%Cpu(s):'-line). The > drop_caches trick worked for us too. Our systems guys explained that it was > because of virtual memory space fragmentation, so the OS had to spend a lot > of resources just bookkeeping memory.
There's always at least one exception to any general advice, including whatever I come up with! It's really too bad that it didn't Just Work (tm) for you. Weird things can happen when you start down the path of extreme scaling, though. Thank you for exploring the bleeding edge for us! Shawn
