bq. We're using NRTCachingDirectoryFactory Which uses MMapDirectory under the covers.
The file handle counts will vary. During merging, files are held open and while segments are merged so new and old segments are open. Once merged, the files in the old segment will be deleted so some variance is expected. What are your ulimits set at? I'm wondering if there's some weird reporting going on, which would be true if, say, your file handle limit were lower than what's being reported. Best, Erick On Tue, Sep 11, 2018 at 12:24 PM Boris Pasko <bpa...@exiger.com> wrote: > > On Tue, 2018-09-11 at 12:43 -0600, Shawn Heisey wrote: > > On 9/11/2018 12:14 PM, Boris Pasko wrote: > > > > > > > > > > > Run top, press shift-M to sort by memory usage, then grab a > > > atop: http://oi68.tinypic.com/10pokkk.jpg > > > top: http://oi63.tinypic.com/msbpfp.jpg > > Looking at the second one: > > > > The SHR value is showing 90GB. > > > Thanks! Thats a good catch. > > > > > > Looks like the amount of memory in the machine is too large for "top" > > to > 128Gb > > > > > It does look like you've got in the neighborhood of 600GB of index > > data. > 600-700Gb per node, correct. > > > Only 113GB of that data is cached. For some use cases, this will > > be plenty of cached data for good performance. For others, it won't > > be > > anywhere near enough. For *perfect* performance, there will be > > enough > > memory for ALL of the index data to fit into memory. > > > > Thanks, > > Shawn > > > > > ––––––––––––––––––––––––––––––––––––– > The information contained in this message and any attachments may be > confidential and/or restricted and protected from disclosure. If the reader > of this message is not the intended recipient, disclosure, copying, use, or > distribution of the information included in this message is prohibited - > please destroy all electronic and paper copies and notify the sender > immediately.
