Some further information: The main things use memory that I see from my heap dump are:
1. Arrays of org.apache.lucene.util.fst.FST$Arc classes- which mainly seem to hold nulls. The ones of these I've investigated have been held by org.apache.lucene.util.fst.FST objects, I have 38 cores open and have over 121,000 of these arrays, taking up over 126M of space. 2. Byte arrays, of which I have 384,000 of, taking up 106M of space. When I trace the cycle of references up, I've always ended up at an IndexSearcher or IndexWriter class, causing me to assume the problem was that I was simply opening up too many cores, but I could be mistaken. This was on a freshly started system without many cores having been touched yet- so the memory usage, while larger than I expect, isn't critical yet. It does become critical as the number of cores increases. Thanks, Brian On Mon, Aug 3, 2015 at 4:57 PM, Brian Hurt <bhur...@gmail.com> wrote: > > Is there are an easy way for a client to tell Solr to close or release the > IndexSearcher and/or IndexWriter for a core? > > I have a use case where we're creating a lot of cores with not that many > documents per zone (a few hundred to maybe 10's of thousands). Writes come > in batches, and reads also tend to be bursty, if less so than the writes. > > And we're having problems with ram usage on the server. Poking around a > heap dump, the problem is that every IndexSearcher or IndexWriter being > opened is taking up large amounts of memory. > > I've looked at the unload call, and while it is unclear, it seems like it > deletes the data on disk as well. I don't want to delete the data on disk, > I just want to unload the searcher and writer, and free up the memory. > > So I'm wondering if there is a call I can make when I know or suspect that > the core isn't going to be used in the near future to release these objects > and return the memory? Or a configuration option I can set to do so after, > say, being idle for 5 seconds? It's OK for there to be a performance hit > the first time I reopen the core. > > Thanks, > > Brian > >
