Re: Index growing and growing until restart

2020-01-21 Thread Jörn Franke
Ok, i created collection from scratch based on config Unfortunately, it does not improve. It is just growing and growing. Except when I stop solr and then during startup the unnecessary index files are purged. Even with the previous config this did not happen in older Solr versions (for sure not i

Re: Index growing and growing until restart

2020-01-21 Thread Jörn Franke
After testing the update?commit=true i now face an error: "Maximum lock count exceeded". strange this is the first time i see this in the lockfiles and when doing commit=true ava.lang.Error: Maximum lock count exceeded at java.base/java.util.concurrent.locks.ReentrantReadWriteLock$Sync.fullTryA

Re: Index growing and growing until restart

2020-01-21 Thread Jörn Franke
The only weird thing is I see that for instance I have ${solr.autoCommit.maxTime:15000} and similar entries. It looks like a template gone wrong, but this was not caused due to an internal development. It must have been come from a Solr version. On Tue, Jan 21, 2020 at 10:49 PM Jörn Franke wrote

Re: Index growing and growing until restart

2020-01-21 Thread Jörn Franke
It is btw. a Linux system and autosoftcommit is set to -1. However, indeed openSearcher is set to false. A commit is set to true after doing all the updates, but the index is not shrinking. The files are not disappearing during shutdown, but they disappear after starting up again. On Tue, Jan 21,

Re: Index growing and growing until restart

2020-01-21 Thread Jörn Franke
thanks for the answer I will look into it - it is a possible explanation. > Am 20.01.2020 um 14:30 schrieb Erick Erickson : > > Jörn: > > The only thing I can think of that _might_ cause this (I’m not all that > familiar with the code) is if your solrconfig settings never open a searcher. >

Re: Index growing and growing until restart

2020-01-20 Thread Erick Erickson
Jörn: The only thing I can think of that _might_ cause this (I’m not all that familiar with the code) is if your solrconfig settings never open a searcher. Either you need to be sure openSearcher is set to true in the autocommit section in solrconfig.xml or your autoSoftCommit is set to somethi

Re: Index growing and growing until restart

2020-01-20 Thread Jörn Franke
From what is see it basically duplicates the index files, but does not delete the old ones. It uses caffeine cache. What I observe is that there is an exception when shutting down for the collection that is updated - timeout waiting for all directory ref counts to be released - gave up waiting

Re: Index growing and growing until restart

2020-01-20 Thread Jörn Franke
Sorry I missed a line - not tlog is growing but the /data/index folder is growing - until restart when it seems to be purged. > Am 20.01.2020 um 10:47 schrieb Jörn Franke : > > Hi, > > I have a test system here with Solr 8.4 (but this is also reproducible in > older Solr versions), which has

Index growing and growing until restart

2020-01-20 Thread Jörn Franke
Hi, I have a test system here with Solr 8.4 (but this is also reproducible in older Solr versions), which has an index which is growing and growing - until the SolrCloud instance is restarted - then it is reduced tot the expected normal size. The collection is configured to do auto commit afte