[
https://issues.apache.org/jira/browse/SOLR-14202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Erickson updated SOLR-14202:
----------------------------------
Attachment: eoe.zip
Status: Open (was: Open)
Jörn:
I don't see it. I downloaded 8.4 and created a 2-shard, 2-replica collection
and ran the attached program against it.
The program first indexes 1M docs, then endlessly repeats updating those docs
with both a new update and an atomic update.
Autocommit is 15 seconds with openSearcher=true. There's also a thread that
fires a query at the collection just for completeness' sake.
After 12 cycles, the file counts are very close, I don't expect them to be
identical.
And the file counts are identical when I stopped the indexing program, took a
tally, then shut the Solr servers down and took another tally.
Do you have _any_ custom code running? The symptom you're reporting sounds an
awful lot like opening a searcher then not closing it on commit.
Anyway, if you can make it happen with the attached program, then maybe we can
figure out what's different between your setup and mine. I'm running on OS X
FWIW.
> Old segments are not deleted after commit
> -----------------------------------------
>
> Key: SOLR-14202
> URL: https://issues.apache.org/jira/browse/SOLR-14202
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud
> Affects Versions: 8.4
> Reporter: Jörn Franke
> Priority: Major
> Attachments: eoe.zip
>
>
> The data directory of a collection is growing and growing. It seems that old
> segments are not deleted. They are only deleting during start of Solr.
> How to reproduce. Have any collection (e.g. the example collection) and start
> indexing documents. Even during the indexing the data directory is growing
> significantly - much more than expected (several magnitudes). if certain
> documents are updated (without significantly increasing the amount of data)
> the index data directory grows again several magnitudes. Even for small
> collections the needed space explodes.
> This reduces significantly if Solr is stopped and then started. During
> startup (not shutdown) Solr purges all those segments if not needed (*
> sometimes some but not a significant amount is deleted during shutdown). This
> is of course not a good workaround for normal operations.
> It does not seem to have a affect on queries (their performance do not seem
> to change).
> The configs have not changed before the upgrade and after (e.g. from Solr 8.2
> to 8.3 to 8.4, not cross major versions), so I assume it could be related to
> Solr 8.4. It may have been also in Solr 8.3 (not sure), but not in 8.2.
>
> IndexConfig is pretty much default: Lock type: native, autoCommit: 15000,
> openSearcher=false, autoSoftCommit -1 (reproducible with autoCommit 5000).
> Nevertheless, it did not happen in previous versions of Solr and the config
> did not change.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
