[
https://issues.apache.org/jira/browse/SOLR-14778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Megan Carey updated SOLR-14778:
-------------------------------
Description:
Solr currently "supports" disabling the UpdateLog, though it is "required" for
NRT replicas (per the [docs|#transaction-log]]). However, when the update log
is disabled and a replica is in BUFFERING state (e.g. during MigrateCmd or
SplitShardCmd), updates are *lost silently*. While most users will likely never
consider disabling the updateLog, it seems pertinent to provide a better
support option.
Options as discussed in [ASF
Slack|[https://the-asf.slack.com/archives/CEKUCUNE9/p1598373062262300]]:
# No longer support disabling the updateLog as it is considered a pertinent
feature in SolrCloud. This might be undesirable for use cases where some data
loss is acceptable and the updateLog takes up too much space.
# Improve Solr documentation to explicitly outline the risks of disabling the
updateLog.
# Add logging to indicate when an update is swallowed in this state.
# _My preferred option:_ Support disabling the updateLog by providing
additional replica states besides BUFFERING, so that there is no data loss when
updateLog is disabled and replica goes offline for an operation like split.
Some ideas:
## REJECTING: Fail updates so that the client can retry again once the
operation is complete.
## BLOCKING: Stall update until operation is complete, and then execute update.
Feedback is welcome; once we establish a path forward I'd be happy to pick it
up. If others are interested I can document my findings as well.
was:
Solr currently "supports" disabling the UpdateLog, though it is "required" for
NRT replicas (per the
[docs|[https://lucene.apache.org/solr/guide/8_6/updatehandlers-in-solrconfig.html#transaction-log]]).
However, when the update log is disabled and a replica is in BUFFERING state
(e.g. during MigrateCmd or SplitShardCmd), updates are *lost silently*. While
most users will likely never consider disabling the updateLog, it seems
pertinent to provide a better support option.
Options as discussed in [ASF
Slack|[https://the-asf.slack.com/archives/CEKUCUNE9/p1598373062262300]:]
# No longer support disabling the updateLog as it is considered a pertinent
feature in SolrCloud. This might be undesirable for use cases where some data
loss is acceptable and the updateLog takes up too much space.
# Improve Solr documentation to explicitly outline the risks of disabling the
updateLog.
# Add logging to indicate when an update is swallowed in this state.
# _My preferred option:_ Support disabling the updateLog by providing
additional replica states so that there is no data loss when updateLog is
disabled and replica goes offline for an operation like split.
## REJECTING: Fail updates so that the client can retry again once the
operation is complete.
## BLOCKING: Stall update until operation is complete, and then execute update.
Feedback is welcome; once we establish a path forward I'll be happy to pick it
up. If others are interested I can document my findings as well.
> Disabling UpdateLog leads to silently lost updates
> --------------------------------------------------
>
> Key: SOLR-14778
> URL: https://issues.apache.org/jira/browse/SOLR-14778
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: SolrCloud, update
> Affects Versions: 8.6.1
> Reporter: Megan Carey
> Priority: Minor
>
> Solr currently "supports" disabling the UpdateLog, though it is "required"
> for NRT replicas (per the [docs|#transaction-log]]). However, when the update
> log is disabled and a replica is in BUFFERING state (e.g. during MigrateCmd
> or SplitShardCmd), updates are *lost silently*. While most users will likely
> never consider disabling the updateLog, it seems pertinent to provide a
> better support option.
> Options as discussed in [ASF
> Slack|[https://the-asf.slack.com/archives/CEKUCUNE9/p1598373062262300]]:
> # No longer support disabling the updateLog as it is considered a pertinent
> feature in SolrCloud. This might be undesirable for use cases where some data
> loss is acceptable and the updateLog takes up too much space.
> # Improve Solr documentation to explicitly outline the risks of disabling
> the updateLog.
> # Add logging to indicate when an update is swallowed in this state.
> # _My preferred option:_ Support disabling the updateLog by providing
> additional replica states besides BUFFERING, so that there is no data loss
> when updateLog is disabled and replica goes offline for an operation like
> split. Some ideas:
> ## REJECTING: Fail updates so that the client can retry again once the
> operation is complete.
> ## BLOCKING: Stall update until operation is complete, and then execute
> update.
> Feedback is welcome; once we establish a path forward I'd be happy to pick it
> up. If others are interested I can document my findings as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
