[
https://issues.apache.org/jira/browse/IGNITE-25286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18002685#comment-18002685
]
Roman Puchkovskiy commented on IGNITE-25286:
--------------------------------------------
Terminology change: in PR, it's 'schema safe time' instead of 'catalog safe
time' used in the ticket description.
> Only concern Catalog updates application for Schema sync
> --------------------------------------------------------
>
> Key: IGNITE-25286
> URL: https://issues.apache.org/jira/browse/IGNITE-25286
> Project: Ignite
> Issue Type: Improvement
> Reporter: Roman Puchkovskiy
> Assignee: Roman Puchkovskiy
> Priority: Major
> Labels: ignite-3
>
> Currently, schema sync mechanism makes sure that AppliedMetastoreSafeTime
> >=(operationTs - DD) for each operation (including KV operations and SQL).
> AppliedMetastoreSafeTime is updated when the corresponding Metastore command
> is both applied to the Metastore storage and all watch listeners fully
> handled the revision corresponding to the command. Some watch listeners
> perform long actions (like creating storages (this involves I/O) and starting
> replicas (involves I/O and time to elect a leader); as a result,
> read/write/SQL operations might be held by these long operations.
> Schema sync mechanism actually only concerns Catalog updates (as the Catalog
> stores schemas to sync on); this mechanism was implemented to make sure that,
> when processing a transactional operation, we don't miss a schema update
> relevant to this operation. Schemas are rarely updated. If we only concern
> safe time of Catalog updates, long-running actions like storage creation and
> replica starts will not affect transactional operations.
> We could do the following:
> # Introduce another tracker called Catalog safe time
> # Inside WatchProcessor, have a chained future called
> catalogSafeTimeUpdateFuture
> # When notifying the WatchProcessor about a Metastore command (including
> Metastore Idle Safe Time command):
> ## If the command is a Catalog update, then, after reassigning
> notificationFuture, do catalogSafeTimeUpdateFuture =
> catalogSafeTimeUpdateFuture.thenCompose(v -> notificationFuture)
> ## Do catalogSafeTimeUpdateFuture = catalogSafeTimeUpdateFuture.thenRun(()
> -> <update Catalog safe time with the command safe time>)
> # In SchemaSyncServiceImpl, use Catalog safe time tracker instead of raw
> Metastore safe time tracker
> # Make sure that everywhere we use SchemaSyncService, we are actually only
> concerned about Catalog updates. If there are places where we are concerned
> about other Metastore updates, we should introduce another service/method
> that would still use the original Metastore safe time tracker
> Note that item 3.2, for non-Catalog updates, we update the Catalog safe time
> tracker with a safe time value that could not yet be handled by the Watch
> listeners. This will allow the schema sync mechanism to ignore delays
> introduced by listeners not related to Catalog updates.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
