SergeyTimofeev1989 commented on issue #18608:
URL: https://github.com/apache/iceberg/issues/18608#issuecomment-6363594573
> 2\. I'm really confused on the expectation here. Why would changing the
isolation level matter for appends at all?
> > Our team set write.delete.isolation-level = snapshot on these
tables, expecting it to reduce
> > commit conflicts or commit duration.
>
>
> This doesn't make sense to me ... why would it do so?
Fair question, and you're right: it doesn't, and it shouldn't. We're not
asking for a behavior change, only for the docs to say this, because as written
they led us to expect the opposite. Two things caused it:
1. "Isolation" means two different things in the docs. The Reliability page
lists serializable isolation as a table guarantee ("All table changes occur in
a linear history of atomic table updates"). The Configuration page then offers
`write.delete.isolation-level` with default `serializable` and the description
"Isolation level for delete commands: serializable or snapshot". Read together,
it looks like a table-level switch for that guarantee, which would cover
appends too.
2. On our tables it did have an effect, just not on the side we expected.
Flink appends and Spark DELETEs (tiering and TTL jobs) commit to the same
tables concurrently. After switching to `snapshot`, the Spark DELETEs mostly
stopped failing with `Found conflicting files`, but the Flink side didn't
change at all: same 409s in production, same commit duration in our test
environment. That's when we went to the code.
What would have saved us the detour is one sentence saying which side of a
concurrent commit these properties apply to, for example:
> Applies only to the DELETE, UPDATE or MERGE command being committed: it
controls whether data files added by concurrent commits (including appends)
count as conflicts. It has no effect on how appends are committed; an append
that loses a commit race is retried according to `commit.retry.*`.
Happy to open a PR with that wording. If you don't think it's worth it, feel
free to close the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
