1991santhu commented on PR #1640: URL: https://github.com/apache/iceberg-go/pull/1640#issuecomment-5184514909
Adding a measurement, since the case for this is better made as a number. I simulated the arrival times this backoff produces for 50 clients contending for the same table lock. Contention is the precondition for entering the retry loop at all, so those clients are waiting simultaneously by construction. The metric is the peak number of `CheckLock` calls landing inside any 10ms window at the metastore. Parameters are the defaults in `options.go`: `lock-check-min-wait-time=100ms`, `lock-check-max-wait-time=60s`, `lock-check-retries=4`, giving 100ms, 200ms, 400ms, 800ms. ``` attempt none equal added 1 50.0 16.1 10.3 2 50.0 9.7 6.5 3 50.0 6.2 5.0 4 50.0 4.7 3.3 ``` Without jitter all 50 clients re-check in the same 10ms window at every attempt, and the peak never decays, because a deterministic schedule preserves whatever correlation already exists. With the jitter in this PR the first re-check peaks at ~10 and falls to ~3, a **5x reduction rising to 15x**. The `equal` column is the equal-jitter variant. It decorrelates less — `[d/2, d]` is a narrower window than `[d, 2d]` — and it would poll sooner than `lock-check-min-wait-time` promises, which is why the jitter is added rather than centred. To be clear about what this is: a simulation of the arrival process, not a production report. I have not seen this happen against a real metastore and am not claiming anyone has; it quantifies the mechanism the code implies. The harness is dependency-free Go and I can attach it. ``` go run . -clients=50 -schedule=100ms,200ms,400ms,800ms -cap=60s -floor=100ms -window=10ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
