jnioche opened a new pull request, #1855: URL: https://github.com/apache/stormcrawler/pull/1855
Fixes #1247
Both `okhttp/HttpProtocol` and `httpclient/HttpProtocol` have a race
condition in `getProtocolOutput()`: the shared field-level `builder` is mutated
with proxy and authentication settings on every request. When multiple fetcher
threads
use different proxies concurrently, interleaving causes:
- **Wrong proxy + auth pairing** — Thread A's proxy with Thread B's
credentials
- **Wrong proxy used entirely** — Thread B overwrites Thread A's proxy
before `builder.build()` is called
### Fix
- **OkHttp**: Create a per-request builder via `client.newBuilder()`
instead of mutating the shared `builder` field
- **HttpClient**: Create a local `HttpClientBuilder` and
`RequestConfig.Builder` copy for each proxied request instead of mutating the
shared fields
Non-proxy requests are unaffected — they continue using the pre-built
shared client.
## Test plan
- [x] Added `HttpProtocolProxyConcurrencyTest` that verifies the shared
builder is not polluted after a proxied request (fails without the fix, passes
with it)
- [x] All existing tests pass (`mvn test -pl core` — 214 tests, 0 failures)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
