morningman commented on code in PR #70312:
URL: https://github.com/apache/doris/pull/70312#discussion_r4489157426
##########
regression-test/suites/compaction/test_base_compaction_no_value.groovy:
##########
@@ -81,6 +81,7 @@ suite("test_base_compaction_no_value", "p2") {
file """${getS3Url()}/regression/tpch/sf1/lineitem.csv.split01.gz"""
time 10000 // limit inflight 10s
+ retryIfHttpError true
Review Comment:
Not changed in this PR. The reader does accept `200` and any `206` on a
reopen, but this PR does not touch it, and that response cannot occur where
these suites run.
- `ResumableHttpInputStream` comes from #54613 and is unchanged here: this
PR only turns on `retryIfHttpError` in 35 suites, and four master suites
already use it.
- Every pipeline config in the repo sets `s3Endpoint =
"oss-cn-hongkong-internal.aliyuncs.com"`, `getS3Url()` connects to it directly,
and `StreamLoadAction` builds its HttpClient without a proxy. OSS answers a
range inside the object, such as `bytes=500-` on a 1,000-byte object, with
`206` and `Content-Range: bytes 500-999/1000`; it ignores `Range` and returns
the whole object with `200` only when the range is out of bounds ([OSS
docs](https://www.alibabacloud.com/help/en/oss/how-to-obtain-oss-resources-by-segmenting-http-range-requests)).
- The reader only asks for ranges inside the object. The body is
Content-Length delimited: once every byte is read, `ContentLengthInputStream`
returns -1 without reading the socket again, so an I/O error, and with it a
reopen, happens only while `offset` is below the object size.
The consequence is also narrower than described. 35 of the 45 `streamLoad`
blocks, this one included, load gzip: bytes from the start of the object after
a consumed prefix break the deflate stream and the gzip CRC, so the load fails.
The nine group-commit blocks that read plain `lineitem.tbl.N` compare
`NumberTotalRows` with a fixed row count per file, so duplicated input fails as
well. Of the plain-text ones, only `create_commit_mtmv_many_task` compares
nothing but total and loaded rows, and it could pass only if the break fell
exactly on a line boundary.
Rejecting a non-ranged reopen in the framework would change the input path
of all 39 suites that use the option, so it belongs in a separate framework
change rather than in this port of #65337.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
