github-actions[bot] commented on code in PR #70121:
URL: https://github.com/apache/doris/pull/70121#discussion_r4455688190
##########
fe/fe-core/src/main/java/org/apache/doris/load/routineload/RoutineLoadJob.java:
##########
@@ -2047,8 +2054,24 @@ public void gsonPostProcess() throws IOException {
} else {
partialUpdateNewKeyPolicy =
TPartialUpdateNewRowPolicy.APPEND;
}
+ } else if
(k.equals(CreateRoutineLoadInfo.MAX_FILTER_RATIO_PROPERTY)) {
+ maxFilterRatio = Double.parseDouble(v);
+ } else if (k.equals(CreateRoutineLoadInfo.SEND_BATCH_PARALLELISM))
{
+ sendBatchParallelism = Integer.parseInt(v);
+ } else if (k.equals(CreateRoutineLoadInfo.LOAD_TO_SINGLE_TABLET)) {
+ loadToSingleTablet = Boolean.parseBoolean(v);
+ } else if (k.equals(CsvFileFormatProperties.PROP_ENCLOSE)) {
+ enclose = csvControlByte(k, v, enclose);
Review Comment:
[P2] Preserve non-ASCII CSV control bytes across replay. `setOptional`
persists `enclose`/`escape` as `new String(new byte[]{...})`, but this replay
path decodes that String again with `parseEnclose`/`parseEscape`. A valid
one-character value such as `é` yields byte `0xE9` on CREATE, becomes U+FFFD in
a UTF-8 String, and replays as `0xFD` (or `0xEF` for escape), so every task
after FE restart parses different CSV data. Persist/replay the byte losslessly
or use a canonical encoding, and add a high-bit round-trip test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
