github-actions[bot] commented on code in PR #69045:
URL: https://github.com/apache/doris/pull/69045#discussion_r4266114347
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonTypeMapping.java:
##########
@@ -243,10 +240,11 @@ public static DataType toPaimonType(ConnectorType type) {
case "DOUBLE":
return new DoubleType();
case "CHAR":
+ // DDL must retain bounds: widening to STRING changes
historical narrowing casts.
+ return new CharType(type.getPrecision());
Review Comment:
[P2] Normalize CHAR values before writing the new CHAR schema
Returning `CharType(n)` also changes newly Doris-created tables from
VARCHAR(MAX) to real Paimon CHAR, but the Doris sink does not implement
Paimon's CHAR padding. With `k CHAR(3) NOT NULL` as a primary key, inserting
`'a'` passes the one-character string through `BindSink`, Arrow, and
`PaimonWriteSchema.tableRow`; the SDK extracts/stores that key without casting.
Paimon's [CHAR
cast](https://github.com/apache/paimon/blob/release-1.4.2/paimon-common/src/main/java/org/apache/paimon/utils/BinaryStringUtils.java#L330-L340)
produces `'a '`, and its [CHAR
comparator](https://github.com/apache/paimon/blob/release-1.4.2/paimon-codegen/src/main/scala/org/apache/paimon/codegen/GenerateUtils.scala#L103-L111)
compares the raw strings, so another writer using the typed CHAR value creates
a separate key instead of updating the Doris-written record. Please normalize
CHAR values before key/bucket extraction and storage (including nested CHAR),
and add a write/cast parity test with a short value
. The former VARCHAR(MAX) schema did not require this padding.
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonTypeMapping.java:
##########
@@ -243,10 +240,11 @@ public static DataType toPaimonType(ConnectorType type) {
case "DOUBLE":
return new DoubleType();
case "CHAR":
+ // DDL must retain bounds: widening to STRING changes
historical narrowing casts.
+ return new CharType(type.getPrecision());
case "VARCHAR":
+ return new VarCharType(type.getPrecision());
Review Comment:
[P1] Apply narrowing casts on the native Paimon read path
This mapping is also used by MODIFY COLUMN. For an append-only Parquet table
`t(s STRING)` containing `abcdef`, `ALTER TABLE t MODIFY COLUMN s VARCHAR(3)`
now persists the bound, but the default native read still returns `abcdef`:
`PaimonScanPlanProvider` routes raw files to native without checking character
casts, the v1 string converter is `ConsistentConverter`, and v2 treats string
types as equal regardless of length. Paimon's [historical
cast](https://github.com/apache/paimon/blob/release-1.4.2/paimon-common/src/main/java/org/apache/paimon/utils/BinaryStringUtils.java#L330-L340)
instead returns `abc` (also with `force_jni_scanner=true`). Previously this
ALTER mapped to VARCHAR(MAX), so the change exposes inconsistent results after
Doris DDL. Please apply Paimon character casts in native reads or route
affected splits through JNI, and cover ALTER/read parity with an end-to-end
test; the new direct CastExecutors assertion bypasses this path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
