morningman commented on code in PR #69045:
URL: https://github.com/apache/doris/pull/69045#discussion_r4267754157
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonTypeMapping.java:
##########
@@ -243,10 +240,11 @@ public static DataType toPaimonType(ConnectorType type) {
case "DOUBLE":
return new DoubleType();
case "CHAR":
+ // DDL must retain bounds: widening to STRING changes
historical narrowing casts.
+ return new CharType(type.getPrecision());
case "VARCHAR":
+ return new VarCharType(type.getPrecision());
Review Comment:
The difference is real, but it is how Doris's native readers already treat
the CHAR/VARCHAR bounds of external tables, not something this PR adds. They
apply the table's length only when `truncate_char_or_varchar_columns` is on:
the format_v2 `TableReader` handles exactly this case (a table VARCHAR(n) over
a wider or STRING file column, `_should_truncate_char_or_varchar_column`), and
Hive and Iceberg tables behave the same. Master already reads Paimon VARCHAR(n)
as VARCHAR(n), so a column narrowed by Spark or Flink gives the same native/JNI
difference today, and with the session variable on the native read returns
`abc`. This PR only makes Doris DDL store the declared type, as branch-4.1 and
branch-4.2 do. Making native reads follow Paimon's casts by default would
change those existing tables as well, so I'd keep it out of this PR. I updated
the description, which said older values are truncated without mentioning the
native default.
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonTypeMapping.java:
##########
@@ -243,10 +240,11 @@ public static DataType toPaimonType(ConnectorType type) {
case "DOUBLE":
return new DoubleType();
case "CHAR":
+ // DDL must retain bounds: widening to STRING changes
historical narrowing casts.
+ return new CharType(type.getPrecision());
Review Comment:
Confirmed: neither write backend pads CHAR. `PaimonWriteSchema.tableRow`
(JNI) and the FFI backend pass the string through as Doris holds it. That is
already the case on master for CHAR(n) columns of tables created by Spark or
Flink, which Doris reads as CHAR(n) and writes to unpadded, and branch-4.1/4.2
behave the same with this mapping. The fix belongs in the write path (both
backends, nested fields included), where it also covers those existing tables;
mapping CHAR(n) to a wider type here would only hide it for tables created by
Doris. I'll do the padding as a separate change, and the description now states
the limitation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
