github-actions[bot] commented on code in PR #68934:
URL: https://github.com/apache/doris/pull/68934#discussion_r4278367745
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonTypeMapping.java:
##########
@@ -156,6 +156,10 @@ private static ConnectorType toDecimalType(DecimalType
type) {
private static ConnectorType toTimestampType(DataType dataType) {
int scale = DEFAULT_TIMESTAMP_SCALE;
if (dataType instanceof TimestampType) {
+ // Keep all source fractional digits instead of truncating
high-precision wall-clock values.
+ if (((TimestampType) dataType).getPrecision() >
MAX_TIMESTAMP_SCALE) {
+ return ConnectorType.of("TIMESTAMP_NS");
Review Comment:
[P2] Keep ORC OUTFILE compatible with the new timestamp mapping
For an existing Paimon `TIMESTAMP(7/8/9)` column, `SELECT ts FROM t INTO
OUTFILE ... FORMAT AS ORC` now fails during planning:
`PhysicalPlanTranslator.visitPhysicalFileSink` calls
`OutFileClause.analyzeForOrcFormat`, whose schema generator and validation
switch omit `TIMESTAMP_NS` and throw `currently orc do not support column type:
TIMESTAMP_NS`. The previous `DATETIMEV2(6)` mapping exported successfully, so
ordinary in-range values regress too. Wire the new type through OUTFILE schema
generation/validation and the explicit-schema writer path, and add
scalar/nested export/readback coverage; OUTFILE supplies `orc_schema` and
bypasses the new `_build_orc_type` branch.
##########
fe/fe-connector/fe-connector-paimon/src/main/java/org/apache/doris/connector/paimon/PaimonTypeMapping.java:
##########
@@ -156,6 +156,10 @@ private static ConnectorType toDecimalType(DecimalType
type) {
private static ConnectorType toTimestampType(DataType dataType) {
int scale = DEFAULT_TIMESTAMP_SCALE;
if (dataType instanceof TimestampType) {
+ // Keep all source fractional digits instead of truncating
high-precision wall-clock values.
+ if (((TimestampType) dataType).getPrecision() >
MAX_TIMESTAMP_SCALE) {
Review Comment:
[P2] Select nanos-compatible Parquet options for generic exports
For an existing Paimon `TIMESTAMP(9)` value such as `2024-01-02
03:04:05.123456789`, `SELECT ts FROM t INTO OUTFILE ... FORMAT AS PARQUET` now
fails during BE writing. `ParquetFileFormatProperties` still defaults to
Parquet 1.0 with INT96 disabled, and `VFileResultWriter` forwards those options
to the generic writer. Its new Arrow `timestamp[ns]` reaches [Arrow 24's
mandatory nanos-to-micros
coercion](https://github.com/apache/arrow/blob/apache-arrow-24.0.0/cpp/src/parquet/column_writer.cc#L2398-L2408),
which rejects the nonzero remainder as `would lose data`. The previous
microsecond mapping exported successfully. Apply nanos-compatible version
selection to generic OUTFILE, including nested fields, and test the default
export/readback. The Iceberg-only version upgrade does not cover this caller;
explicit `parquet.version=latest` currently works. This is a write-side failure
separate from the existing legacy TVF/LOAD decoding thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
