Gabriel39 commented on code in PR #68027:
URL: https://github.com/apache/doris/pull/68027#discussion_r4025398153
##########
fe/fe-connector/fe-connector-spi/src/main/java/org/apache/doris/connector/spi/scan/ScanNodePropertyKeys.java:
##########
@@ -132,6 +132,9 @@ public final class ScanNodePropertyKeys {
/** Quote character enclosing a field; a single character. */
public static final String TEXT_ENCLOSE = TEXT_PROPERTY_PREFIX + "enclose";
+ /** {@code "true"} for Hive OpenCSV field states and physical record
boundaries. */
+ public static final String TEXT_HIVE_OPEN_CSV = TEXT_PROPERTY_PREFIX +
"hive_open_csv";
Review Comment:
Addressed in 3cb283539e. The connector API is now 9.0, with its pinned
version updated in the same commit. ConnectorPluginSurfaceTest records every
public ScanNodePropertyKeys field by name, type, and literal value, so an
inlined key/value change becomes a visible API-surface change. Both baselines
were regenerated; the metadata-method baseline remains identical.
The actual plugin directory loader now rejects an API 8.0 probe jar on this
FE; the existing matching-version and other-major tests also pass. Validation:
all 142 connector SPI tests, 480 Hive connector tests, and 34 FE Core tests
passed. The old-version loader and version/surface checks reproduced the
failures before the fix.
##########
gensrc/thrift/PlanNodes.thrift:
##########
@@ -313,6 +313,9 @@ struct TFileAttributes {
// org.openx.data.jsonserde.JsonSerDe
13: optional bool openx_json_ignore_malformed = false;
+ // Hive OpenCSVSerde has different field states and physical record
boundaries from load CSV.
+ 14: optional bool hive_open_csv = false;
Review Comment:
Addressed in 3cb283539e. FE and BE now define OpenCSV support at execution
version 15. The real scan attribute builder rejects OpenCSV when the configured
query-wide execution version is below 15, and Hive emits
REQUIRED_CURRENT_BACKEND_SEMANTICS so the existing scheduler fence rejects
eligible smooth-upgrade source backends even at version 15.
Added FE tests for version 14 rejection, version 15 acceptance, a mixed
candidate list containing a smooth-upgrade source, and the absent-flag wire
contract. Both actual BE scanners additionally verify that an absent flag
retains legacy decoding while the same bytes with the flag use OpenCSV
decoding. The BE version manager accepts both legacy version 14 and the new
version 15. Validation: 34 FE Core tests and 47 BE tests under ASAN passed; the
missing FE fences and unsupported BE version were reproduced before the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
