github-actions[bot] commented on code in PR #68968:
URL: https://github.com/apache/doris/pull/68968#discussion_r4246934643
##########
fe/fe-connector/fe-connector-jdbc/src/main/java/org/apache/doris/connector/jdbc/client/JdbcMySQLConnectorClient.java:
##########
@@ -420,8 +420,17 @@ private ConnectorType dorisTypeToConnectorType(String
rawType, JdbcFieldInfo fie
return ConnectorType.of("BITMAP");
case "QUANTILE_STATE":
return ConnectorType.of("QUANTILE_STATE");
- case "VARBINARY":
- return ConnectorType.of("STRING");
+ case "VARBINARY": {
+ // Keep the remote Doris bytes and length, like the binary
types of every other source.
+ if (openParen == -1) {
+ if (fieldInfo != null) {
+ return ConnectorType.of("VARBINARY",
fieldInfo.requiredColumnSize(), -1);
Review Comment:
[P1] Release binary vector metadata before selecting this scan path. This
changes remote-Doris scalar columns from STRING to VARBINARY and routes their
scans through the binary VectorColumn cleanup path. In that path,
reserveCapacity() allocates 16 * capacity native bytes for [length,address]
records, but close() only calls freeVarbinaryData() (which frees the row
payloads) and then sets data to zero without freeing the metadata buffer.
JniScanner creates/releases a VectorTable for every JDBC batch, including EOF,
so large scans and repeated queries permanently grow BE native memory; the
previous STRING path releases its storage. The JNI allocator is malloc/free, so
query cleanup and Java GC do not reclaim these lost buffers. Free the binary
metadata allocation after its payloads, and cover repeated batch/EOF cleanup
before enabling this mapping.
##########
fe/fe-connector/fe-connector-jdbc/src/main/java/org/apache/doris/connector/jdbc/client/JdbcMySQLConnectorClient.java:
##########
@@ -420,8 +420,17 @@ private ConnectorType dorisTypeToConnectorType(String
rawType, JdbcFieldInfo fie
return ConnectorType.of("BITMAP");
case "QUANTILE_STATE":
return ConnectorType.of("QUANTILE_STATE");
- case "VARBINARY":
- return ConnectorType.of("STRING");
+ case "VARBINARY": {
+ // Keep the remote Doris bytes and length, like the binary
types of every other source.
+ if (openParen == -1) {
+ if (fieldInfo != null) {
+ return ConnectorType.of("VARBINARY",
fieldInfo.requiredColumnSize(), -1);
+ }
+ return ConnectorType.of("VARBINARY");
+ }
+ int len = Integer.parseInt(upperType.substring(openParen + 1,
upperType.length() - 1));
+ return ConnectorType.of("VARBINARY", len, -1);
Review Comment:
[P2] Decode binary array children before publishing their type. The existing
ARRAY branch recursively calls this mapper, so a supported remote view such as
`CREATE VIEW db.v AS SELECT array(cast('abc' AS varbinary(16))) AS a` now
becomes ARRAY<VARBINARY> rather than ARRAY<STRING>. Selecting `a` through the
JDBC catalog reaches `MySQLTypeHandler.convertArray()` ->
`getListTypeForArray()`, which has no VARBINARY case and throws `Unsupported
array child type: VARBINARY`. Before this change the STRING child selects the
working List<String> decoder. Add a binary array leaf decoder that converts
Doris's nested hexadecimal values to byte[] and preserves NULL/empty elements,
with a self-catalog array-view regression.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
