github-actions[bot] commented on code in PR #65647:
URL: https://github.com/apache/doris/pull/65647#discussion_r3627145585
##########
fs_brokers/cdc_client/src/main/java/org/apache/doris/cdcclient/source/reader/mysql/MySqlSourceReader.java:
##########
@@ -1018,6 +1019,13 @@ private MySqlSourceConfig generateMySqlConfig(
// Keep genuinely ancient (<100) DATE/DATETIME years; MySQL already
completes 2-digit years.
dbzProps.setProperty("enable.time.adjuster", "false");
+ // The converter is valid only when snapshot JDBC exposes YEAR values
as numbers.
Review Comment:
**[P1] Parse Connector/J's effective boolean before gating this converter**
Connector/J accepts both `FALSE` and `NO` as false boolean values. Because the
FE normalizer preserves any explicit `yearIsDateType` value, a valid URL ending
in `yearIsDateType=NO` configures numeric snapshot reads, but this literal
comparison leaves the custom converter disabled. Debezium's default YEAR path
then adjusts source YEAR 0 to 2000 in snapshot and 1900 in binlog instead of
preserving 0. Please gate on Connector/J's parsed boolean value (or otherwise
canonicalize all supported false spellings) and add `NO` snapshot/binlog
coverage.
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingJdbcUrlNormalizer.java:
##########
@@ -0,0 +1,83 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.job.extensions.insert.streaming;
+
+import org.apache.doris.job.common.DataSourceType;
+
+import java.util.HashSet;
+import java.util.Set;
+
+/**
+ * Normalizes JDBC URLs before streaming ingestion uses them for metadata
discovery and CDC reads.
+ * Database-specific rules are kept here so every streaming entry point
applies the same read-side
+ * semantics while leaving unrelated JDBC Catalog write optimizations out of
scope.
+ */
+public final class StreamingJdbcUrlNormalizer {
+
+ private StreamingJdbcUrlNormalizer() {
+ }
+
+ public static String normalize(DataSourceType sourceType, String jdbcUrl) {
+ switch (sourceType) {
+ case MYSQL:
+ return normalizeMysql(jdbcUrl);
+ case POSTGRES:
+ return jdbcUrl;
+ default:
+ throw new IllegalArgumentException("Unsupported data source
type: " + sourceType);
+ }
+ }
+
+ private static String normalizeMysql(String jdbcUrl) {
+ String normalizedUrl = jdbcUrl.replace(" ", "");
+ Set<String> params = getParams(normalizedUrl);
+ StringBuilder result = new StringBuilder(normalizedUrl);
+ setDefaultParam(result, params, "yearIsDateType", "false");
Review Comment:
**[P1] Preserve Connector/J host-specific options across FE and cdc-client**
Connector/J supports URLs such as
`jdbc:mysql://address=(host=h)(port=3306)(yearIsDateType=true)(tinyInt1isBit=true)/db`,
where host attributes override common query properties. `getParams()` sees
none of those attributes and appends global `false` values. FE metadata still
consumes the full URL and honors the host-level `true` settings, but
`MySqlSourceReader` extracts only host/port and copies
`ConnectionUrl.getOriginalProperties()`—which excludes per-host attributes—so
runtime CDC uses the appended `false` settings. This can auto-create BOOLEAN
metadata while the reader emits numeric TINYINT values and can split YEAR
semantics. Please normalize and propagate the effective main-host properties
consistently, with direct and TVF tests for the `address=(...)` form.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
