This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 5ffdc28db08 [improvement](binlog) Make binlog.format property
case-insensitive (#68982)
5ffdc28db08 is described below
commit 5ffdc28db08cb7c2d54bd0a3993b06e2f3179869
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Tue Oct 13 16:01:18 2026 +0800
[improvement](binlog) Make binlog.format property case-insensitive (#68982)
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary:
#### In short
The table and database property `binlog.format` only accepted the exact
upper case names `ROW` and `STATEMENT_AND_SNAPSHOT`. `'binlog.format' =
'row'` was rejected by every DDL, and CREATE TABLE even answered with
Java's internal `No enum constant ...` message. This PR matches the
value case-insensitively at every place it is parsed and always stores
the canonical upper case name, including for database properties, which
Doris persists as written and parses again on replay. Unknown values are
still rejected, now with the same message everywhere.
#### Background
- `binlog.format` is a table and database property. `ROW` turns on row
binlog (table streams, IVM, the `binlog()` TVF);
`STATEMENT_AND_SNAPSHOT` is the CCR binlog. A table's format cannot be
changed after it is created.
- The value is parsed in three places, all with `BinlogFormat.valueOf`,
which is case-sensitive:
- `PropertyAnalyzer#analyzeBinlogConfig`: validation used by ALTER
TABLE, CREATE DATABASE and ALTER DATABASE (and by CREATE TABLE, but only
later, see below).
- `BinlogConfig#mergeFromProperties`: builds an effective config. CREATE
TABLE calls it on the raw properties first, in
`Database#getBinlogConfigsForCreateTable`, before anything validates
them. Database properties also go through it on every replay and image
load (`Database#gsonPostProcess`, `replayUpdateDbProperties`).
- `TableProperty#buildBinlogConfig`: rebuilds a table's config from its
persisted properties.
- Table metadata stores the format from the enum
(`BinlogConfig#toProperties`), so it is always canonical. Database
properties are different: `DatabaseProperty` keeps the map exactly as
the user wrote it, and that map is what goes to the edit log and the
image.
- Neighbouring enum-like properties such as `storage_medium` and
`compression` are already matched case-insensitively.
#### The problem, and what it cost
- `'binlog.format' = 'row'`, or any spelling other than all upper case,
is rejected by all four DDLs that take the property. CREATE TABLE fails
with `No enum constant
org.apache.doris.catalog.BinlogConfig.BinlogFormat.row`, which does not
even name the property; ALTER TABLE, CREATE DATABASE and ALTER DATABASE
fail with `Invalid binlog format value: row`.
- Users have to know that this one value is case-sensitive while the
properties next to it are not. The full before / after comparison is in
Results below.
#### How this PR fixes it
1. `BinlogConfig.BinlogFormat#fromString` looks the value up by enum
name, ignoring case, and throws `IllegalArgumentException("Invalid
binlog format value: <value>")` for anything else. The three parse sites
above all use it.
2. `PropertyAnalyzer#analyzeBinlogConfig` returns the canonical name, so
the table property, `SHOW CREATE TABLE` and
`information_schema.table_properties` show `ROW`.
3. `PropertyAnalyzer#normalizeBinlogFormat`, called by
`CreateDatabaseCommand` and `AlterDatabasePropertiesCommand` right after
validation, rewrites the database property to its canonical name before
it is stored. This matters for compatibility: the stored map goes to the
edit log and the image, and an FE without this change (an old follower
during a rolling upgrade, or after a downgrade) parses it with
`valueOf`. Had it been stored as `row`, that FE would fail to replay it
with the same `No enum constant` exception. With the rewrite, the stored
value is always one that an FE without this change could have written
itself.
What it buys:
- Any case is accepted by all four DDLs, and the stored and displayed
value is always upper case.
- Persisted metadata keeps exactly the old format, so mixed-version
clusters and downgrades are unaffected.
- An unknown value such as `rows` gets `Invalid binlog format value:
rows` from CREATE TABLE too, instead of the `No enum constant` message.
#### Results
"Before" is current master: the messages were reproduced by switching
`fromString` back to `valueOf` in the FE unit tests. "After" is this
branch on a local 1 FE + 1 BE cluster, checked by the new regression
suite.
| Statement | Before | After |
|---|---|---|
| `CREATE TABLE ... ('binlog.enable' = 'true', 'binlog.format' = 'row')`
| `No enum constant
org.apache.doris.catalog.BinlogConfig.BinlogFormat.row` | ROW binlog
table; `information_schema.table_properties` shows `ROW`; the `binlog()`
TVF returns the inserted rows |
| `CREATE TABLE ... ('binlog.enable' = 'true', 'binlog.format' =
'Statement_And_Snapshot')` | `No enum constant
...BinlogFormat.Statement_And_Snapshot` | CCR binlog table, stored as
`STATEMENT_AND_SNAPSHOT` |
| `ALTER TABLE <ROW table> SET ('binlog.format' = 'Row')` | `Invalid
binlog format value: Row` | accepted, no-op |
| `ALTER TABLE <ROW table> SET ('binlog.format' =
'statement_and_snapshot')` | `Invalid binlog format value:
statement_and_snapshot` | `not support change binlog format from ROW to
STATEMENT_AND_SNAPSHOT` (the real reason) |
| `CREATE DATABASE d PROPERTIES ('binlog.enable' = 'true',
'binlog.format' = 'row')` | `Invalid binlog format value: row` |
created; `information_schema.database_properties` shows `ROW`; a new
table in `d` inherits ROW binlog |
| `ALTER DATABASE d SET PROPERTIES ('binlog.enable' = 'false',
'binlog.format' = 'statement_and_snapshot')` | `Invalid binlog format
value: statement_and_snapshot` | accepted, stored as
`STATEMENT_AND_SNAPSHOT` |
| `CREATE TABLE ... ('binlog.format' = 'rows')` | `No enum constant
...BinlogFormat.rows` | `Invalid binlog format value: rows` |
| ALTER TABLE / CREATE DATABASE / ALTER DATABASE with `'rows'` |
`Invalid binlog format value: rows` | unchanged |
#### Classes, and how they call each other
- `BinlogConfig.BinlogFormat#fromString` (new): the only parser of the
value.
- `PropertyAnalyzer#analyzeBinlogConfig`: validates, now returns the
canonical name.
- `PropertyAnalyzer#normalizeBinlogFormat` (new): rewrites a map's value
in place; used only for database properties.
- `BinlogConfig#mergeFromProperties`, `TableProperty#buildBinlogConfig`:
switched from `valueOf` to `fromString`.
- `CreateDatabaseCommand#validate`,
`AlterDatabasePropertiesCommand#validate`: call `normalizeBinlogFormat`
after `analyzeBinlogConfig`.
```
CREATE TABLE InternalCatalog#createOlapTable
Database#getBinlogConfigsForCreateTable ->
BinlogConfig#mergeFromProperties -> fromString
PropertyAnalyzer#analyzeBinlogConfig
-> fromString
(the table stores BinlogConfig#toProperties, i.e. the
enum name)
ALTER TABLE ModifyTablePropertiesOp#validate ->
PropertyAnalyzer#analyzeBinlogConfig -> fromString
SchemaChangeHandler#updateBinlogConfig ->
analyzeBinlogConfig + mergeFromProperties
CREATE DATABASE CreateDatabaseCommand#validate ->
analyzeBinlogConfig, then normalizeBinlogFormat -> fromString
ALTER DATABASE AlterDatabasePropertiesCommand#validate ->
analyzeBinlogConfig, then normalizeBinlogFormat -> fromString
(the map is then stored in DatabaseProperty and
journaled as is)
replay / image Database#gsonPostProcess, #replayUpdateDbProperties
-> DatabaseProperty#getBinlogConfig ->
BinlogConfig#mergeFromProperties -> fromString
TableProperty#buildBinlogConfig
-> fromString
```
Not changed: BE, thrift and protobuf (they receive the enum, not the
string), the rule that a table's format is immutable, and every other
`binlog.*` property.
---
.../org/apache/doris/catalog/BinlogConfig.java | 12 +-
.../org/apache/doris/catalog/TableProperty.java | 2 +-
.../apache/doris/common/util/PropertyAnalyzer.java | 13 +-
.../commands/AlterDatabasePropertiesCommand.java | 1 +
.../plans/commands/CreateDatabaseCommand.java | 1 +
.../catalog/BinlogFormatCaseInsensitiveTest.java | 147 +++++++++++++++++
.../apache/doris/common/PropertyAnalyzerTest.java | 27 ++++
.../test_binlog_format_case_insensitive.out | 21 +++
.../test_binlog_format_case_insensitive.groovy | 180 +++++++++++++++++++++
9 files changed, 400 insertions(+), 4 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/catalog/BinlogConfig.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/BinlogConfig.java
index fc6a65f824d..aeaa64da7ce 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/BinlogConfig.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/BinlogConfig.java
@@ -69,6 +69,16 @@ public class BinlogConfig {
public int value() {
return this.value;
}
+
+ // binlog.format is a user-facing property value, so it is matched
case-insensitively.
+ public static BinlogFormat fromString(String format) {
+ for (BinlogFormat binlogFormat : values()) {
+ if (binlogFormat.name().equalsIgnoreCase(format)) {
+ return binlogFormat;
+ }
+ }
+ throw new IllegalArgumentException("Invalid binlog format value: "
+ format);
+ }
}
@SerializedName("needHistoricalValue")
@@ -147,7 +157,7 @@ public class BinlogConfig {
}
if (properties.containsKey(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT))
{
BinlogFormat tmpBinlogFormat =
-
BinlogFormat.valueOf(properties.get(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT));
+
BinlogFormat.fromString(properties.get(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT));
if (!force && tmpBinlogFormat != binlogFormat) {
LOG.warn("not support change {} from {} to {}",
PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT,
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/catalog/TableProperty.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/TableProperty.java
index 1378b39233d..9d0c3d96f6f 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/TableProperty.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/TableProperty.java
@@ -616,7 +616,7 @@ public class TableProperty implements GsonPostProcessable {
Long.parseLong(properties.get(PropertyAnalyzer.PROPERTIES_BINLOG_MAX_HISTORY_NUMS)));
}
if (properties.containsKey(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT))
{
- binlogConfig.setBinlogFormat(BinlogConfig.BinlogFormat.valueOf(
+ binlogConfig.setBinlogFormat(BinlogConfig.BinlogFormat.fromString(
properties.get(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT)));
}
if
(properties.containsKey(PropertyAnalyzer.PROPERTIES_BINLOG_NEED_HISTORICAL_VALUE))
{
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
b/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
index c8bccb9449f..21a0fe74ff1 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
@@ -1574,8 +1574,7 @@ public class PropertyAnalyzer {
if (properties.containsKey(PROPERTIES_BINLOG_FORMAT)) {
String format = properties.get(PROPERTIES_BINLOG_FORMAT);
try {
- BinlogConfig.BinlogFormat.valueOf(format);
- binlogConfigMap.put(PROPERTIES_BINLOG_FORMAT, format);
+ binlogConfigMap.put(PROPERTIES_BINLOG_FORMAT,
BinlogConfig.BinlogFormat.fromString(format).name());
properties.remove(PROPERTIES_BINLOG_FORMAT);
} catch (Exception e) {
throw new AnalysisException("Invalid binlog format value: " +
format);
@@ -1597,6 +1596,16 @@ public class PropertyAnalyzer {
return binlogConfigMap;
}
+ /**
+ * Rewrite binlog.format to its canonical name, for properties persisted
as written (database
+ * properties): they are parsed again on replay, and an FE that predates
case-insensitive parsing
+ * only accepts the canonical name. The value must already be validated by
analyzeBinlogConfig.
+ */
+ public static void normalizeBinlogFormat(Map<String, String> properties) {
+ properties.computeIfPresent(PROPERTIES_BINLOG_FORMAT,
+ (key, format) ->
BinlogConfig.BinlogFormat.fromString(format).name());
+ }
+
public static boolean analyzeIsBeingSynced(Map<String, String> properties,
boolean defaultValue) {
if (properties != null &&
properties.containsKey(PROPERTIES_IS_BEING_SYNCED)) {
String value = properties.remove(PROPERTIES_IS_BEING_SYNCED);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/AlterDatabasePropertiesCommand.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/AlterDatabasePropertiesCommand.java
index a6350ae0d0e..d507d7bebc9 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/AlterDatabasePropertiesCommand.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/AlterDatabasePropertiesCommand.java
@@ -66,6 +66,7 @@ public class AlterDatabasePropertiesCommand extends
AlterCommand {
// clone properties for analyse
Map<String, String> analysisProperties = new HashMap<String,
String>(properties);
PropertyAnalyzer.analyzeBinlogConfig(analysisProperties);
+ PropertyAnalyzer.normalizeBinlogFormat(properties);
}
@Override
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateDatabaseCommand.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateDatabaseCommand.java
index 75c81808d85..b754fb96a66 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateDatabaseCommand.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/CreateDatabaseCommand.java
@@ -81,6 +81,7 @@ public class CreateDatabaseCommand extends Command implements
ForwardWithSync, N
FeNameFormat.checkDbName(dbName);
InternalDatabaseUtil.checkDatabase(dbName, ConnectContext.get());
PropertyAnalyzer.analyzeBinlogConfig(new HashMap<>(properties));
+ PropertyAnalyzer.normalizeBinlogFormat(properties);
BinlogConfig config = BinlogConfig.fromProperties(properties);
if (config.isEnableForStreaming() &&
properties.containsKey(PropertyAnalyzer.PROPERTIES_BINLOG_TTL_SECONDS)) {
config.applyExplicitRowTtl(config.getTtlSeconds());
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/catalog/BinlogFormatCaseInsensitiveTest.java
b/fe/fe-core/src/test/java/org/apache/doris/catalog/BinlogFormatCaseInsensitiveTest.java
new file mode 100644
index 00000000000..5d86e35354d
--- /dev/null
+++
b/fe/fe-core/src/test/java/org/apache/doris/catalog/BinlogFormatCaseInsensitiveTest.java
@@ -0,0 +1,147 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog;
+
+import org.apache.doris.catalog.BinlogConfig.BinlogFormat;
+import org.apache.doris.common.AnalysisException;
+import org.apache.doris.common.Config;
+import org.apache.doris.common.DdlException;
+import org.apache.doris.common.util.PropertyAnalyzer;
+import org.apache.doris.utframe.TestWithFeService;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public class BinlogFormatCaseInsensitiveTest extends TestWithFeService {
+ private Database db;
+
+ @Override
+ protected void runBeforeAll() throws Exception {
+ Config.enable_feature_binlog = true;
+ createDatabase("binlog_format_case");
+ db =
Env.getCurrentInternalCatalog().getDbOrDdlException("binlog_format_case");
+ }
+
+ @Test
+ public void testFromStringIgnoresCase() {
+ Assertions.assertEquals(BinlogFormat.ROW,
BinlogFormat.fromString("row"));
+ Assertions.assertEquals(BinlogFormat.ROW,
BinlogFormat.fromString("Row"));
+ Assertions.assertEquals(BinlogFormat.STATEMENT_AND_SNAPSHOT,
+ BinlogFormat.fromString("statement_and_snapshot"));
+ Assertions.assertThrows(IllegalArgumentException.class, () ->
BinlogFormat.fromString("rows"));
+ }
+
+ @Test
+ public void testCreateRowTableWithLowerCaseFormat() throws Exception {
+ createTable("CREATE TABLE binlog_format_case.lower_row (k INT)
DUPLICATE KEY(k) "
+ + "DISTRIBUTED BY HASH(k) BUCKETS 1 "
+ + "PROPERTIES('replication_num'='1', 'binlog.enable'='true',
'binlog.format'='row')");
+ OlapTable table = (OlapTable) db.getTableOrDdlException("lower_row");
+ Assertions.assertTrue(table.getBinlogConfig().isEnableForStreaming());
+ Assertions.assertTrue(table.enableTso());
+ Assertions.assertEquals("ROW",
+
table.getTableProperty().getProperties().get(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT));
+
+ // The format is still immutable, whatever the case of the new value.
+ DdlException exception = Assertions.assertThrows(DdlException.class,
() -> alterTableSync(
+ "ALTER TABLE binlog_format_case.lower_row SET
('binlog.format'='statement_and_snapshot')"));
+ Assertions.assertTrue(exception.getMessage().contains(
+ "not support change binlog format from ROW to
STATEMENT_AND_SNAPSHOT"), exception.getMessage());
+ Assertions.assertTrue(table.getBinlogConfig().isEnableForStreaming());
+ }
+
+ @Test
+ public void testAlterStatementTableWithLowerCaseFormat() throws Exception {
+ createTable("CREATE TABLE binlog_format_case.lower_statement (k INT)
DUPLICATE KEY(k) "
+ + "DISTRIBUTED BY HASH(k) BUCKETS 1 "
+ + "PROPERTIES('replication_num'='1', 'binlog.enable'='true', "
+ + "'binlog.format'='Statement_And_Snapshot')");
+ OlapTable table = (OlapTable)
db.getTableOrDdlException("lower_statement");
+ Assertions.assertTrue(table.getBinlogConfig().isEnableForCCR());
+ Assertions.assertEquals("STATEMENT_AND_SNAPSHOT",
+
table.getTableProperty().getProperties().get(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT));
+
+ // Repeating the current format in another case is a no-op.
+ alterTableSync("ALTER TABLE binlog_format_case.lower_statement "
+ + "SET ('binlog.format'='statement_and_snapshot')");
+ Assertions.assertTrue(table.getBinlogConfig().isEnableForCCR());
+
+ DdlException exception = Assertions.assertThrows(DdlException.class,
() -> alterTableSync(
+ "ALTER TABLE binlog_format_case.lower_statement SET
('binlog.format'='row')"));
+ Assertions.assertTrue(exception.getMessage().contains(
+ "not support change binlog format from STATEMENT_AND_SNAPSHOT
to ROW"), exception.getMessage());
+ Assertions.assertTrue(table.getBinlogConfig().isEnableForCCR());
+ }
+
+ @Test
+ public void testRejectUnknownTableFormat() {
+ IllegalStateException exception =
Assertions.assertThrows(IllegalStateException.class,
+ () -> executeSql("CREATE TABLE
binlog_format_case.unknown_format (k INT) DUPLICATE KEY(k) "
+ + "DISTRIBUTED BY HASH(k) BUCKETS 1 "
+ + "PROPERTIES('replication_num'='1',
'binlog.enable'='true', 'binlog.format'='rows')"));
+ Assertions.assertTrue(exception.getMessage().contains("Invalid binlog
format value: rows"),
+ exception.getMessage());
+ Assertions.assertNull(db.getTableNullable("unknown_format"));
+ }
+
+ @Test
+ public void testDatabaseFormatIsStoredByCanonicalName() throws Exception {
+ createDatabaseWithSql("CREATE DATABASE binlog_format_case_db "
+ + "PROPERTIES ('binlog.enable'='true',
'binlog.format'='row')");
+ Database rowDb =
Env.getCurrentInternalCatalog().getDbOrDdlException("binlog_format_case_db");
+ // Database properties are persisted as given and parsed again on
replay.
+ Assertions.assertEquals("ROW",
rowDb.getDbProperties().get(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT));
+ Assertions.assertTrue(rowDb.getBinlogConfig().isEnableForStreaming());
+
Assertions.assertTrue(rowDb.getDbProperties().getBinlogConfig().isEnableForStreaming());
+
+ // A table created in the database inherits the ROW format.
+ createTable("CREATE TABLE binlog_format_case_db.inherited_row (k INT)
DUPLICATE KEY(k) "
+ + "DISTRIBUTED BY HASH(k) BUCKETS 1
PROPERTIES('replication_num'='1')");
+ OlapTable table = (OlapTable)
rowDb.getTableOrDdlException("inherited_row");
+ Assertions.assertTrue(table.getBinlogConfig().isEnableForStreaming());
+
+ executeSql("ALTER DATABASE binlog_format_case_db SET PROPERTIES "
+ + "('binlog.enable'='false',
'binlog.format'='statement_and_snapshot')");
+ Assertions.assertEquals("STATEMENT_AND_SNAPSHOT",
+
rowDb.getDbProperties().get(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT));
+ Assertions.assertEquals(BinlogFormat.STATEMENT_AND_SNAPSHOT,
rowDb.getBinlogConfig().getBinlogFormat());
+ }
+
+ @Test
+ public void testRejectUnknownDatabaseFormat() {
+ AnalysisException exception =
Assertions.assertThrows(AnalysisException.class,
+ () -> createDatabaseWithSql("CREATE DATABASE
binlog_format_case_unknown "
+ + "PROPERTIES ('binlog.enable'='true',
'binlog.format'='rows')"));
+ Assertions.assertTrue(exception.getMessage().contains("Invalid binlog
format value: rows"),
+ exception.getMessage());
+
Assertions.assertNull(Env.getCurrentInternalCatalog().getDbNullable("binlog_format_case_unknown"));
+ }
+
+ @Test
+ public void testReplayParsesNonCanonicalFormat() {
+ Map<String, String> properties = new HashMap<>();
+ properties.put(PropertyAnalyzer.PROPERTIES_BINLOG_ENABLE, "true");
+ properties.put(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT, "row");
+ Assertions.assertTrue(new
TableProperty(properties).buildBinlogConfig().getBinlogConfig()
+ .isEnableForStreaming());
+ Assertions.assertTrue(new
DatabaseProperty(properties).getBinlogConfig().isEnableForStreaming());
+ }
+}
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/common/PropertyAnalyzerTest.java
b/fe/fe-core/src/test/java/org/apache/doris/common/PropertyAnalyzerTest.java
index 8e8fda29030..4afa2d44f7c 100644
--- a/fe/fe-core/src/test/java/org/apache/doris/common/PropertyAnalyzerTest.java
+++ b/fe/fe-core/src/test/java/org/apache/doris/common/PropertyAnalyzerTest.java
@@ -39,6 +39,8 @@ import com.google.common.collect.Maps;
import com.google.common.collect.Sets;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.CsvSource;
import java.time.Instant;
import java.time.ZoneId;
@@ -65,6 +67,31 @@ public class PropertyAnalyzerTest {
() -> PropertyAnalyzer.analyzeBinlogConfig(properties));
}
+ @ParameterizedTest
+ @CsvSource({"ROW, ROW", "row, ROW", "Row, ROW", "STATEMENT_AND_SNAPSHOT,
STATEMENT_AND_SNAPSHOT",
+ "statement_and_snapshot, STATEMENT_AND_SNAPSHOT"})
+ public void testBinlogFormatIgnoresCase(String format, String
canonicalName) throws AnalysisException {
+ Map<String, String> properties = new HashMap<>();
+ properties.put(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT, format);
+ Assertions.assertEquals(canonicalName,
PropertyAnalyzer.analyzeBinlogConfig(properties)
+ .get(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT));
+ Assertions.assertTrue(properties.isEmpty());
+
+ properties.put(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT, format);
+ PropertyAnalyzer.normalizeBinlogFormat(properties);
+ Assertions.assertEquals(canonicalName,
properties.get(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT));
+ }
+
+ @Test
+ public void testBinlogFormatRejectsUnknownValue() {
+ Map<String, String> properties = new HashMap<>();
+ properties.put(PropertyAnalyzer.PROPERTIES_BINLOG_FORMAT, "rows");
+ AnalysisException exception =
Assertions.assertThrows(AnalysisException.class,
+ () -> PropertyAnalyzer.analyzeBinlogConfig(properties));
+ Assertions.assertTrue(exception.getMessage().contains("Invalid binlog
format value: rows"),
+ exception.getMessage());
+ }
+
@Test
public void testBfColumns() throws AnalysisException {
List<Column> columns = Lists.newArrayList();
diff --git
a/regression-test/data/row_binlog_p0/test_binlog_format_case_insensitive.out
b/regression-test/data/row_binlog_p0/test_binlog_format_case_insensitive.out
new file mode 100644
index 00000000000..419567edea5
--- /dev/null
+++ b/regression-test/data/row_binlog_p0/test_binlog_format_case_insensitive.out
@@ -0,0 +1,21 @@
+-- This file is automatically generated. You should know what you did if you
want to edit this
+-- !table_format --
+test_binlog_format_case_row_tbl ROW
+test_binlog_format_case_statement_tbl STATEMENT_AND_SNAPSHOT
+
+-- !row_binlog --
+0 1 10
+0 2 20
+
+-- !db_format --
+binlog.enable true
+binlog.format ROW
+
+-- !inherited_table_format --
+binlog.enable true
+binlog.format ROW
+
+-- !altered_db_format --
+binlog.enable false
+binlog.format STATEMENT_AND_SNAPSHOT
+
diff --git
a/regression-test/suites/row_binlog_p0/test_binlog_format_case_insensitive.groovy
b/regression-test/suites/row_binlog_p0/test_binlog_format_case_insensitive.groovy
new file mode 100644
index 00000000000..9748c7e186d
--- /dev/null
+++
b/regression-test/suites/row_binlog_p0/test_binlog_format_case_insensitive.groovy
@@ -0,0 +1,180 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+// "binlog.format" is matched case-insensitively and always stored by its
canonical (upper case) name,
+// at both the table and the database level.
+suite("test_binlog_format_case_insensitive") {
+ if (isCloudMode()) {
+ logger.info("skip test_binlog_format_case_insensitive in cloud mode")
+ return
+ }
+
+ // enable_feature_binlog is an EXPERIMENTAL config, so SHOW FRONTEND
CONFIG reports it as
+ // experimental_enable_feature_binlog. checkEnableFeatureBinlog() accounts
for that prefix.
+ if (!getSyncer().checkEnableFeatureBinlog()) {
+ logger.info("fe enable_feature_binlog is false, skip case
test_binlog_format_case_insensitive")
+ return
+ }
+
+ sql "DROP TABLE IF EXISTS test_binlog_format_case_row_tbl FORCE"
+ sql "DROP TABLE IF EXISTS test_binlog_format_case_statement_tbl FORCE"
+ sql "DROP TABLE IF EXISTS test_binlog_format_case_invalid_tbl FORCE"
+ sql "DROP DATABASE IF EXISTS test_binlog_format_case_db FORCE"
+ sql "DROP DATABASE IF EXISTS test_binlog_format_case_invalid_db FORCE"
+
+ sql """
+ CREATE TABLE test_binlog_format_case_row_tbl (
+ k1 INT NOT NULL,
+ v1 INT
+ )
+ DUPLICATE KEY(k1)
+ DISTRIBUTED BY HASH(k1) BUCKETS 1
+ PROPERTIES (
+ "replication_num" = "1",
+ "binlog.enable" = "true",
+ "binlog.format" = "row"
+ )
+ """
+ sql """
+ CREATE TABLE test_binlog_format_case_statement_tbl (
+ k1 INT NOT NULL,
+ v1 INT
+ )
+ DUPLICATE KEY(k1)
+ DISTRIBUTED BY HASH(k1) BUCKETS 1
+ PROPERTIES (
+ "replication_num" = "1",
+ "binlog.enable" = "true",
+ "binlog.format" = "Statement_And_Snapshot"
+ )
+ """
+ order_qt_table_format """
+ SELECT TABLE_NAME, PROPERTY_VALUE
+ FROM information_schema.table_properties
+ WHERE TABLE_CATALOG = 'internal'
+ AND TABLE_SCHEMA = '${context.dbName}'
+ AND TABLE_NAME IN ('test_binlog_format_case_row_tbl',
'test_binlog_format_case_statement_tbl')
+ AND PROPERTY_NAME = 'binlog.format'
+ """
+
+ // The lower case value created a real ROW binlog: the write below is
readable from it.
+ sql "INSERT INTO test_binlog_format_case_row_tbl VALUES (1, 10), (2, 20)"
+ order_qt_row_binlog """
+ SELECT __DORIS_BINLOG_OP__ AS op, k1, v1
+ FROM binlog("table" = "test_binlog_format_case_row_tbl")
+ """
+
+ // Repeating the current format in another case is a no-op.
+ sql """ALTER TABLE test_binlog_format_case_row_tbl SET ("binlog.format" =
"Row")"""
+ sql """ALTER TABLE test_binlog_format_case_statement_tbl SET
("binlog.format" = "statement_and_snapshot")"""
+
+ // The format stays immutable, whatever the case of the new value.
+ test {
+ sql """ALTER TABLE test_binlog_format_case_row_tbl SET
("binlog.format" = "statement_and_snapshot")"""
+ exception "not support change binlog format from ROW to
STATEMENT_AND_SNAPSHOT"
+ }
+ test {
+ sql """ALTER TABLE test_binlog_format_case_statement_tbl SET
("binlog.format" = "row")"""
+ exception "not support change binlog format from
STATEMENT_AND_SNAPSHOT to ROW"
+ }
+
+ // Values that are not a format in any case are still rejected.
+ test {
+ sql """
+ CREATE TABLE test_binlog_format_case_invalid_tbl (
+ k1 INT NOT NULL,
+ v1 INT
+ )
+ DUPLICATE KEY(k1)
+ DISTRIBUTED BY HASH(k1) BUCKETS 1
+ PROPERTIES (
+ "replication_num" = "1",
+ "binlog.enable" = "true",
+ "binlog.format" = "rows"
+ )
+ """
+ exception "Invalid binlog format value: rows"
+ }
+ test {
+ sql """ALTER TABLE test_binlog_format_case_row_tbl SET
("binlog.format" = "rows")"""
+ exception "Invalid binlog format value: rows"
+ }
+
+ // Database properties are stored as given, so the value is rewritten to
its canonical name first.
+ sql """
+ CREATE DATABASE test_binlog_format_case_db PROPERTIES (
+ "binlog.enable" = "true",
+ "binlog.format" = "row"
+ )
+ """
+ order_qt_db_format """
+ SELECT PROPERTY_NAME, PROPERTY_VALUE
+ FROM information_schema.database_properties
+ WHERE CATALOG_NAME = 'internal'
+ AND SCHEMA_NAME = 'test_binlog_format_case_db'
+ AND PROPERTY_NAME IN ('binlog.enable', 'binlog.format')
+ """
+
+ // A table created in the database inherits the ROW format.
+ sql """
+ CREATE TABLE
test_binlog_format_case_db.test_binlog_format_case_inherited_tbl (
+ k1 INT NOT NULL,
+ v1 INT
+ )
+ DUPLICATE KEY(k1)
+ DISTRIBUTED BY HASH(k1) BUCKETS 1
+ PROPERTIES (
+ "replication_num" = "1"
+ )
+ """
+ order_qt_inherited_table_format """
+ SELECT PROPERTY_NAME, PROPERTY_VALUE
+ FROM information_schema.table_properties
+ WHERE TABLE_CATALOG = 'internal'
+ AND TABLE_SCHEMA = 'test_binlog_format_case_db'
+ AND TABLE_NAME = 'test_binlog_format_case_inherited_tbl'
+ AND PROPERTY_NAME IN ('binlog.enable', 'binlog.format')
+ """
+
+ sql """
+ ALTER DATABASE test_binlog_format_case_db SET PROPERTIES (
+ "binlog.enable" = "false",
+ "binlog.format" = "statement_and_snapshot"
+ )
+ """
+ order_qt_altered_db_format """
+ SELECT PROPERTY_NAME, PROPERTY_VALUE
+ FROM information_schema.database_properties
+ WHERE CATALOG_NAME = 'internal'
+ AND SCHEMA_NAME = 'test_binlog_format_case_db'
+ AND PROPERTY_NAME IN ('binlog.enable', 'binlog.format')
+ """
+
+ test {
+ sql """
+ CREATE DATABASE test_binlog_format_case_invalid_db PROPERTIES (
+ "binlog.enable" = "true",
+ "binlog.format" = "rows"
+ )
+ """
+ exception "Invalid binlog format value: rows"
+ }
+ test {
+ sql """ALTER DATABASE test_binlog_format_case_db SET PROPERTIES
("binlog.format" = "rows")"""
+ exception "Invalid binlog format value: rows"
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
