leekeiabstraction commented on code in PR #2578:
URL: https://github.com/apache/fluss/pull/2578#discussion_r2956282020
##########
fluss-lake/fluss-lake-lance/src/test/java/org/apache/fluss/lake/lance/tiering/LanceTieringITCase.java:
##########
@@ -140,6 +142,112 @@ void testTiering() throws Exception {
jobClient.cancel().get();
}
+ @Test
+ void testTieringWithNestedRowType() throws Exception {
Review Comment:
Nit: should we test row of row as well?
##########
fluss-lake/fluss-lake-lance/src/main/java/org/apache/fluss/lake/lance/utils/ArrowDataConverter.java:
##########
@@ -94,6 +95,17 @@ private static void copyVectorData(
}
}
+ if (shadedVector
Review Comment:
I wonder if we should be more defensive in this class. Currently if only one
of the vector class is struct, this will be skipped and use non-struct copy
logic, failure will appear much later. (Appreciate that this is the convention
of the class i.e. listvector, consider this as nit)
##########
website/docs/_configs/_partial_config.mdx:
##########
@@ -284,7 +288,7 @@
| `table.auto-partition.enabled` | `false` | Boolean | Whether enable auto
partition for the table. Disable by default. When auto partition is enabled,
the partitions of the table will be created automatically. |
| `table.auto-partition.key` | `none` | String | This configuration defines
the time-based partition key to be used for auto-partitioning when a table is
partitioned with multiple keys. Auto-partitioning utilizes a time-based
partition key to handle partitions automatically, including creating new ones
and removing outdated ones, by comparing the time value of the partition with
the current system time. In the case of a table using multiple partition keys
(such as a composite partitioning strategy), this feature determines which key
should serve as the primary time dimension for making auto-partitioning
decisions.And If the table has only one partition key, this config is not
necessary. Otherwise, it must be specified. |
| `table.auto-partition.time-unit` | `DAY` | AutoPartitionTimeUnit | The time
granularity for auto created partitions. The default value is `DAY`. Valid
values are `HOUR`, `DAY`, `MONTH`, `QUARTER`, `YEAR`. If the value is `HOUR`,
the partition format for auto created is yyyyMMddHH. If the value is `DAY`, the
partition format for auto created is yyyyMMdd. If the value is `MONTH`, the
partition format for auto created is yyyyMM. If the value is `QUARTER`, the
partition format for auto created is yyyyQ. If the value is `YEAR`, the
partition format for auto created is yyyy. |
-| `table.auto-partition.time-zone` | `Europe/Paris` | String | The time zone
for auto partitions, which is by default the same as the system time zone. |
+| `table.auto-partition.time-zone` | `Asia/Shanghai` | String | The time zone
for auto partitions, which is by default the same as the system time zone. |
Review Comment:
Ditto
##########
website/docs/_configs/_partial_config.mdx:
##########
@@ -238,9 +238,13 @@
| Key | Default | Type | Description |
| :--- | :--- | :--- | :--- |
-| `remote.data.dir` | `none` | String | The directory used for storing the kv
snapshot data files and remote log for log tiered storage in a Fluss supported
filesystem. |
+| `remote.data.dir` | `none` | String | The directory used for storing the kv
snapshot data files and remote log for log tiered storage in a Fluss supported
filesystem. When upgrading to `remote.data.dirs`, please ensure this value is
placed as the first entry in the new configuration.For new clusters, it is
recommended to use `remote.data.dirs` instead. If `remote.data.dirs` is
configured, this value will be ignored. |
+| `remote.data.dirs` | `[]` | ArrayList | A comma-separated list of
directories in Fluss supported filesystems for storing the kv snapshot data
files and remote log files of tables/partitions. If configured, when a new
table or a new partition is created, one of the directories from this list will
be selected according to the strategy specified by `remote.data.dirs.strategy`
(`ROUND_ROBIN` by default). If not configured, the system uses
`remote.data.dir` as the sole remote data directory for all data. |
Review Comment:
Are these part of the lance row support change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
