This is an automated email from the ASF dual-hosted git repository.
JingsongLi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/paimon.git
The following commit(s) were added to refs/heads/master by this push:
new 0963071a76 [docs] Reorganize migration guides and add SVG diagrams
(#9733)
0963071a76 is described below
commit 0963071a765f5387a4bb16956d028cd417d4d7a8
Author: Jingsong Lee <[email protected]>
AuthorDate: Fri Sep 11 13:39:52 2026 +0800
[docs] Reorganize migration guides and add SVG diagrams (#9733)
---
docs/docs/migration/clone-to-paimon.md | 174 ++++++++++++++-----
docs/docs/migration/index.md | 39 +++++
docs/docs/migration/migration-from-hive.mdx | 228 ++++++++++++++++++-------
docs/docs/migration/upsert-to-partitioned.mdx | 222 ++++++++++++------------
docs/sidebars.js | 4 +-
docs/static/img/migration-clone-overwrite.svg | 52 ++++++
docs/static/img/migration-strategies.svg | 60 +++++++
docs/static/img/migration-tag-to-partition.svg | 60 +++++++
8 files changed, 626 insertions(+), 213 deletions(-)
diff --git a/docs/docs/migration/clone-to-paimon.md
b/docs/docs/migration/clone-to-paimon.md
index b8383875f3..a75075a938 100644
--- a/docs/docs/migration/clone-to-paimon.md
+++ b/docs/docs/migration/clone-to-paimon.md
@@ -1,6 +1,6 @@
---
-title: "Clone To Paimon"
-sidebar_position: 5
+title: "Clone to Paimon"
+sidebar_position: 2
---
<!--
@@ -22,65 +22,149 @@ specific language governing permissions and limitations
under the License.
-->
-# Clone To Paimon
+# Clone to Paimon
-Clone supports cloning tables to Paimon tables.
+Clone copies data into separate Paimon tables while keeping source tables and
files available.
+Use it to validate a Paimon copy before switching workloads, or to refresh
selected target
+partitions. Unlike [migration](./migration-from-hive), clone does not move
source files.
-1. Clone is `OVERWRITE` semantic that will overwrite the partitions of the
target table according to the data.
-2. Clone is reentrant, but it requires existing tables to contain all fields
from the source table and have the
- same partition fields.
+The Flink `clone` action supports Hive and Paimon sources. This guide starts
with Hive tables
+in ORC, Parquet, or Avro format, which become Paimon [append
tables](../append-table/).
-Currently, clone supports clone Hive tables in Hive Catalog to Paimon Catalog,
supports Parquet, ORC, Avro formats,
-target table will be append table.
+## Before You Start
+
+- Install the matching [Flink action jar](../flink/action-jars) and set
`FLINK_HOME`.
+- Configure source and target catalogs separately. Hive sources require a Hive
catalog; the
+ examples use a filesystem catalog with a separate warehouse for the target.
+- Ensure the job can read the source and write the target. Keep source data
stable during the
+ copy when you need a consistent validation baseline.
+- Use distinct source and target locations. For an existing Hive-clone target,
use an append
+ table with `bucket = -1`, compatible file format, all source fields with
compatible types,
+ and the same partition fields.
+
+## Understand Overwrite Behavior
+
+With the default target option `dynamic-partition-overwrite = true`, clone
replaces target
+partitions represented in the copied data. Other target partitions remain
unchanged. For a
+non-partitioned target, the copied data replaces the table contents.
+
+
+
+Re-running clone refreshes the copied partitions; it does not append a second
copy. It also
+does not synchronize source deletions for partitions absent from the copied
data. If an
+existing target sets `dynamic-partition-overwrite = false`, overwrite can
replace the entire
+target table even when the source is filtered. Check this option before
cloning a subset.
## Clone Hive Table
+The following example copies `default.hivetable` to `analytics.hivetable_copy`:
+
```bash
-<FLINK_HOME>/flink run ./paimon-flink-action-@@VERSION@@.jar \
-clone \
---database default \
---table hivetable \
---catalog_conf metastore=hive \
---catalog_conf uri=thrift://localhost:9088 \
---target_database test \
---target_table test_table \
---target_catalog_conf warehouse=my_warehouse \
---parallelism 10 \
---where <filter_spec>
+"$FLINK_HOME/bin/flink" run /path/to/paimon-flink-action-@@VERSION@@.jar \
+ clone \
+ --clone_from hive \
+ --database default \
+ --table hivetable \
+ --catalog_conf metastore=hive \
+ --catalog_conf uri=thrift://localhost:9083 \
+ --target_database analytics \
+ --target_table hivetable_copy \
+ --target_catalog_conf warehouse=hdfs:///paimon-warehouse \
+ --parallelism 10
```
-You can use filter spec to specify the filtering condition for the partition.
+To copy only selected partitions, add a quoted partition predicate. For a
source partitioned
+by the string column `dt`, append this argument to the command:
+
+```bash
+--where "dt = '2026-09-10'"
+```
+
+`--where` selects partitions, not arbitrary rows within a partition. Omit it
to copy all
+partitions. Table include/exclude lists are not accepted when cloning a single
table.
## Clone Hive Database
+Omit `--table` and supply `--target_database`. Source table names are
preserved in the target
+database, which is created if needed.
+
```bash
-<FLINK_HOME>/flink run ./paimon-flink-action-@@VERSION@@.jar \
-clone \
---database default \
---catalog_conf metastore=hive \
---catalog_conf uri=thrift://localhost:9088 \
---target_database test \
---parallelism 10 \
---target_catalog_conf warehouse=my_warehouse
---included_tables <included_tables_spec> \
---excluded_tables <excluded_tables_spec>
+"$FLINK_HOME/bin/flink" run /path/to/paimon-flink-action-@@VERSION@@.jar \
+ clone \
+ --clone_from hive \
+ --database default \
+ --catalog_conf metastore=hive \
+ --catalog_conf uri=thrift://localhost:9083 \
+ --target_database analytics \
+ --target_catalog_conf warehouse=hdfs:///paimon-warehouse \
+ --parallelism 10 \
+ --included_tables default.orders,default.customers
```
-"--included_tables" and "--excluded_tables" are optional parameters, which are
used to specify the tables that need or don't need to be cloned.
-The format is `<database1>.<table1>,<database2>.<table2>,<database3>.<table3>`.
-"--excluded_tables" has higher priority than "--included_tables" if you
specified both.
+
+Omit `--included_tables` to consider all tables in the source database. Use
+`--excluded_tables` to remove tables from that selection.
## Clone Hive Catalog
+Omit source and target database/table arguments to preserve database and table
names across
+the catalog. The following example limits the selection to two source tables:
+
```bash
-<FLINK_HOME>/flink run ./paimon-flink-action-@@VERSION@@.jar \
-clone \
---catalog_conf metastore=hive \
---catalog_conf uri=thrift://localhost:9088 \
---parallelism 10 \
---target_catalog_conf warehouse=my_warehouse \
---included_tables <included_tables_spec> \
---excluded_tables <excluded_tables_spec>
+"$FLINK_HOME/bin/flink" run /path/to/paimon-flink-action-@@VERSION@@.jar \
+ clone \
+ --clone_from hive \
+ --catalog_conf metastore=hive \
+ --catalog_conf uri=thrift://localhost:9083 \
+ --target_catalog_conf warehouse=hdfs:///paimon-warehouse \
+ --parallelism 10 \
+ --included_tables sales.orders,crm.customers
```
-"--included_tables" and "--excluded_tables" are optional parameters, which are
used to specify the tables that need or don't need to be cloned.
-The format is `<database1>.<table1>,<database2>.<table2>,<database3>.<table3>`.
-"--excluded_tables" has higher priority than "--included_tables" if you
specified both.
+
+## Selection and Retry Options
+
+| Option | Behavior |
+| --- | --- |
+| `--included_tables db.table,db.other` | Selects fully qualified source table
names for database or catalog cloning. Omit to consider all tables in scope. |
+| `--excluded_tables db.table,db.other` | Removes tables from the selection.
Exclusion wins when a table is also included. |
+| `--where "dt = '2026-09-10'"` | Restricts copied data using a partition
predicate. |
+| `--clone_if_exists false` | Skips targets that already exist. The default is
`true`, which clones into existing compatible targets. |
+| `--meta_only true` | Clones the schema without copying data. The default is
`false`. |
+| `--parallelism 10` | Sets the clone job parallelism. |
+
+A multi-table clone does not commit all tables atomically. After a failure,
inspect target
+tables and rerun the required scope. `--clone_if_exists false` is useful for
skipping existing
+tables, but is not a resume mechanism: an existing table may have been created
before its
+data was copied.
+
+## Clone a Paimon Source
+
+Use `--clone_from paimon` and configure the source Paimon catalog. For
example, to copy between
+two filesystem catalogs:
+
+```bash
+"$FLINK_HOME/bin/flink" run /path/to/paimon-flink-action-@@VERSION@@.jar \
+ clone \
+ --clone_from paimon \
+ --catalog_conf warehouse=hdfs:///source-paimon-warehouse \
+ --database sales \
+ --table orders \
+ --target_catalog_conf warehouse=hdfs:///target-paimon-warehouse \
+ --target_database analytics \
+ --target_table orders_copy \
+ --parallelism 10
+```
+
+Paimon sources can retain primary keys in the newly created target. The Hive
append-table
+restriction above applies to Hive sources. For Paimon sources, repeated
+`--target_table_conf key=value` arguments can override options when creating
the target;
+these overrides are not supported for Hive sources.
+
+## Verify the Clone
+
+1. Confirm that the Flink job completed and that every selected target exists.
+2. Connect an engine to the target catalog and inspect the schema and
partition keys.
+3. Compare row counts and aggregates with the stable source data, using the
same partition
+ predicate when cloning a subset. Confirm that unrelated target partitions
remain as expected.
+4. Verify that source readers still work, then switch workloads when the
target is ready.
+
+For SQL invocation instead of the action jar, see the [Flink `clone`
procedure](../flink/procedures/table-operations#clone).
diff --git a/docs/docs/migration/index.md b/docs/docs/migration/index.md
index bc01176fbc..4068865371 100644
--- a/docs/docs/migration/index.md
+++ b/docs/docs/migration/index.md
@@ -21,3 +21,42 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
+
+# Migration
+
+Choose a migration path based on whether you need to replace a Hive table,
keep a separate
+copy, or expose historical Paimon data to existing Hive queries.
+
+
+
+## Choose a Guide
+
+| Goal | Guide | Effect on the source | Result |
+| --- | --- | --- | --- |
+| Convert a Hive table or database to Paimon | [Migrate from
Hive](./migration-from-hive) | Moves data files; removes the original Hive
table by default | Paimon append tables in a Hive catalog |
+| Copy tables while keeping the source available | [Clone to
Paimon](./clone-to-paimon) | Keeps source tables and data | Separate Paimon
tables; Hive sources become append tables |
+| Query daily views of an updating Paimon table using Hive partition filters |
[Expose Tags as Hive Partitions](./upsert-to-partitioned) | Keeps the Paimon
table and its write path | Hive partition values that select tags or preview
snapshots |
+
+Migration and clone import existing data. Tag-to-partition changes how Hive
reads a Paimon
+table; it does not migrate Hive files or physically repartition the Paimon
table.
+
+## Before Moving Data
+
+1. **Choose the target table model.** Hive migration and Hive clone create
+ [append tables](../append-table/). If the target needs primary-key updates
or a different
+ schema or partition layout, plan a data rewrite into a table with that
model.
+2. **Prepare the runtime and catalogs.** Configure access to the Hive
metastore and storage.
+ Use the engine setup and connector requirements linked from each guide.
+3. **Define the validation scope.** Record source schemas, partitions, row
counts, and key
+ aggregates before the operation. For a consistent comparison, keep source
data stable
+ while it is being moved or copied.
+4. **Plan the switch.** In-place migration is not atomic and requires a
backup. Clone lets
+ you validate a separate target before switching readers and writers.
+
+## Related Guides
+
+- [Flink Procedures](../flink/procedures/table-operations#migrate_table) and
+ [Spark Migration Procedures](../spark/procedures/migration): engine-specific
arguments.
+- [COPY INTO](../spark/copy-into): import files with Spark SQL.
+- [Manage Tags](../maintenance/manage-tags): create and retain historical
views.
+- [Hive](../ecosystem/hive): install the connector used to query Paimon from
Hive.
diff --git a/docs/docs/migration/migration-from-hive.mdx
b/docs/docs/migration/migration-from-hive.mdx
index f893d6dfc3..fd95902973 100644
--- a/docs/docs/migration/migration-from-hive.mdx
+++ b/docs/docs/migration/migration-from-hive.mdx
@@ -1,5 +1,5 @@
---
-title: "Migration From Hive"
+title: "Migrate from Hive"
sidebar_position: 1
---
@@ -27,114 +27,222 @@ under the License.
# Hive Table Migration
-Apache Hive supports ORC, Parquet file formats that could be migrated to
Paimon.
-When migrating data to a paimon table, the origin table will be permanently
disappeared. So please back up your data if you
-still need the original table. The migrated table will be [append
table](../append-table/).
+Migrate existing Hive data files into Paimon [append tables](../append-table/)
using a Paimon
+Hive catalog. The migrator moves files and registers them in Paimon metadata.
It supports
+ORC, Parquet, and Avro source files, for both partitioned and non-partitioned
tables.
-Now, we can use paimon hive catalog with Migrate Table Procedure to totally
migrate a table from hive to paimon.
-At the same time, you can use paimon hive catalog with Migrate Database
Procedure to fully synchronize all tables in the database to paimon.
+By default, the original Hive table is replaced by a Paimon table with the
same name. Use
+[Clone to Paimon](./clone-to-paimon) when you need a separate copy and a
readable source table.
-* Migrate Table Procedure: Paimon table does not exist, use the procedure
upgrade hive table to paimon table. Hive table will disappear after action done.
-* Migrate Database Procedure: Paimon table does not exist, use the procedure
upgrade all hive tables in database to paimon table. All hive tables will
disappear after action done.
+:::warning Migration is not atomic
-These two actions now support file format of hive "orc" and "parquet" and
"avro".
+Back up the source data and Hive metadata before migrating, and stop writes to
the source
+tables. An interruption can leave files or metadata partially migrated. Do not
rely on
+`delete_origin => false` as a backup: it keeps the Hive metadata, but files
are still moved
+out of the source location.
-<span style={{color: "red"}}> **We highly recommend to back up hive table data
before migrating, because migrating action is not atomic. If been interrupted
while migrating, you may lose your data.** </span>
+:::
+
+## Before You Start
+
+- Configure a Paimon catalog with `metastore = hive` and access to the source
Hive metastore.
+ See [Catalog](../concepts/catalog), [Flink Procedures](../flink/procedures),
or
+ [Spark Catalogs](../spark/catalogs) for setup.
+- Give the migration runtime access to source and target storage, including
the ability to
+ move files and update Hive metadata. This is a file-move workflow, not a
cross-storage copy.
+- Use a target append table with no primary key and `bucket = -1`. For an
existing target,
+ check that the source data schema is compatible and partition names and
types match.
+- Record source row counts, partition values, and representative query results
for validation.
+ The examples below use an ORC source table named `default.hivetable`; adjust
identifiers,
+ paths, and `file.format` for your source.
+
+## Choose the Target Behavior
+
+| Procedure arguments | Target | Source after success |
+| --- | --- | --- |
+| Omit `target_table` | Replaces the original table under its original name |
Original Hive table is removed |
+| Set a different `target_table`; omit `delete_origin` or set it to `true` |
Creates or imports files into the named Paimon table | Original Hive table is
removed |
+| Set a different `target_table` and `delete_origin => false` | Creates or
imports files into the named Paimon table | Hive metadata remains, but its
files have moved |
+
+For an existing target, migration adds imported files; it does not use clone's
partition
+overwrite behavior. Check the existing target data before importing to avoid
duplicates.
+`target_table` and `delete_origin` are SQL procedure options; the Flink
migration action shown
+below uses the default replacement behavior.
## Migrate Hive Table
-<Tabs groupId="migrate-table">
+Choose one of the following entry points. The SQL examples select the Paimon
Hive catalog
+before calling the procedure.
+<Tabs groupId="migration-engine">
<TabItem value="flink-sql" label="Flink SQL">
```sql
-CREATE CATALOG PAIMON WITH (
- 'type'='paimon',
- 'metastore' = 'hive',
- 'uri' = 'thrift://localhost:9083',
- 'warehouse'='/path/to/warehouse/');
+CREATE CATALOG paimon_hive WITH (
+ 'type' = 'paimon',
+ 'metastore' = 'hive',
+ 'uri' = 'thrift://localhost:9083',
+ 'warehouse' = 'hdfs:///warehouse'
+);
-USE CATALOG PAIMON;
+USE CATALOG paimon_hive;
+-- Flink 1.19 and later: replace default.hivetable with a Paimon table.
CALL sys.migrate_table(
connector => 'hive',
source_table => 'default.hivetable',
- -- You can specify the target table, and if the target table already exists
- -- the file will be migrated directly to it
- -- target_table => 'default.paimontarget',
- -- You can specify delete_origin is false, this won't delete hivetable
- -- delete_origin => false,
- options => 'file.format=orc');
+ options => 'file.format=orc'
+);
+```
+
+For Flink 1.18, use positional arguments instead of the call above:
+
+```sql
+CALL sys.migrate_table('hive', 'default.hivetable', 'file.format=orc');
+```
+
+To import into a different target with Flink 1.19 or later, use this call
**instead of** the
+replacement call. The original Hive table is removed by default.
+
+```sql
+CALL sys.migrate_table(
+ connector => 'hive',
+ source_table => 'default.hivetable',
+ target_table => 'default.paimon_target',
+ options => 'file.format=orc'
+);
```
</TabItem>
+<TabItem value="spark-sql" label="Spark SQL">
-<TabItem value="flink-action" label="Flink Action">
+Configure a [Spark Hive catalog](../spark/catalogs) named `paimon_hive` with
the following
+Spark startup options:
```bash
-<FLINK_HOME>/flink run ./paimon-flink-action-@@VERSION@@.jar \
-migrate_table \
---warehouse /path/to/warehouse \
---catalog_conf uri=thrift://localhost:9083 \
---catalog_conf metastore=hive \
---source_type hive \
---table default.hive_or_paimon
+--conf spark.sql.catalog.paimon_hive=org.apache.paimon.spark.SparkCatalog \
+--conf spark.sql.catalog.paimon_hive.metastore=hive \
+--conf spark.sql.catalog.paimon_hive.uri=thrift://localhost:9083 \
+--conf spark.sql.catalog.paimon_hive.warehouse=hdfs:///warehouse \
+--conf
spark.sql.extensions=org.apache.paimon.spark.extensions.PaimonSparkSessionExtensions
+```
+
+Then run:
+
+```sql
+USE paimon_hive.default;
+
+CALL sys.migrate_table(
+ source_type => 'hive',
+ table => 'default.hivetable',
+ options => 'file.format=orc'
+);
```
+Spark uses `source_type` and `table` for these arguments. See
+[Spark Migration Procedures](../spark/procedures/migration#migrate_table) for
optional target
+and parallelism arguments.
+
</TabItem>
+<TabItem value="flink-action" label="Flink Action">
+Set `FLINK_HOME` to your Flink installation and use the matching
+[Flink action jar](../flink/action-jars).
+
+```bash
+"$FLINK_HOME/bin/flink" run /path/to/paimon-flink-action-@@VERSION@@.jar \
+ migrate_table \
+ --warehouse hdfs:///warehouse \
+ --catalog_conf metastore=hive \
+ --catalog_conf uri=thrift://localhost:9083 \
+ --source_type hive \
+ --table default.hivetable \
+ --options file.format=orc
+```
+
+</TabItem>
</Tabs>
-After invoke, "hivetable" will totally convert to paimon format. Writing and
reading the table by old "hive way" will fail.
+After success, read and write the target through Paimon. Existing jobs that
expect the
+original Hive storage format must be updated. Hive queries need the
+[Paimon Hive connector](../ecosystem/hive).
## Migrate Hive Database
-<Tabs groupId="migrate-database">
+Database migration attempts each table in the source database and replaces
successfully
+migrated tables under their original names. It is not a transaction across all
tables: a
+failure can leave a mixture of Hive and Paimon tables. Review every source
table's format and
+requirements first. Use individual table migration for a selected subset.
+<Tabs groupId="migration-engine">
<TabItem value="flink-sql" label="Flink SQL">
-```sql
-CREATE CATALOG PAIMON WITH (
- 'type'='paimon',
- 'metastore' = 'hive',
- 'uri' = 'thrift://localhost:9083',
- 'warehouse'='/path/to/warehouse/');
+Using the Paimon Hive catalog configured above:
-USE CATALOG PAIMON;
+```sql
+USE CATALOG paimon_hive;
+-- Flink 1.19 and later
CALL sys.migrate_database(
connector => 'hive',
source_database => 'default',
- options => 'file.format=orc');
+ options => 'file.format=orc'
+);
+```
+
+For Flink 1.18, use this call instead:
+
+```sql
+CALL sys.migrate_database('hive', 'default', 'file.format=orc');
```
</TabItem>
+<TabItem value="spark-sql" label="Spark SQL">
-<TabItem value="flink-action" label="Flink Action">
+```sql
+USE paimon_hive.default;
-```bash
-<FLINK_HOME>/bin/flink run \
-/path/to/paimon-flink-action-@@VERSION@@.jar \
-migrate_database \
---warehouse <warehouse-path> \
---source_type hive \
---database <database> \
-[--catalog_conf <paimon-catalog-conf> [--catalog_conf <paimon-catalog-conf>
...]] \
-[--options <paimon-table-conf [,paimon-table-conf ...]> ]
+CALL sys.migrate_database(
+ source_type => 'hive',
+ database => 'default',
+ options => 'file.format=orc'
+);
```
-Example:
+</TabItem>
+<TabItem value="flink-action" label="Flink Action">
+
```bash
-<FLINK_HOME>/flink run ./paimon-flink-action-@@VERSION@@.jar migrate_table \
---warehouse /path/to/warehouse \
---catalog_conf uri=thrift://localhost:9083 \
---catalog_conf metastore=hive \
---source_type hive \
---database default
+"$FLINK_HOME/bin/flink" run /path/to/paimon-flink-action-@@VERSION@@.jar \
+ migrate_database \
+ --warehouse hdfs:///warehouse \
+ --catalog_conf metastore=hive \
+ --catalog_conf uri=thrift://localhost:9083 \
+ --source_type hive \
+ --database default \
+ --options file.format=orc
```
</TabItem>
-
</Tabs>
-After invoke, all tables in "default" database will totally convert to paimon
format. Writing and reading the table by old "hive way" will fail.
+## Verify the Migration
+
+1. Check the procedure output and logs. For a database, inspect both success
and failure
+ counts; completion of the call does not mean every table migrated.
+2. Inspect the target schema, partition keys, and table options. Confirm that
it is a Paimon
+ append table with `bucket = -1`.
+3. Query the target through Paimon and compare row counts and aggregates
against the values
+ recorded before migration. Include null partition values and representative
partitions.
+4. Update reader and writer configurations, then verify them against the
migrated table.
+
+For example, in the selected Paimon catalog:
+
+```sql
+SHOW CREATE TABLE default.hivetable;
+SELECT COUNT(*) FROM default.hivetable;
+```
+
+If migration fails, inspect the source and target metadata and file locations
before retrying.
+For database migration, identify the failed tables and handle them
individually after resolving
+the cause; do not assume another database-wide call will safely resume the
previous run.
diff --git a/docs/docs/migration/upsert-to-partitioned.mdx
b/docs/docs/migration/upsert-to-partitioned.mdx
index eb82a8deeb..0eff41cd25 100644
--- a/docs/docs/migration/upsert-to-partitioned.mdx
+++ b/docs/docs/migration/upsert-to-partitioned.mdx
@@ -1,11 +1,8 @@
---
-title: "Upsert To Partitioned"
-sidebar_position: 1
+title: "Expose Tags as Hive Partitions"
+sidebar_position: 3
---
-import Tabs from '@theme/Tabs';
-import TabItem from '@theme/TabItem';
-
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
@@ -25,169 +22,182 @@ specific language governing permissions and limitations
under the License.
-->
-# Upsert To Partitioned
+# Expose Tags as Hive Partitions {#upsert-to-partitioned}
-:::warning
+Keep a non-partitioned Paimon primary-key table updated from an upstream
database, while
+letting Hive batch jobs read historical views using a partition filter such as
+`WHERE dt = '2023-10-16'`.
-__Note:__ Only Hive Engine can be used to query these upsert-to-partitioned
tables.
+Set `metastore.tag-to-partition` to the Hive partition field name. Paimon
exposes tag names
+as partition values in the Hive metastore. Each partition reads a full table
view at the tag's
+snapshot; it does not contain only the rows changed on that date.
-:::
+
-The [Tag Management](../maintenance/manage-tags) will maintain the manifests
and data files of the snapshot.
-A typical usage is creating tags daily, then you can maintain the historical
data of each day for batch reading.
+:::note Query engine
-When using primary key tables, a non-partitioned approach is often used to
maintain updates, in order to mirror and
-synchronize tables from upstream database tables. This allows users to query
the latest data. The tradition of Hive
-data warehouses is not like this. Offline data warehouses require an immutable
view every day to ensure the idempotence
-of calculations. So we created a Tag mechanism to output these views.
+The partition view described here is for the Hive engine and requires the
+[Paimon Hive connector](../ecosystem/hive). Continue writing the underlying
Paimon table with
+Flink. The Hive partition field is a metadata mapping; do not add it as a
physical partition
+column to the Paimon table.
-However, the traditional use of Hive data warehouses is more accustomed to
using partitions to specify the query's Tag,
-and is more accustomed to using Hive computing engines.
+:::
-So, we introduce `'metastore.tag-to-partition'` and
`'metastore.tag-to-partition.preview'` to mapping a non-partitioned
-primary key table to the partition table in Hive metastore, and mapping the
partition field to the name of the Tag to be
-fully compatible with Hive.
+## Choose a View
-## Example for Tag to Partition
+| View | Configuration | What Hive reads |
+| --- | --- | --- |
+| Created tag | `metastore.tag-to-partition = dt` | The snapshot referenced by
the named tag. Later upserts do not change that snapshot. |
+| Preview before tag creation | Also set `metastore.tag-to-partition.preview =
process-time` | The latest eligible retained snapshot for the requested period.
Results can change as new snapshots arrive. |
-**Step 1: Create table and tag in Flink SQL**
+Tags retain the manifests and files needed to read their snapshots. Preview
alone does not
+provide that retention. Use [tag creation and
retention](../maintenance/manage-tags) for
+reproducible historical reads.
-<Tabs groupId="create-table-and-tag-in-flink-sql-1">
+## Example for Tag to Partition
+
+### 1. Create the Paimon Table in Flink
-<TabItem value="flink" label="Flink">
+Configure a Hive catalog and create a non-partitioned table with an explicit
primary key.
+Use a new table for this example.
```sql
CREATE CATALOG my_hive WITH (
'type' = 'paimon',
'metastore' = 'hive',
- -- 'uri' = 'thrift://<hive-metastore-host-name>:<port>', default use
'hive.metastore.uris' in HiveConf
- -- 'hive-conf-dir' = '...', this is recommended in the kerberos environment
- -- 'hadoop-conf-dir' = '...', this is recommended in the kerberos
environment
- -- 'warehouse' = 'hdfs:///path/to/table/store/warehouse', default use
'hive.metastore.warehouse.dir' in HiveConf
+ 'uri' = 'thrift://localhost:9083',
+ 'warehouse' = 'hdfs:///warehouse'
);
USE CATALOG my_hive;
+CREATE DATABASE IF NOT EXISTS mydb;
+
+SET 'execution.runtime-mode' = 'batch';
+SET 'table.dml-sync' = 'true';
CREATE TABLE mydb.t (
pk INT,
col1 STRING,
- col2 STRING
+ col2 STRING,
+ PRIMARY KEY (pk) NOT ENFORCED
) WITH (
- 'bucket' = '-1',
+ 'bucket' = '2',
'metastore.tag-to-partition' = 'dt'
);
-INSERT INTO t VALUES (1, '10', '100'), (2, '20', '200');
+INSERT INTO mydb.t VALUES (1, '10', '100'), (2, '20', '200');
--- create tag '2023-10-16' for snapshot 1
-CALL sys.create_tag('mydb.t', '2023-10-16', 1);
+-- Pin the latest committed snapshot. This tag name is a label chosen for the
example.
+CALL sys.create_tag('mydb.t', '2023-10-16');
```
-</TabItem>
+Wait for the insert to finish before creating a tag. Here, `table.dml-sync`
makes the SQL
+client wait for each insert; omitting the snapshot argument selects the latest
snapshot.
-</Tabs>
+### 2. Read the Tag in Hive
-**Step 2: Query table in Hive with Partition Pruning**
+Connect Hive to the same metastore, with the Paimon Hive connector installed:
-<Tabs groupId="query-table-in-hive-with-partition-pruning-1">
+```sql
+SHOW PARTITIONS mydb.t;
+-- dt=2023-10-16
+
+SELECT * FROM mydb.t WHERE dt = '2023-10-16' ORDER BY pk;
+-- pk col1 col2 dt
+-- 1 10 100 2023-10-16
+-- 2 20 200 2023-10-16
+```
+
+### 3. Upsert and Create Another View
-<TabItem value="hive" label="Hive">
+Back in the same Flink session, update an existing key and insert a new key:
```sql
-SHOW PARTITIONS t;
-/*
-OK
-dt=2023-10-16
-*/
-
-SELECT * FROM t WHERE dt='2023-10-16';
-/*
-OK
-1 10 100 2023-10-16
-2 20 200 2023-10-16
-*/
+INSERT INTO mydb.t VALUES (1, '11', '110'), (3, '30', '300');
+CALL sys.create_tag('mydb.t', '2023-10-17');
```
-</TabItem>
+In Hive, the earlier tag still returns the original value for key `1`. The new
tag returns
+the updated value and all rows present at the new snapshot, including
unchanged key `2`:
-</Tabs>
+```sql
+SELECT * FROM mydb.t WHERE dt = '2023-10-16' ORDER BY pk;
+-- 1 10 100 2023-10-16
+-- 2 20 200 2023-10-16
+
+SELECT * FROM mydb.t WHERE dt = '2023-10-17' ORDER BY pk;
+-- 1 11 110 2023-10-17
+-- 2 20 200 2023-10-17
+-- 3 30 300 2023-10-17
+```
-## Example for Tag Preview
+Always select the tag you intend to read. A query across multiple tag
partitions can return
+multiple historical versions of the same primary key.
-The above example can only query tags that have already been created, but
Paimon is a real-time data lake, and you also
-need to query the latest data. Therefore, Paimon provides a preview feature:
+## Example for Tag Preview
-**Step 1: Create table and tag in Flink SQL**
+Preview exposes a period before a tag has been created for it. Use it when
Hive readers need
+to see data still arriving during that period.
-<Tabs groupId="create-table-and-tag-in-flink-sql-2">
+### 1. Enable Preview in Flink
-<TabItem value="flink" label="Flink">
+Using the catalog and database above, create a separate example table:
```sql
-CREATE CATALOG my_hive WITH (
- 'type' = 'paimon',
- 'metastore' = 'hive',
- -- 'uri' = 'thrift://<hive-metastore-host-name>:<port>', default use
'hive.metastore.uris' in HiveConf
- -- 'hive-conf-dir' = '...', this is recommended in the kerberos environment
- -- 'hadoop-conf-dir' = '...', this is recommended in the kerberos
environment
- -- 'warehouse' = 'hdfs:///path/to/table/store/warehouse', default use
'hive.metastore.warehouse.dir' in HiveConf
-);
-
-USE CATALOG my_hive;
-
-CREATE TABLE mydb.t (
+CREATE TABLE mydb.t_preview (
pk INT,
col1 STRING,
- col2 STRING
+ col2 STRING,
+ PRIMARY KEY (pk) NOT ENFORCED
) WITH (
- 'bucket' = '-1',
+ 'bucket' = '2',
'metastore.tag-to-partition' = 'dt',
- -- preview tag creation mode process-time
- -- paimon will create partitions early based on process-time
- 'metastore.tag-to-partition.preview' = 'process-time'
+ 'metastore.tag-to-partition.preview' = 'process-time',
+ 'tag.creation-period' = 'daily'
);
-INSERT INTO t VALUES (1, '10', '100'), (2, '20', '200');
+INSERT INTO mydb.t_preview VALUES (1, '10', '100'), (2, '20', '200');
+```
--- create tag '2023-10-16' for snapshot 1
-CALL sys.create_tag('mydb.t', '2023-10-16', 1);
+After a commit, Paimon registers a preview partition based on processing time
and the tag
+creation period. Its actual value depends on when the job runs and the
configured
+`sink.process-time-zone`; inserting example data does not create a historical
date partition.
--- new data in '2023-10-17'
-INSERT INTO t VALUES (3, '30', '300'), (4, '40', '400');
+### 2. Discover and Query the Preview in Hive
--- haven't finished writing the data for '2023-10-17' yet, so there's no need
to create a tag for now
--- but the data is already visible for Hive
+```sql
+SHOW PARTITIONS mydb.t_preview;
```
-</TabItem>
+Copy the partition value returned by `SHOW PARTITIONS` into the predicate
below, replacing
+`<preview-partition>`:
-</Tabs>
+```sql
+SELECT * FROM mydb.t_preview
+WHERE dt = '<preview-partition>'
+ORDER BY pk;
+```
-**Step 2: Query table in Hive with Partition Pruning**
+Without a real tag of that name, Hive resolves the preview to the latest
available snapshot
+whose period is no later than the requested period, or an eligible retained
tag if needed.
+It fails if no suitable snapshot or tag remains. Further commits in the same
period can
+change the result. Once a tag with that name exists, reads use the tag's
snapshot.
-<Tabs groupId="query-table-in-hive-with-partition-pruning-2">
+## Retain Historical Views
-<TabItem value="hive" label="Hive">
+Preview does not create permanent tags. To generate daily tags while keeping
the current
+period visible, configure the writing job with matching creation and preview
modes:
```sql
-SHOW PARTITIONS t;
-/*
-OK
-dt=2023-10-16
-dt=2023-10-17
-*/
-
-SELECT * FROM t WHERE dt='2023-10-17';
--- preview tag '2023-10-17'
-/*
-OK
-1 10 100 2023-10-17
-2 20 200 2023-10-17
-3 30 300 2023-10-17
-4 40 400 2023-10-17
-*/
+-- Table options for a continuously written table
+'metastore.tag-to-partition' = 'dt',
+'metastore.tag-to-partition.preview' = 'process-time',
+'tag.automatic-creation' = 'process-time',
+'tag.creation-period' = 'daily',
+'tag.num-retained-max' = '90'
```
-</TabItem>
-
-</Tabs>
+Apply these as table options when creating the table, or through `ALTER TABLE`
before starting
+the writer. Choose retention for your batch workloads. Removing a tag also
removes its Hive
+partition; do not expire tags still needed by downstream jobs. See
+[Manage Tags](../maintenance/manage-tags) for automatic creation, delay, and
retention options.
diff --git a/docs/sidebars.js b/docs/sidebars.js
index 89b2926a90..788cb2c615 100644
--- a/docs/sidebars.js
+++ b/docs/sidebars.js
@@ -625,8 +625,8 @@ const sidebars = {
},
"items": [
"migration/migration-from-hive",
- "migration/upsert-to-partitioned",
- "migration/clone-to-paimon"
+ "migration/clone-to-paimon",
+ "migration/upsert-to-partitioned"
]
},
{
diff --git a/docs/static/img/migration-clone-overwrite.svg
b/docs/static/img/migration-clone-overwrite.svg
new file mode 100644
index 0000000000..fadb194398
--- /dev/null
+++ b/docs/static/img/migration-clone-overwrite.svg
@@ -0,0 +1,52 @@
+<svg xmlns="http://www.w3.org/2000/svg"; width="900" height="490" viewBox="0 0
900 490" role="img" aria-labelledby="title desc">
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+ <title id="title">Clone replaces the copied partitions</title>
+ <desc id="desc">With dynamic-partition-overwrite enabled, copying Hive
partition dt=2026-09-10 replaces old rows in that target partition and retains
dt=2026-09-09. The source table and files stay available. The diagram assumes
data is copied for the selected partition.</desc>
+ <defs>
+ <marker id="arrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7"
markerHeight="7" orient="auto">
+ <path d="M 0 0 L 10 5 L 0 10 z" fill="#526277"/>
+ </marker>
+ </defs>
+ <g font-family="Arial, Helvetica, sans-serif">
+ <rect x="1" y="1" width="898" height="488" rx="12" fill="#ffffff"
stroke="#d7dfeb" stroke-width="1.5"/>
+ <text x="28" y="43" font-size="27" fill="#172b4d" font-weight="700"
text-anchor="start">Clone replaces the copied partitions</text>
+ <text x="28" y="77" font-size="18" fill="#526277" font-weight="400"
text-anchor="start">Example: copy dt = 2026-09-10 with
dynamic-partition-overwrite = true</text>
+ <rect x="28" y="104" width="844" height="79" rx="8" fill="#eaf2ff"
stroke="#bfd2ed" stroke-width="1.5"/>
+ <text x="48" y="135" font-size="21" fill="#2463b4" font-weight="700"
text-anchor="start">Hive source</text>
+ <text x="48" y="164" font-size="20" fill="#172b4d" font-weight="400"
text-anchor="start">dt=2026-09-10 → new rows</text>
+ <text x="848" y="150" font-size="19" fill="#2463b4" font-weight="400"
text-anchor="end">Source table and files stay available</text>
+ <text x="28" y="228" font-size="16" fill="#526277" font-weight="700"
text-anchor="start">TARGET BEFORE</text>
+ <text x="539" y="228" font-size="16" fill="#526277" font-weight="700"
text-anchor="start">TARGET AFTER</text>
+ <rect x="28" y="246" width="333" height="143" rx="8" fill="#f5f7fb"
stroke="#d7dfeb" stroke-width="1.5"/>
+ <rect x="539" y="246" width="333" height="143" rx="8" fill="#f5f7fb"
stroke="#d7dfeb" stroke-width="1.5"/>
+ <rect x="44" y="262" width="301" height="47" rx="8" fill="#f5f7fb"
stroke="#d7dfeb" stroke-width="1.5"/>
+ <text x="60" y="292" font-size="20" fill="#172b4d" font-weight="400"
text-anchor="start">09-09 | retained rows</text>
+ <rect x="44" y="326" width="301" height="47" rx="8" fill="#fff7e6"
stroke="#e4cb92" stroke-width="1.5"/>
+ <text x="60" y="356" font-size="20" fill="#946200" font-weight="400"
text-anchor="start">09-10 | old rows</text>
+ <rect x="555" y="262" width="301" height="47" rx="8" fill="#f5f7fb"
stroke="#d7dfeb" stroke-width="1.5"/>
+ <text x="571" y="292" font-size="20" fill="#172b4d" font-weight="400"
text-anchor="start">09-09 | retained rows</text>
+ <rect x="555" y="326" width="301" height="47" rx="8" fill="#e8f7f2"
stroke="#b8dcd2" stroke-width="1.5"/>
+ <text x="571" y="356" font-size="20" fill="#087f6e" font-weight="700"
text-anchor="start">09-10 | new rows</text>
+ <text x="450" y="303" font-size="20" fill="#2463b4" font-weight="700"
text-anchor="middle">Clone</text>
+ <path d="M 375 324 H 525" fill="none" stroke="#526277" stroke-width="2"
marker-end="url(#arrow)"/>
+ <text x="28" y="426" font-size="18" fill="#526277" font-weight="400"
text-anchor="start">Re-run: refresh the copied partition. Partitions absent
from the copy remain unchanged.</text>
+ <text x="28" y="460" font-size="18" fill="#946200" font-weight="400"
text-anchor="start">Check existing target options: disabling dynamic overwrite
can replace the whole table.</text>
+ </g>
+</svg>
diff --git a/docs/static/img/migration-strategies.svg
b/docs/static/img/migration-strategies.svg
new file mode 100644
index 0000000000..2aa61d7e58
--- /dev/null
+++ b/docs/static/img/migration-strategies.svg
@@ -0,0 +1,60 @@
+<svg xmlns="http://www.w3.org/2000/svg"; width="900" height="520" viewBox="0 0
900 520" role="img" aria-labelledby="title desc">
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+ <title id="title">Choose the path by what must stay</title>
+ <desc id="desc">Migrate moves Hive files into Paimon and replaces the Hive
table by default. Clone copies data into a separate Paimon table and keeps the
source. Tag-to-partition exposes snapshots of an existing Paimon table as Hive
partitions without moving its data.</desc>
+ <defs>
+ <marker id="arrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7"
markerHeight="7" orient="auto">
+ <path d="M 0 0 L 10 5 L 0 10 z" fill="#526277"/>
+ </marker>
+ </defs>
+ <g font-family="Arial, Helvetica, sans-serif">
+ <rect x="1" y="1" width="898" height="518" rx="12" fill="#ffffff"
stroke="#d7dfeb" stroke-width="1.5"/>
+ <text x="28" y="43" font-size="27" fill="#172b4d" font-weight="700"
text-anchor="start">Choose the path by what must stay</text>
+ <text x="28" y="77" font-size="18" fill="#526277" font-weight="400"
text-anchor="start">Source ownership and the reader interface determine the
workflow</text>
+ <text x="28" y="126" font-size="16" fill="#946200" font-weight="700"
text-anchor="start">MIGRATE</text>
+ <rect x="28" y="141" width="292" height="73" rx="8" fill="#f5f7fb"
stroke="#d7dfeb" stroke-width="1.5"/>
+ <text x="44" y="170" font-size="21" fill="#172b4d" font-weight="700"
text-anchor="start">Hive table</text>
+ <text x="44" y="197" font-size="18" fill="#526277" font-weight="400"
text-anchor="start">Original table replaced</text>
+ <text x="454" y="162" font-size="19" fill="#946200" font-weight="700"
text-anchor="middle">Move files</text>
+ <path d="M 334 181 H 573" fill="none" stroke="#526277" stroke-width="2"
marker-end="url(#arrow)"/>
+ <rect x="588" y="141" width="284" height="73" rx="8" fill="#fff7e6"
stroke="#e4cb92" stroke-width="1.5"/>
+ <text x="604" y="170" font-size="21" fill="#946200" font-weight="700"
text-anchor="start">Paimon append table</text>
+ <text x="604" y="197" font-size="18" fill="#526277" font-weight="400"
text-anchor="start">Same table name by default</text>
+ <text x="28" y="252" font-size="16" fill="#2463b4" font-weight="700"
text-anchor="start">CLONE</text>
+ <rect x="28" y="267" width="292" height="73" rx="8" fill="#f5f7fb"
stroke="#d7dfeb" stroke-width="1.5"/>
+ <text x="44" y="296" font-size="21" fill="#172b4d" font-weight="700"
text-anchor="start">Hive / Paimon table</text>
+ <text x="44" y="323" font-size="18" fill="#526277" font-weight="400"
text-anchor="start">Source remains available</text>
+ <text x="454" y="288" font-size="19" fill="#2463b4" font-weight="700"
text-anchor="middle">Copy data</text>
+ <path d="M 334 307 H 573" fill="none" stroke="#526277" stroke-width="2"
marker-end="url(#arrow)"/>
+ <rect x="588" y="267" width="284" height="73" rx="8" fill="#eaf2ff"
stroke="#bfd2ed" stroke-width="1.5"/>
+ <text x="604" y="296" font-size="21" fill="#2463b4" font-weight="700"
text-anchor="start">Separate Paimon table</text>
+ <text x="604" y="323" font-size="18" fill="#526277" font-weight="400"
text-anchor="start">Validate before switching</text>
+ <text x="28" y="378" font-size="16" fill="#087f6e" font-weight="700"
text-anchor="start">TAG TO PARTITION</text>
+ <rect x="28" y="393" width="292" height="73" rx="8" fill="#f5f7fb"
stroke="#d7dfeb" stroke-width="1.5"/>
+ <text x="44" y="422" font-size="21" fill="#172b4d" font-weight="700"
text-anchor="start">Paimon upsert table</text>
+ <text x="44" y="449" font-size="18" fill="#526277" font-weight="400"
text-anchor="start">Keep updating this table</text>
+ <text x="454" y="414" font-size="19" fill="#087f6e" font-weight="700"
text-anchor="middle">Expose views</text>
+ <path d="M 334 433 H 573" fill="none" stroke="#526277" stroke-width="2"
marker-end="url(#arrow)"/>
+ <rect x="588" y="393" width="284" height="73" rx="8" fill="#e8f7f2"
stroke="#b8dcd2" stroke-width="1.5"/>
+ <text x="604" y="422" font-size="21" fill="#087f6e" font-weight="700"
text-anchor="start">Hive partition interface</text>
+ <text x="604" y="449" font-size="18" fill="#526277" font-weight="400"
text-anchor="start">dt selects a tag or preview</text>
+ <text x="28" y="499" font-size="18" fill="#526277" font-weight="400"
text-anchor="start">Hive migration and Hive clone produce append tables; tag
mapping keeps the Paimon table.</text>
+ </g>
+</svg>
diff --git a/docs/static/img/migration-tag-to-partition.svg
b/docs/static/img/migration-tag-to-partition.svg
new file mode 100644
index 0000000000..6d37cf590f
--- /dev/null
+++ b/docs/static/img/migration-tag-to-partition.svg
@@ -0,0 +1,60 @@
+<svg xmlns="http://www.w3.org/2000/svg"; width="900" height="662" viewBox="0 0
900 662" role="img" aria-labelledby="title desc">
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+ <title id="title">One updating table, multiple Hive views</title>
+ <desc id="desc">A non-partitioned Paimon primary-key table exposes tag names
as Hive dt partitions. The first tag retains key 1 with value 10 and key 2 with
value 20. The second tag retains the update to key 1 with value 11 and the new
key 3, as well as unchanged key 2. A preview partition reads an eligible
snapshot until a real tag exists; preview results can change.</desc>
+ <defs>
+ <marker id="arrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7"
markerHeight="7" orient="auto">
+ <path d="M 0 0 L 10 5 L 0 10 z" fill="#526277"/>
+ </marker>
+ </defs>
+ <g font-family="Arial, Helvetica, sans-serif">
+ <rect x="1" y="1" width="898" height="660" rx="12" fill="#ffffff"
stroke="#d7dfeb" stroke-width="1.5"/>
+ <text x="28" y="43" font-size="27" fill="#172b4d" font-weight="700"
text-anchor="start">One updating table, multiple Hive views</text>
+ <text x="28" y="77" font-size="18" fill="#526277" font-weight="400"
text-anchor="start">Hive partitions select full table snapshots, not the rows
changed that day</text>
+ <rect x="28" y="106" width="844" height="60" rx="8" fill="#f5f7fb"
stroke="#d7dfeb" stroke-width="1.5"/>
+ <text x="48" y="143" font-size="22" fill="#172b4d" font-weight="700"
text-anchor="start">Upserts → non-partitioned Paimon table</text>
+ <text x="848" y="143" font-size="19" fill="#526277" font-weight="400"
text-anchor="end">Primary key: pk</text>
+ <text x="28" y="206" font-size="16" fill="#526277" font-weight="700"
text-anchor="start">PAIMON SNAPSHOT / TAG</text>
+ <text x="520" y="206" font-size="16" fill="#526277" font-weight="700"
text-anchor="start">HIVE PARTITION VIEW</text>
+ <rect x="28" y="226" width="337" height="95" rx="8" fill="#eaf2ff"
stroke="#bfd2ed" stroke-width="1.5"/>
+ <text x="44" y="258" font-size="21" fill="#2463b4" font-weight="700"
text-anchor="start">Tag: 2023-10-16</text>
+ <text x="44" y="291" font-size="19" fill="#172b4d" font-weight="400"
text-anchor="start">pk/value: 1/10, 2/20</text>
+ <path d="M 379 273 H 504" fill="none" stroke="#526277" stroke-width="2"
marker-end="url(#arrow)"/>
+ <rect x="520" y="226" width="352" height="95" rx="8" fill="#e8f7f2"
stroke="#b8dcd2" stroke-width="1.5"/>
+ <text x="536" y="258" font-size="22" fill="#087f6e" font-weight="700"
text-anchor="start">dt=2023-10-16</text>
+ <text x="536" y="291" font-size="19" fill="#172b4d" font-weight="400"
text-anchor="start">Fixed at the tagged snapshot</text>
+ <rect x="28" y="350" width="337" height="95" rx="8" fill="#eaf2ff"
stroke="#bfd2ed" stroke-width="1.5"/>
+ <text x="44" y="382" font-size="21" fill="#2463b4" font-weight="700"
text-anchor="start">Tag: 2023-10-17</text>
+ <text x="44" y="415" font-size="19" fill="#172b4d" font-weight="400"
text-anchor="start">pk/value: 1/11, 2/20, 3/30</text>
+ <path d="M 379 397 H 504" fill="none" stroke="#526277" stroke-width="2"
marker-end="url(#arrow)"/>
+ <rect x="520" y="350" width="352" height="95" rx="8" fill="#e8f7f2"
stroke="#b8dcd2" stroke-width="1.5"/>
+ <text x="536" y="382" font-size="22" fill="#087f6e" font-weight="700"
text-anchor="start">dt=2023-10-17</text>
+ <text x="536" y="415" font-size="19" fill="#172b4d" font-weight="400"
text-anchor="start">Fixed at the tagged snapshot</text>
+ <rect x="28" y="474" width="337" height="97" rx="8" fill="#fff7e6"
stroke="#e4cb92" stroke-width="1.5" stroke-dasharray="7 5"/>
+ <text x="44" y="507" font-size="21" fill="#946200" font-weight="700"
text-anchor="start">Eligible retained snapshot</text>
+ <text x="44" y="542" font-size="18" fill="#172b4d" font-weight="400"
text-anchor="start">Period based on processing time</text>
+ <path d="M 379 522 H 504" fill="none" stroke="#526277" stroke-width="2"
marker-end="url(#arrow)" stroke-dasharray="7 5"/>
+ <rect x="520" y="474" width="352" height="97" rx="8" fill="#fff7e6"
stroke="#e4cb92" stroke-width="1.5" stroke-dasharray="7 5"/>
+ <text x="536" y="507" font-size="22" fill="#946200" font-weight="700"
text-anchor="start">Preview partition</text>
+ <text x="536" y="542" font-size="19" fill="#172b4d" font-weight="400"
text-anchor="start">Can change until a tag exists</text>
+ <text x="28" y="610" font-size="18" fill="#526277" font-weight="400"
text-anchor="start">dt is a Hive metadata field. Paimon data is not physically
repartitioned by tag date.</text>
+ <text x="28" y="638" font-size="18" fill="#526277" font-weight="400"
text-anchor="start">Create and retain real tags for reproducible history;
preview alone does not retain data.</text>
+ </g>
+</svg>
