This is an automated email from the ASF dual-hosted git repository.
JingsongLi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/paimon.git
The following commit(s) were added to refs/heads/master by this push:
new c2aea2bac1 [docs] Clarify snapshot record count semantics (#9202)
c2aea2bac1 is described below
commit c2aea2bac1820e25744e7f85e9944a2130dd211f
Author: wangwj <[email protected]>
AuthorDate: Thu Aug 13 21:56:08 2026 +0800
[docs] Clarify snapshot record count semantics (#9202)
---
docs/docs/concepts/spec/snapshot.md | 9 +++++++--
docs/docs/concepts/system-tables.mdx | 6 ++++++
docs/docs/pypaimon/system-tables.md | 9 +++++++--
paimon-api/src/main/java/org/apache/paimon/Snapshot.java | 4 ++--
.../src/test/java/org/apache/paimon/append/BlobTableTest.java | 5 +++++
.../java/org/apache/paimon/append/MultipleBlobTableTest.java | 5 +++++
6 files changed, 32 insertions(+), 6 deletions(-)
diff --git a/docs/docs/concepts/spec/snapshot.md
b/docs/docs/concepts/spec/snapshot.md
index d09db56a51..a9ae2586b0 100644
--- a/docs/docs/concepts/spec/snapshot.md
+++ b/docs/docs/concepts/spec/snapshot.md
@@ -58,11 +58,16 @@ Snapshot File is JSON, it includes:
11. commitKind: type of changes in this snapshot, including append, compact,
overwrite and analyze.
12. timeMillis: commit time millis.
13. logOffsets: commit log offsets.
-14. totalRecordCount: record count of all changes occurred in this snapshot.
-15. deltaRecordCount: record count of all new changes occurred in this
snapshot.
+14. totalRecordCount: unmerged record count of all live data files in this
snapshot.
+15. deltaRecordCount: net change of the unmerged record count from data files
added and deleted in this snapshot.
16. changelogRecordCount: record count of all changelog produced in this
snapshot.
17. watermark: watermark for input records, from Flink watermark mechanism,
Long.MIN_VALUE if there is no watermark.
18. statistics: stats file name for statistics of this table.
19. properties: additional key-value properties of this snapshot.
20. nextRowId: next row id for row tracking.
21. operation: logical operation type, e.g. WRITE, DELETE, UPDATE, MERGE. Null
if not set.
+
+The record counts are calculated per data file and are not logical row counts.
For example, a
+Dedicated Format table stores the regular columns and each dedicated BLOB
column in separate data
+files. Appending `N` logical rows to a table with one dedicated BLOB column
therefore increases the
+`deltaRecordCount` by `2 * N`. Use `COUNT(*)` when you need the logical row
count.
diff --git a/docs/docs/concepts/system-tables.mdx
b/docs/docs/concepts/system-tables.mdx
index 67fb73a541..0bb6d2629c 100644
--- a/docs/docs/concepts/system-tables.mdx
+++ b/docs/docs/concepts/system-tables.mdx
@@ -64,6 +64,12 @@ SELECT * FROM my_table$snapshots;
By querying the snapshots table, you can know the commit and expiration
information about that table and time travel through the data.
+`total_record_count` and `delta_record_count` are unmerged counts calculated
from data files, not
+logical row counts. All dedicated data files contribute to these values. For
example, appending `N`
+logical rows to a table with one [dedicated BLOB
column](../multimodal-table/blob#storage-layout)
+adds `N` records to the regular data files and `N` records to the BLOB files,
so
+`delta_record_count` increases by `2 * N`. Use `COUNT(*)` when you need the
logical row count.
+
### Schemas Table
You can query the historical schemas of the table through schemas table.
diff --git a/docs/docs/pypaimon/system-tables.md
b/docs/docs/pypaimon/system-tables.md
index fdbf287745..35f2532afd 100644
--- a/docs/docs/pypaimon/system-tables.md
+++ b/docs/docs/pypaimon/system-tables.md
@@ -86,12 +86,17 @@ One row per persisted snapshot.
| `base_manifest_list` | STRING NOT NULL |
|
| `delta_manifest_list` | STRING NOT NULL |
|
| `changelog_manifest_list` | STRING |
|
-| `total_record_count` | BIGINT |
|
-| `delta_record_count` | BIGINT |
|
+| `total_record_count` | BIGINT | Unmerged record count of all
live data files; not a logical row count |
+| `delta_record_count` | BIGINT | Net change of the unmerged
record count from added and deleted data files |
| `changelog_record_count` | BIGINT |
|
| `watermark` | BIGINT |
|
| `next_row_id` | BIGINT |
|
+Dedicated Format columns are stored in separate data files, and each of those
files contributes to
+the snapshot record counts. For example, appending `N` logical rows to a table
with one dedicated
+BLOB column increases `delta_record_count` by `2 * N`. Use `COUNT(*)` when you
need the logical row
+count.
+
### `$schemas`
Every committed schema version, with `fields` / `partition_keys` /
diff --git a/paimon-api/src/main/java/org/apache/paimon/Snapshot.java
b/paimon-api/src/main/java/org/apache/paimon/Snapshot.java
index d34b061b78..2f98783ecc 100644
--- a/paimon-api/src/main/java/org/apache/paimon/Snapshot.java
+++ b/paimon-api/src/main/java/org/apache/paimon/Snapshot.java
@@ -148,11 +148,11 @@ public class Snapshot implements Serializable {
@JsonProperty(FIELD_TIME_MILLIS)
protected final long timeMillis;
- // record count of all changes occurred in this snapshot
+ // unmerged record count of all live data files in this snapshot
@JsonProperty(FIELD_TOTAL_RECORD_COUNT)
protected final long totalRecordCount;
- // record count of all new changes occurred in this snapshot
+ // net change of the unmerged record count from data files added and
deleted in this snapshot
@JsonProperty(FIELD_DELTA_RECORD_COUNT)
protected final long deltaRecordCount;
diff --git
a/paimon-core/src/test/java/org/apache/paimon/append/BlobTableTest.java
b/paimon-core/src/test/java/org/apache/paimon/append/BlobTableTest.java
index 0c6833ce28..caa268e0ca 100644
--- a/paimon-core/src/test/java/org/apache/paimon/append/BlobTableTest.java
+++ b/paimon-core/src/test/java/org/apache/paimon/append/BlobTableTest.java
@@ -134,6 +134,11 @@ public class BlobTableTest extends TableTestBase {
commitDefault(writeDataDefault(1000, 1));
+
assertThat(getTableDefault().snapshotManager().latestSnapshot().totalRecordCount())
+ .isEqualTo(2000L);
+
assertThat(getTableDefault().snapshotManager().latestSnapshot().deltaRecordCount())
+ .isEqualTo(2000L);
+
AtomicInteger integer = new AtomicInteger(0);
List<DataFileMeta> filesMetas =
diff --git
a/paimon-core/src/test/java/org/apache/paimon/append/MultipleBlobTableTest.java
b/paimon-core/src/test/java/org/apache/paimon/append/MultipleBlobTableTest.java
index 8275558526..a8fd2cc328 100644
---
a/paimon-core/src/test/java/org/apache/paimon/append/MultipleBlobTableTest.java
+++
b/paimon-core/src/test/java/org/apache/paimon/append/MultipleBlobTableTest.java
@@ -62,6 +62,11 @@ public class MultipleBlobTableTest extends TableTestBase {
commitDefault(writeDataDefault(1000, 1));
+
assertThat(getTableDefault().snapshotManager().latestSnapshot().totalRecordCount())
+ .isEqualTo(3000L);
+
assertThat(getTableDefault().snapshotManager().latestSnapshot().deltaRecordCount())
+ .isEqualTo(3000L);
+
AtomicInteger integer = new AtomicInteger(0);
FileStoreTable table = getTableDefault();
