This is an automated email from the ASF dual-hosted git repository.
sarvekshayr pushed a commit to branch HDDS-9225-website-v2
in repository https://gitbox.apache.org/repos/asf/ozone-site.git
The following commit(s) were added to refs/heads/HDDS-9225-website-v2 by this
push:
new ccf4d9d87 HDDS-14295. [Website v2] [Docs] [Administrator Guide] Ozone
command line tools (#256)
ccf4d9d87 is described below
commit ccf4d9d87b89c534a03efb21bd2a6d480f67246e
Author: Russole <[email protected]>
AuthorDate: Thu Jan 29 16:41:42 2026 +0800
HDDS-14295. [Website v2] [Docs] [Administrator Guide] Ozone command line
tools (#256)
Co-authored-by: Wei-Chiu Chuang <[email protected]>
---
cspell.yaml | 8 +
.../03-operations/11-tools/01-ozone-repair.md | 274 ++++++++++++++++++
.../03-operations/11-tools/02-ozone-admin.md | 60 ++++
.../03-ozone-debug/01-ldb-tool.md} | 312 +++++++++++----------
.../11-tools/03-ozone-debug/02-debug-om.md | 42 +++
.../11-tools/03-ozone-debug/03-debug-datanode.md | 19 ++
.../11-tools/03-ozone-debug/04-debug-replicas.md | 101 +++++++
.../11-tools/03-ozone-debug/05-ratis-log-parser.md | 18 ++
.../03-ozone-debug/06-audit-parser-exact.md | 59 ++++
.../07-container-replica-debugger-tool.md | 250 +++++++++++++++++
.../11-tools/03-ozone-debug/README.mdx | 26 ++
.../03-operations/11-tools/README.mdx | 55 ++++
12 files changed, 1076 insertions(+), 148 deletions(-)
diff --git a/cspell.yaml b/cspell.yaml
index 15b22642e..c0ec4278c 100644
--- a/cspell.yaml
+++ b/cspell.yaml
@@ -212,6 +212,14 @@ words:
- utilisation
- utilised
- hotspots
+- activetimebyseconds
+- charindex
+- thecount
+- debugability
+- auditparser
+- checknative
+- subpages
+- analyse
# Infrastructure and hardware terms
- Chrony
- SAS
diff --git
a/docs/05-administrator-guide/03-operations/11-tools/01-ozone-repair.md
b/docs/05-administrator-guide/03-operations/11-tools/01-ozone-repair.md
new file mode 100644
index 000000000..b687ef210
--- /dev/null
+++ b/docs/05-administrator-guide/03-operations/11-tools/01-ozone-repair.md
@@ -0,0 +1,274 @@
+# Ozone Repair
+
+Ozone Repair (`ozone repair`) is an advanced tool to repair Ozone. The nodes
being repaired must be stopped before the tool is run.
+
+:::note
+All repair commands support a `--dry-run` option which allows a user to see
what repair the command will be performing without actually making any changes
to the cluster.
+
+Use the `--force` flag to override the running service check in false-positive
cases.
+:::
+
+```bash
+Usage: ozone repair [-hV] [--verbose] [-conf=<configurationPath>]
+ [-D=<String=String>]... [COMMAND]
+Advanced tool to repair Ozone. The nodes being repaired must be stopped before
+the tool is run.
+ -conf=<configurationPath>
+
+ -D, --set=<String=String>
+
+ -h, --help Show this help message and exit.
+ -V, --version Print version information and exit.
+ --verbose More verbose output. Show the stack trace of the errors.
+Commands:
+ datanode Tools to repair Datanode
+ ldb Operational tool to repair ldb.
+ om Operational tool to repair OM.
+ scm Operational tool to repair SCM.
+```
+
+For more detailed usage see the output of --help for each of the subcommands.
+
+## ozone repair Datanode
+
+Operational tool to repair Datanode.
+
+### upgrade-container-schema
+
+Upgrade all schema V2 containers to schema V3 for a Datanode in offline mode.
+Optionally takes `--volume` option to specify which volume needs the upgrade.
+
+## ozone repair ldb
+
+Operational tool to repair ldb.
+
+### compact
+
+Compact a column family in the DB to clean up tombstones while the service is
offline.
+
+```bash
+Usage: ozone repair ldb compact [-hV] [--dry-run] [--force] [--verbose]
+ --cf=<columnFamilyName> --db=<dbPath>
+CLI to compact a column-family in the DB while the service is offline.
+Note: If om.db is compacted with this tool then it will negatively impact the
+Ozone Manager's efficient snapshot diff.
+ --cf, --column-family, --column_family=<columnFamilyName>
+ Column family name
+ --db=<dbPath> Database File Path
+```
+
+## ozone repair OM
+
+Operational tool to repair OM.
+
+### Subcommands under OM
+
+- FSO-tree
+- snapshot
+- update-transaction
+- quota
+- compact
+- skip-ratis-transaction
+
+### FSO-tree
+
+Identify and repair a disconnected FSO tree by marking unreferenced entries
for deletion.
+Reports the reachable, unreachable (pending delete) and unreferenced
(orphaned) directories and files.
+OM should be stopped while this tool is run.
+
+```bash
+Usage: ozone repair om fso-tree [-hV] [--dry-run] [--force] [--verbose]
+ [-b=<bucketFilter>] --db=<omDBPath>
+ [-v=<volumeFilter>]
+Identify and repair a disconnected FSO tree by marking unreferenced entries for
+deletion. OM should be stopped while this tool is run.
+ -b, --bucket=<bucketFilter>
+ Filter by bucket name
+ --db=<omDBPath> Path to OM RocksDB
+ -v, --volume=<volumeFilter>
+ Filter by volume name. Add '/' before the volume name.
+
+```
+
+### snapshot
+
+Subcommand for all snapshot related repairs.
+
+#### chain
+
+Update global and path previous snapshot for a snapshot in case snapshot chain
is corrupted.
+
+```bash
+Usage: ozone repair om snapshot chain [-hV] [--dry-run] [--force] [--verbose]
+ --db=<dbPath>
+ --gp=<globalPreviousSnapshotId>
+ --pp=<pathPreviousSnapshotId> <value>
+ <snapshotName>
+CLI to update global and path previous snapshot for a snapshot in case snapshot
+chain is corrupted.
+ <value> URI of the bucket (format: volume/bucket).
+ <snapshotName> Snapshot name to update
+ --db=<dbPath> Database File Path
+ --gp, --global-previous=<globalPreviousSnapshotId>
+ Global previous snapshotId to set for the given snapshot
+ --pp, --path-previous=<pathPreviousSnapshotId>
+ Path previous snapshotId to set for the given snapshot
+```
+
+### update-transaction
+
+To avoid modifying Ratis logs and only update the latest applied transaction,
use `update-transaction` command.
+This updates the highest transaction index in the OM transaction info table.
+
+```bash
+Usage: ozone repair om update-transaction [-hV] [--dry-run] [--force]
+ [--verbose] --db=<dbPath> --index=<highestTransactionIndex>
+ --term=<highestTransactionTerm>
+CLI to update the highest index in transaction info table.
+ --db=<dbPath> Database File Path
+ --index=<highestTransactionIndex>
+ Highest index to set. The input should be non-zero long
+ integer.
+ --term=<highestTransactionTerm>
+ Highest term to set. The input should be non-zero long
+ integer.
+
+```
+
+### quota
+
+Operational tool to repair quota in OM DB.
+
+#### start
+
+To trigger quota repair use the `start` command.
+
+```bash
+Usage: ozone repair om quota start [-hV] [--dry-run] [--force] [--verbose]
+ [--buckets=<buckets>]
+ [--service-host=<omHost>]
+ [--service-id=<omServiceId>]
+CLI to trigger quota repair.
+ --buckets=<buckets> start quota repair for specific buckets. Input will
+ be list of uri separated by comma as
+ /<volume>/<bucket>[,...]
+ --service-host=<omHost>
+ Ozone Manager Host. If OM HA is enabled, use
+ --service-id instead. If you must use
+ --service-host with OM HA, this must point
+ directly to the leader OM. This option is
+ required when --service-id is not provided or
+ when HA is not enabled.
+ --service-id, --om-service-id=<omServiceId>
+ Ozone Manager Service ID
+
+```
+
+#### status
+
+Get the status of last triggered quota repair.
+
+```bash
+Usage: ozone repair om quota status [-hV] [--verbose]
+ [--service-host=<omHost>]
+ [--service-id=<omServiceId>]
+CLI to get the status of last trigger quota repair if available.
+ --service-host=<omHost>
+ Ozone Manager Host. If OM HA is enabled, use --service-id
+ instead. If you must use --service-host with OM HA, this
+ must point directly to the leader OM. This option is
+ required when --service-id is not provided or when HA is
+ not enabled.
+ --service-id, --om-service-id=<omServiceId>
+ Ozone Manager Service ID
+
+```
+
+#### compact
+
+Compact a column family in the OM DB to clean up tombstones. The compaction
happens asynchronously. Requires admin privileges.
+
+```bash
+Usage: ozone repair om compact [-hV] [--dry-run] [--force] [--verbose]
+ --cf=<columnFamilyName> [--node-id=<nodeId>]
+ [--service-id=<omServiceId>]
+CLI to compact a column family in the om.db. The compaction happens
+asynchronously. Requires admin privileges.
+ --cf, --column-family, --column_family=<columnFamilyName>
+ Column family name
+ --node-id=<nodeId> NodeID of the OM for which db needs to be compacted.
+ --service-id, --om-service-id=<omServiceId>
+ Ozone Manager Service ID
+
+```
+
+#### skip-ratis-transaction, srt
+
+Omit a raft log in a Ratis segment file by replacing the specified index with
a dummy EchoOM command.
+This is an offline tool meant to be used only when all 3 OMs crash on the same
transaction.
+If the issue is isolated to one OM, manually copy the DB from a healthy OM
instead.
+
+```bash
+Usage: ozone repair om skip-ratis-transaction [-hV] [--dry-run] [--force]
+ [--verbose] -b=<backupDir> --index=<index> (-s=<segmentFile> |
+ -d=<logDir>)
+CLI to omit a raft log in a ratis segment file. The raft log at the index
+specified is replaced with an EchoOM command (which is a dummy command). It is
+an offline command i.e., doesn't require OM to be running. The command should
+be run for the same transaction on all 3 OMs only when all the OMs are crashing
+while applying the same transaction. If only one OM is crashing and the other
+OMs have executed the log successfully, then the DB should be manually copied
+from one of the good OMs to the crashing OM instead.
+ -b, --backup=<backupDir> Directory to put the backup of the original
+ repaired segment file before the repair.
+ -d, --ratis-log-dir=<logDir>
+ Path of the ratis log directory
+ --index=<index> Index of the failing transaction that should be
+ removed
+ -s, --segment-path=<segmentFile>
+ Path of the input segment file
+```
+
+## ozone repair SCM
+
+Operational tool to repair SCM.
+
+### Subcommands under SCM
+
+- cert
+- update-transaction
+
+### cert
+
+Subcommand for all certificate related repairs on SCM.
+
+#### recover
+
+Recover Deleted SCM Certificate from RocksDB.
+
+```bash
+Usage: ozone repair scm cert recover [-hV] [--dry-run] [--force] [--verbose]
+ --db=<dbPath>
+Recover Deleted SCM Certificate from RocksDB
+ --db=<dbPath> SCM DB Path
+```
+
+### update-transaction
+
+To avoid modifying Ratis logs and only update the latest applied transaction,
use update-transaction command.
+This updates the highest transaction index in the SCM transaction info table.
+
+```bash
+Usage: ozone repair scm update-transaction [-hV] [--dry-run] [--force]
+ [--verbose] --db=<dbPath> --index=<highestTransactionIndex>
+ --term=<highestTransactionTerm>
+CLI to update the highest index in transaction info table.
+ --db=<dbPath> Database File Path
+ --index=<highestTransactionIndex>
+ Highest index to set. The input should be non-zero long
+ integer.
+ --term=<highestTransactionTerm>
+ Highest term to set. The input should be non-zero long
+ integer.
+
+```
diff --git
a/docs/05-administrator-guide/03-operations/11-tools/02-ozone-admin.md
b/docs/05-administrator-guide/03-operations/11-tools/02-ozone-admin.md
new file mode 100644
index 000000000..30bddf1be
--- /dev/null
+++ b/docs/05-administrator-guide/03-operations/11-tools/02-ozone-admin.md
@@ -0,0 +1,60 @@
+# Ozone Admin
+
+`ozone admin` command is a collection of tools intended to be used only by
admins.
+
+A quick overview of the available functionalities:
+
+- `ozone admin safemode`
+ Check the safe mode status and force entering or leaving safe mode.
+ The `--verbose` option prints the validation status of all rules that
evaluate safe mode status.
+
+- `ozone admin container`
+ Containers are the unit of replication.
+ Subcommands help debug the current container state (list / get / create / …).
+
+- `ozone admin pipeline`
+ Helps check available pipelines (Datanodes sets).
+
+- `ozone admin datanode`
+ Provides information about Datanodes.
+
+- `ozone admin printTopology`
+ Displays rack-awareness related information.
+
+- `ozone admin replicationmanager`
+ Checks replication status and can start or stop replication in emergencies.
+
+- `ozone admin om`
+ Ozone Manager HA related tools to retrieve cluster information.
+
+For more detailed usage, see the output of `--help`.
+
+```bash
+$ ozone admin --help
+
+Usage: ozone admin [-hV] [--verbose] [-conf=<configurationPath>]
+ [-D=<String=String>]... [COMMAND]
+
+Developer tools for Ozone Admin operations
+
+Options:
+ -conf=<configurationPath>
+ -D, --set=<String=String>
+ -h, --help Show this help message and exit.
+ -V, --version Print version information and exit.
+ --verbose More verbose output. Show the stack trace of the errors.
+
+Commands:
+ containerbalancer ContainerBalancer specific operations
+ replicationmanager ReplicationManager specific operations
+ safemode Safe mode specific operations
+ printTopology Print a tree of the network topology as reported by SCM
+ cert Certificate related operations
+ container Container specific operations
+ datanode Datanode specific operations
+ pipeline Pipeline specific operations
+ namespace Namespace Summary specific admin operations
+ om Ozone Manager specific admin operations
+ reconfig Dynamically reconfigure server without restarting it
+ scm Ozone Storage Container Manager specific admin operations
+```
diff --git a/docs/05-administrator-guide/03-operations/13-ldb.md
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/01-ldb-tool.md
similarity index 70%
rename from docs/05-administrator-guide/03-operations/13-ldb.md
rename to
docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/01-ldb-tool.md
index 86b98ab9b..767f3daaa 100644
--- a/docs/05-administrator-guide/03-operations/13-ldb.md
+++
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/01-ldb-tool.md
@@ -6,7 +6,7 @@ sidebar_label: LDB Tool
Ozone heavily uses RocksDB for storing metadata.
This tool helps parse the contents of RocksDB belonging to Ozone Roles.
-Supported DB's : Ozone Manager (`om.db`) , StorageContainerManager (`scm.db`),
Datanode/Container (`container.db`)
+Supported DB's : Ozone Manager (`om.db`) , StorageContainerManager (`scm.db`),
Datanode/Container (`container.db`)
Below is the usage:
```bash
@@ -117,45 +117,50 @@ ozone debug ldb --db=/path/to/om.db scan --cf=volumeTable
--startkey=vol3 --endk
```
```json
-{ "/vol3": {
- "metadata" : { },
- "objectID" : -9999,
- "updateID" : 4000,
- "adminName" : "om",
- "ownerName" : "om",
- "volume" : "vol3",
- "creationTime" : 1707192335309,
- "modificationTime" : 1714057412205,
- "quotaInBytes" : 22854448694951936,
- "quotaInNamespace" : 100000000,
- "usedNamespace" : 1,
- "acls" : [ {
- "type" : "USER",
- "name" : "om",
- "aclScope" : "ACCESS"
- } ],
- "refCount" : 0
-}
-, "/vol4": {
- "metadata" : { },
- "objectID" : -888,
- "updateID" : 5000,
- "adminName" : "om",
- "ownerName" : "om",
- "volume" : "vol4",
- "creationTime" : 1696280979907,
- "modificationTime" : 1696280979907,
- "quotaInBytes" : 2251799813685250,
- "quotaInNamespace" : 100000000,
- "usedNamespace" : 2,
- "acls" : [ {
- "type" : "USER",
- "name" : "om",
- "aclScope" : "ACCESS"
- } ],
- "refCount" : 0
-}
+{
+ "/vol3": {
+ "metadata": {},
+ "objectID": -9999,
+ "updateID": 4000,
+ "adminName": "om",
+ "ownerName": "om",
+ "volume": "vol3",
+ "creationTime": 1707192335309,
+ "modificationTime": 1714057412205,
+ "quotaInBytes": 22854448694951936,
+ "quotaInNamespace": 100000000,
+ "usedNamespace": 1,
+ "acls": [
+ {
+ "type": "USER",
+ "name": "om",
+ "aclScope": "ACCESS"
+ }
+ ],
+ "refCount": 0
+ },
+ "/vol4": {
+ "metadata": {},
+ "objectID": -888,
+ "updateID": 5000,
+ "adminName": "om",
+ "ownerName": "om",
+ "volume": "vol4",
+ "creationTime": 1696280979907,
+ "modificationTime": 1696280979907,
+ "quotaInBytes": 2251799813685250,
+ "quotaInNamespace": 100000000,
+ "usedNamespace": 2,
+ "acls": [
+ {
+ "type": "USER",
+ "name": "om",
+ "aclScope": "ACCESS"
+ }
+ ],
+ "refCount": 0
}
+}
```
### `--fields`
@@ -167,21 +172,28 @@ ozone debug ldb --db=/path/to/om.db scan --cf=keyTable
-l=1 --fields="volumeName
```
```json
-{ "/vol1/ozone-legacy-bucket/10T-1-terasort-input/": {
- "keyLocationVersions" : [ {
- "version" : 0
- } ],
- "keyName" : "10T-1-terasort-input/",
- "bucketName" : "ozone-legacy-bucket",
- "acls" : [ {
- "name" : "om"
- }, {
- "name" : "scm"
- }, {
- "name" : "testuser"
- } ],
- "volumeName" : "vol1"
-}
+{
+ "/vol1/ozone-legacy-bucket/10T-1-terasort-input/": {
+ "keyLocationVersions": [
+ {
+ "version": 0
+ }
+ ],
+ "keyName": "10T-1-terasort-input/",
+ "bucketName": "ozone-legacy-bucket",
+ "acls": [
+ {
+ "name": "om"
+ },
+ {
+ "name": "scm"
+ },
+ {
+ "name": "testuser"
+ }
+ ],
+ "volumeName": "vol1"
+ }
}
```
@@ -221,27 +233,29 @@ ozone debug ldb --db=/path/to/om.db scan --cf=volumeTable
--filter="usedNamespac
}
],
"refCount": 0
+ },
+ "/vol5": {
+ "metadata": {},
+ "objectID": -956599,
+ "updateID": 45600,
+ "adminName": "om",
+ "ownerName": "om",
+ "volume": "vol5",
+ "creationTime": 1807192332309,
+ "modificationTime": 1914057410005,
+ "quotaInBytes": 7785494951936,
+ "quotaInNamespace": 100000000,
+ "usedNamespace": 2,
+ "acls": [
+ {
+ "type": "USER",
+ "name": "om",
+ "aclScope": "ACCESS"
+ }
+ ],
+ "refCount": 0
}
-, "/vol5": {
- "metadata" : { },
- "objectID" : -956599,
- "updateID" : 45600,
- "adminName" : "om",
- "ownerName" : "om",
- "volume" : "vol5",
- "creationTime" : 1807192332309,
- "modificationTime" : 1914057410005,
- "quotaInBytes" : 7785494951936,
- "quotaInNamespace" : 100000000,
- "usedNamespace" : 2,
- "acls" : [ {
- "type" : "USER",
- "name" : "om",
- "aclScope" : "ACCESS"
- } ],
- "refCount" : 0
}
- }
```
Using `lesser` operator (`greater` operator can also be used in the same way):
@@ -273,7 +287,7 @@ ozone debug ldb --db=/path/to/om.db scan --cf=volumeTable
--filter="usedNamespac
],
"refCount": 0
}
- }
+}
```
Using `regex` operator:
@@ -305,7 +319,7 @@ ozone debug ldb --db=/path/to/om.db scan --cf=volumeTable
--filter="volume:regex
],
"refCount": 0
}
- }
+}
```
Using multiple filters:
@@ -337,7 +351,7 @@ ozone debug ldb --db=/path/to/om.db scan --cf=volumeTable
--filter="usedNamespac
],
"refCount": 0
}
- }
+}
```
## value-schema command
@@ -351,28 +365,28 @@ ozone debug ldb --db=/data/metadata/om.db value-schema
--cf=keyTable --depth=1
```json
{
- "OmKeyInfo" : {
- "bucketName" : "String",
- "metadata" : "struct",
- "fileName" : "String",
- "creationTime" : "long",
- "isFile" : "boolean",
- "acls" : "struct",
- "keyName" : "String",
- "replicationConfig" : "struct",
- "encInfo" : "struct",
- "dataSize" : "long",
- "tags" : "struct",
- "keyLocationVersions" : "struct",
- "updateID" : "long",
- "ownerName" : "String",
- "modificationTime" : "long",
- "parentObjectID" : "long",
- "volumeName" : "String",
- "fileChecksum" : "struct",
- "objectID" : "long"
+ "OmKeyInfo": {
+ "bucketName": "String",
+ "metadata": "struct",
+ "fileName": "String",
+ "creationTime": "long",
+ "isFile": "boolean",
+ "acls": "struct",
+ "keyName": "String",
+ "replicationConfig": "struct",
+ "encInfo": "struct",
+ "dataSize": "long",
+ "tags": "struct",
+ "keyLocationVersions": "struct",
+ "updateID": "long",
+ "ownerName": "String",
+ "modificationTime": "long",
+ "parentObjectID": "long",
+ "volumeName": "String",
+ "fileChecksum": "struct",
+ "objectID": "long"
+ }
}
- }
```
```bash
@@ -381,66 +395,66 @@ ozone debug ldb --db=/data/metadata/om.db value-schema
--cf=keyTable
```json
{
- "OmKeyInfo" : {
- "bucketName" : "String",
- "metadata" : { },
- "fileName" : "String",
- "creationTime" : "long",
- "isFile" : "boolean",
- "acls" : {
- "toStringMethod" : { },
- "hashCodeMethod" : { },
- "name" : "String",
- "type" : {
- "name" : "String",
- "value" : "String",
- "ordinal" : "int"
+ "OmKeyInfo": {
+ "bucketName": "String",
+ "metadata": {},
+ "fileName": "String",
+ "creationTime": "long",
+ "isFile": "boolean",
+ "acls": {
+ "toStringMethod": {},
+ "hashCodeMethod": {},
+ "name": "String",
+ "type": {
+ "name": "String",
+ "value": "String",
+ "ordinal": "int"
},
- "aclScope" : {
- "name" : "String",
- "ordinal" : "int"
+ "aclScope": {
+ "name": "String",
+ "ordinal": "int"
},
- "aclBits" : "int"
+ "aclBits": "int"
},
- "keyName" : "String",
- "replicationConfig" : { },
- "encInfo" : {
- "ezKeyVersionName" : "String",
- "keyName" : "String",
- "edek" : { },
- "cipherSuite" : {
- "unknownValue" : {
- "value" : "int"
+ "keyName": "String",
+ "replicationConfig": {},
+ "encInfo": {
+ "ezKeyVersionName": "String",
+ "keyName": "String",
+ "edek": {},
+ "cipherSuite": {
+ "unknownValue": {
+ "value": "int"
},
- "name" : "String",
- "algoBlockSize" : "int",
- "ordinal" : "int"
+ "name": "String",
+ "algoBlockSize": "int",
+ "ordinal": "int"
},
- "version" : {
- "unknownValue" : {
- "value" : "int"
+ "version": {
+ "unknownValue": {
+ "value": "int"
},
- "name" : "String",
- "description" : "String",
- "version" : "int",
- "ordinal" : "int"
+ "name": "String",
+ "description": "String",
+ "version": "int",
+ "ordinal": "int"
},
- "iv" : { }
+ "iv": {}
},
- "dataSize" : "long",
- "tags" : { },
- "keyLocationVersions" : {
- "isMultipartKey" : "boolean",
- "locationVersionMap" : { },
- "version" : "long"
+ "dataSize": "long",
+ "tags": {},
+ "keyLocationVersions": {
+ "isMultipartKey": "boolean",
+ "locationVersionMap": {},
+ "version": "long"
},
- "updateID" : "long",
- "ownerName" : "String",
- "modificationTime" : "long",
- "parentObjectID" : "long",
- "volumeName" : "String",
- "fileChecksum" : { },
- "objectID" : "long"
+ "updateID": "long",
+ "ownerName": "String",
+ "modificationTime": "long",
+ "parentObjectID": "long",
+ "volumeName": "String",
+ "fileChecksum": {},
+ "objectID": "long"
}
}
```
@@ -452,3 +466,5 @@ ozone debug ldb --db=/data/metadata/om.db value-schema
--cf=keyTable
```bash
ozone debug ldb --db=/data/metadata/om.db checkpoint
--output=/tmp/om-checkpoint
```
+
+[Next >>](/docs/administrator-guide/operations/tools/ozone-debug/debug-om)
diff --git
a/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/02-debug-om.md
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/02-debug-om.md
new file mode 100644
index 000000000..d60791a93
--- /dev/null
+++
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/02-debug-om.md
@@ -0,0 +1,42 @@
+# Debug OM
+
+Debug commands related to OM.
+It has the following subcommands:
+
+## generate-compaction-dag
+
+Creates a DAG image of the current compaction log of an OM.db instance. It is
downloaded to the specified location.
+
+```bash
+Usage: ozone debug om generate-compaction-dag [-hV] [--verbose] --db=<dbPath>
+ -o=<imageLocation>
+Create an image of the current compaction log DAG. This command is an offline
+command. i.e., it can run on any instance of om.db and does not require OM to
+be up.
+ --db=<dbPath> Path to OM RocksDB
+ -h, --help Show this help message and exit.
+ -o, --output-file=<imageLocation>
+ Path to location at which image will be downloaded.
+ Should include the image file name with ".png"
+ extension.
+ -V, --version Print version information and exit.
+ --verbose More verbose output. Show the stack trace of the errors.
+```
+
+## prefix
+
+Parses the contents of a prefix.
+
+```bash
+Usage: ozone debug om prefix [--verbose] --bucket=<bucket> --db=<dbPath>
+ --path=<filePath> --volume=<volume>
+Parse prefix contents
+ --bucket=<bucket> bucket name
+ --db=<dbPath> Path to OM RocksDB
+ --path=<filePath> prefixFile Path
+ --verbose More verbose output. Show the stack trace of the
+ errors.
+ --volume=<volume> volume name
+```
+
+[Next
>>](/docs/administrator-guide/operations/tools/ozone-debug/debug-datanode)
diff --git
a/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/03-debug-datanode.md
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/03-debug-datanode.md
new file mode 100644
index 000000000..75f526a42
--- /dev/null
+++
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/03-debug-datanode.md
@@ -0,0 +1,19 @@
+# Debug Datanode
+
+Debug commands related to Datanode. Currently, only container replica related
commands are available.
+Following is the usage and the subcommands available under the `ozone debug
datanode container` command:
+
+```bash
+Usage: ozone debug datanode container [-hV] [--verbose] [COMMAND]
+Container replica specific operations to be executed on datanodes only
+ -h, --help Show this help message and exit.
+ -V, --version Print version information and exit.
+ --verbose More verbose output. Show the stack trace of the errors.
+Commands:
+ list Show container info of all container replicas on datanode
+ info Show container info of a container replica on datanode
+ export Export one container to a tarball
+ inspect Check the metadata of all container replicas on this datanode.
+```
+
+[Next
>>](/docs/administrator-guide/operations/tools/ozone-debug/debug-replicas)
diff --git
a/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/04-debug-replicas.md
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/04-debug-replicas.md
new file mode 100644
index 000000000..7bce1c28a
--- /dev/null
+++
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/04-debug-replicas.md
@@ -0,0 +1,101 @@
+# Debug Replicas
+
+Debug commands for retrieving information and performing various checks on the
key replicas in Datanodes.
+
+```bash
+Usage: ozone debug replicas [--verbose] [COMMAND]
+Debug commands for replica-related issues, retrieving replica information from
+the OM and performing checks over the network against a running cluster.
+ --verbose More verbose output. Show the stack trace of the errors.
+Commands:
+ chunk-info Returns chunk location information about an existing key
+ verify Run checks to verify data across replicas. By default prints only
+ the keys with failed checks.
+```
+
+## chunk-info
+
+For a given URI of a key, the command returns all the chunks’ location
information.
+
+```bash
+Usage: ozone debug replicas chunk-info [-hV] [--verbose] <value>
+Returns chunk location information about an existing key
+ <value> URI of the key (format: volume/bucket/key).
+ Ozone URI could either be a full URI or short URI.
+ Full URI should start with o3://, in case of non-HA
+ clusters it should be followed by the host name and
+ optionally the port number. In case of HA clusters
+ the service id should be used. Service id provides a
+ logical name for multiple hosts and it is defined
+ in the property ozone.om.service.ids.
+ Example of a full URI with host name and port number
+ for a key:
+ o3://omhostname:9862/vol1/bucket1/key1
+ With a service id for a volume:
+ o3://omserviceid/vol1/
+ Short URI should start from the volume.
+ Example of a short URI for a bucket:
+ vol1/bucket1
+ Any unspecified information will be identified from
+ the config files.
+
+ -h, --help Show this help message and exit.
+ -V, --version Print version information and exit.
+ --verbose More verbose output. Show the stack trace of the errors.
+```
+
+## verify
+
+Verify data across replicas. There are multiple checks available, which can be
selected using the command line options:
+
+```bash
+Usage: ozone debug replicas verify [-hV] [--all-results] [--verbose]
+
[--container-cache-size=<containerCacheSize>]
+ [-id=<scmServiceId>] [--scm=<scm>]
+ ([--checksums] [--block-existence]
+ [--container-state]) <uri>
+Run checks to verify data across replicas. By default prints only the keys with
+failed checks.
+ <uri> Ozone URI could either be a full URI or short URI.
+ Full URI should start with o3://, in case of non-HA
+ clusters it should be followed by the host name and
+ optionally the port number. In case of HA clusters
+ the service id should be used. Service id provides a
+ logical name for multiple hosts and it is defined
+ in the property ozone.om.service.ids.
+ Example of a full URI with host name and port number
+ for a key:
+ o3://omhostname:9862/vol1/bucket1/key1
+ With a service id for a volume:
+ o3://omserviceid/vol1/
+ Short URI should start from the volume.
+ Example of a short URI for a bucket:
+ vol1/bucket1
+ Any unspecified information will be identified from
+ the config files.
+
+ --all-results Print results for all passing and failing keys
+ --block-existence Check for block existence on datanodes.
+ --checksums Do client side data checksum validation of all
+ replicas.
+ --container-cache-size=<containerCacheSize>
+ Size (in number of containers) of the in-memory cache
+ for container state verification
+ '--container-state'. Default is 1 million
+ containers (which takes around 43MB). Value must be
+ greater than zero, otherwise the default of 1
+ million is considered. Note: This option is ignored
+ if '--container-state' option is not used.
+ --container-state Check the container and replica states. Containers in
+ [DELETING, DELETED] states, or its replicas in
+ [DELETED, UNHEALTHY, INVALID] states fail the
check.
+ -h, --help Show this help message and exit.
+ -id, --service-id=<scmServiceId>
+ ServiceId of SCM HA Cluster
+ --scm=<scm> The destination scm (host:port)
+ -V, --version Print version information and exit.
+ --verbose More verbose output. Show the stack trace of the
+ errors.
+```
+
+[Next
>>](/docs/administrator-guide/operations/tools/ozone-debug/ratis-log-parser)
diff --git
a/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/05-ratis-log-parser.md
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/05-ratis-log-parser.md
new file mode 100644
index 000000000..66074feb0
--- /dev/null
+++
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/05-ratis-log-parser.md
@@ -0,0 +1,18 @@
+# Ratis Log Parser
+
+The Ratis log parser tool takes a segment file as input and gives a
human-readable output.
+It can be used to parse Ratis logs from different components by specifying the
corresponding role.
+
+```bash
+Usage: ozone debug ratis parse [-hV] [--verbose] [--role=<role>]
-s=<segmentFile>
+Shell for printing Ratis Log in understandable text
+ -h, --help Show this help message and exit.
+ --role=<role> Component role for parsing. Values: om, scm, datanode
+ Default: generic
+ -s, --segmentPath, --segment-path=<segmentFile>
+ Path of the segment file
+ -V, --version Print version information and exit.
+ --verbose More verbose output. Show the stack trace of the errors.
+```
+
+[Next
>>](/docs/administrator-guide/operations/tools/ozone-debug/audit-parser-exact)
diff --git
a/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/06-audit-parser-exact.md
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/06-audit-parser-exact.md
new file mode 100644
index 000000000..aeb390c34
--- /dev/null
+++
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/06-audit-parser-exact.md
@@ -0,0 +1,59 @@
+# Audit Parser
+
+Audit Parser tool can be used for querying the Ozone audit logs.
+This tool creates a sqlite database at the specified path. If the database
+already exists, it will avoid creating a database.
+
+The database contains only one table called `audit` defined as:
+
+```sql
+CREATE TABLE IF NOT EXISTS audit (
+datetime text,
+level varchar(7),
+logger varchar(7),
+user text,
+ip text,
+op text,
+params text,
+result varchar(7),
+exception text,
+UNIQUE(datetime,level,logger,user,ip,op,params,result))
+```
+
+Usage:
+
+```bash
+ozone debug auditparser <path to db file> [COMMAND] [PARAM]
+```
+
+To load an audit log to database:
+
+```bash
+ozone debug auditparser <path to db file> load <path to audit log>
+```
+
+Load command creates the audit table described above.
+
+To run a custom read-only query:
+
+```bash
+ozone debug auditparser <path to db file> query <select query enclosed within
double quotes>
+```
+
+Audit Parser comes with a set of templates(most commonly used queries).
+
+To run a template query:
+
+```bash
+ozone debug auditparser <path to db file> template <templateName>
+```
+
+Following templates are available:
+
+| Template Name | Description | SQL
|
+| ----------------------- | -------------------------------------- |
-------------------------------------------------------------------------------------------------------------------------------------
|
+| top5users | Top 5 users | select
user,count(\*) as total from audit group by user order by total DESC limit 5
|
+| top5cmds | Top 5 commands | select
op,count(\*) as total from audit group by op order by total DESC limit 5
|
+| top5activetimebyseconds | Top 5 active times, grouped by seconds | select
substr(datetime,1,charindex(',',datetime)-1) as dt,count(\*) as thecount from
audit group by dt order by thecount DESC limit 5 |
+
+[Next
>>](/docs/administrator-guide/operations/tools/ozone-debug/container-replica-debugger-tool)
diff --git
a/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/07-container-replica-debugger-tool.md
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/07-container-replica-debugger-tool.md
new file mode 100644
index 000000000..32979a53a
--- /dev/null
+++
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/07-container-replica-debugger-tool.md
@@ -0,0 +1,250 @@
+# Container Replica Debugger Tool
+
+The tool processes container log files from Ozone Datanodes to track state
transitions. It provides CLI commands for querying containers based on
different attributes.
+
+## Background
+
+Containers are the most important part of Ozone. Most of the Ozone operations
happen on them. Containers in Ozone go through different states in their
lifecycle.
+In the past we have seen different issues on container state. To debug
problems we always use manual steps and identify problems w.r.t containers and
this takes a lot of time. To optimize debugability we can show historical
timeline and other information w.r.t containers.
+
+## Solution Approach
+
+An offline tool that would analyse the dn-container log files and help us find
issues related to the containers across the Datanodes in the cluster.
+
+### Component 1: Parser
+
+This component is responsible for processing log files, extracting relevant
log entries, and storing them in batches to be inserted into the database.
+
+**The command is as follows:**
+
+```bash
+ozone debug log container --db=<path to db> parse --path=<path to logs folder>
--thread-count=<n>
+```
+
+#### Log File Parsing and Validation
+
+- **Directory Traversal**: It recursively scans a specified directory for
container log files,only files with names matching the pattern
`dn-container-<roll>.log.<datanodeId>` are considered.Log files are parsed
concurrently using multiple threads for efficient processing.
+- **Line-by-Line Processing**: Each log line is split using a pipe delimiter
and parsed into key-value components.
+- **Field Extraction**: Key fields include:
+ - **Timestamp**
+ - **logLevel**: INFO, WARN, ERROR.
+ - **ID, BCSID, State, and Index**: Extracted from key-value pairs.
+- **Error Message Capture**: Any remaining unstructured part of the log line
is stored as `errorMessage`, especially relevant for WARN or ERROR level logs.
+- **Replication Index Filtering**: Only log entries with Index = 0 are
processed. This limits current processing to Ratis-replicated containers.
Future improvements may support EC replication.
+
+### Component 2: Database
+
+The tool uses a temporary SQLite database to store and manage information
extracted from container logs. This helps organize the data so that both the
complete history and the latest status of each container replica on a Datanode
can be analyzed easily.
+
+#### There are 2 major tables created/maintained by the tool
+
+1. **DatanodeContainerLogTable — Detailed Log History**
+ This table stores a complete history of state changes for each container
replica on every Datanode.
+ - Container ID
+ - Datanode ID
+ - State (such as OPEN, CLOSED, etc.)
+ - BCSID (Block Commit Sequence ID)
+ - Timestamp
+ - Log Level
+ - Error Message (if any)
+ - Index Value
+
+ The data in this table shows a complete timeline of state transitions and
BCSID changes, helping to trace how each container replica evolved over time.
+
+2. **ContainerLogTable — Latest Status Summary**
+ This table contains only the most recent state and BCSID for each unique
container and Datanode pair.
+ - Container ID
+ - Datanode ID
+ - Latest State
+ - Latest BCSID
+
+ This table provides a simplified view of the current state and BCSID of all
container replicas, which helps with quick status checks and summaries.
+
+### **Component 3: CLI Commands**
+
+#### **To List Containers Having Duplicate Open States**
+
+This Ozone debug CLI command helps to identify containers that were opened
more than the required number (for Ratis, it is 3) by tracking the first three
“OPEN” states and flagging any subsequent “OPEN” state as problematic if it
occurs on the same Datanode or on a different Datanode after the initial events.
+
+This command displays the **Container ID** along with the count of replicas in
the ‘OPEN’ state for each listed container. It also provides the total number
of containers that have duplicate ‘OPEN’ state entries.
+
+**The command is as follows:**
+
+```bash
+ozone debug log container --db=<path to db> duplicate-open
+```
+
+**Sample output:**
+
+```bash
+Container ID: 2187256 - OPEN state count: 4
+.
+.
+.
+Container ID: 12377064 - OPEN state count: 5
+Container ID: 12377223 - OPEN state count: 5
+Container ID: 12377631 - OPEN state count: 4
+Container ID: 12377904 - OPEN state count: 5
+Container ID: 12378161 - OPEN state count: 4
+Container ID: 12378352 - OPEN state count: 5
+Container ID: 12378789 - OPEN state count: 5
+Container ID: 12379337 - OPEN state count: 5
+Container ID: 12379489 - OPEN state count: 5
+Container ID: 12380526 - OPEN state count: 5
+Container ID: 12380898 - OPEN state count: 5
+Container ID: 12642718 - OPEN state count: 4
+Container ID: 12644806 - OPEN state count: 4
+Total containers that might have duplicate OPEN state : 1579
+```
+
+#### **To Display Details of a Single Container Along with Analysis**
+
+This Ozone debug CLI command provides a complete state transition history for
each replica of the container whose ID is provided via the command.
+The command also provides analysis over the container such as if the container
has issues like:
+
+- Duplicate OPEN states
+- Mismatched replication
+- Under-replication or over-replication
+- Unhealthy replicas
+- Open-unhealthy
+- Quasi-closed stuck containers
+
+The command provides key details such as:
+
+- Datanode id
+- Container id
+- BCSID
+- TimeStamp
+- Index Value
+- Message (if any associated with that particular replica)
+
+**The command is as follows:**
+
+```bash
+ozone debug log container --db=<path to database> info <containerID>
+```
+
+**Sample output:**
+
+```bash
+Timestamp | Container ID | Datanode ID | Container State |
BCSID | Message | Index Value
+----------------------------------------------------------------------------------------------------------------------
+2024-06-04 15:07:55,390 | 700 | 100 | QUASI_CLOSED |
353807 | No error | 0
+2024-06-04 14:50:18,177 | 700 | 150 | QUASI_CLOSED |
353807 | No error | 0
+2024-04-04 10:32:29,026 | 700 | 250 | OPEN | 0
| No error | 0
+2024-06-04 14:44:28,126 | 700 | 250 | CLOSING |
353807 | No error | 0
+2024-06-04 14:47:59,893 | 700 | 250 | QUASI_CLOSED |
353807 | Ratis group removed | 0
+2024-06-04 14:50:17,038 | 700 | 250 | QUASI_CLOSED |
353807 | No error | 0
+2024-06-04 14:50:18,184 | 700 | 250 | QUASI_CLOSED |
353807 | No error | 0
+2024-04-04 10:32:29,026 | 700 | 400 | OPEN | 0
| No error | 0
+Container 700 might be QUASI_CLOSED_STUCK.
+```
+
+#### **To List Containers Based on Health State**
+
+This Ozone debug CLI command lists all the containers which are either
UNDER-REPLICATED, OVER-REPLICATED, UNHEALTHY, or QUASI_CLOSED stuck.
+
+This command displays the **Container ID** along with the count of replicas in
the specified health state for each listed container.
+
+**The command options used are:**
+
+- List containers by health type: this by default provides only 100 rows in
the result
+
+```bash
+ozone debug log container --db=<path to db> list --health=<type>
+```
+
+- List containers by health type with a specified row limit:
+
+```bash
+ozone debug log container --db=<path to db> list --health=<type>
--length=<limit>
+```
+
+- List all containers by health type, overriding the row limit:
+
+```bash
+ozone debug log container --db=<path to db> list --health=<type> --all
+```
+
+**Sample output:**
+
+```bash
+ozone debug log container --db=path/to/db list --health=UNHEALTHY --all
+```
+
+```bash
+Container ID = 6002 - Count = 1
+Container ID = 6201 - Count = 1
+.
+.
+.
+Container ID = 136662 - Count = 3
+Container ID = 136837 - Count = 3
+Container ID = 199954 - Count = 3
+Container ID = 237747 - Count = 3
+Container ID = 2579099 - Count = 1
+Container ID = 2626888 - Count = 1
+Container ID = 2627711 - Count = 1
+Container ID = 2627751 - Count = 2
+Number of containers listed: 25085
+```
+
+#### **To List Containers Based on Final State**
+
+This Ozone debug CLI command helps to list replicas of all the containers
which have the state provided via the CLI command as their latest state.
+
+The command provides key details such as:
+
+- Datanode id
+- Container id
+- BCSID
+- TimeStamp - The most recent timestamp associated with the container replica
state
+- Index Value
+- Message (if any associated with that particular replica)
+
+**The command options used are:**
+
+- List containers by state: this by default provides only 100 rows in the
result
+
+```bash
+ozone debug log container --db=<path to db> list --lifecycle=<state>
+```
+
+- List containers by state with a specified row limit:
+
+```bash
+ozone debug log container --db=<path to db> list --lifecycle=<state>
--length=<limit>
+```
+
+- List all containers by state, overriding the row limit:
+
+```bash
+ozone debug log container --db=<path to db> list --lifecycle=<state> --all
+```
+
+**Sample output:**
+
+```bash
+ozone debug log container --db=path/to/db list --lifecycle=CLOSED --all
+```
+
+```bash
+Timestamp | Datanode ID | Container ID | BCSID | Message
| Index Value
+---------------------------------------------------------------------------------------------------
+2024-07-23 12:02:12,981 | 360 | 1 | 75654 | No error
| 0
+2024-07-23 11:56:21,106 | 365 | 1 | 75654 | Volume
failure | 0
+2024-07-23 11:56:21,106 | 365 | 1 | 75654 | Volume
failure | 0
+2024-08-29 14:11:32,879 | 415 | 1 | 30 | No error
| 0
+2024-08-29 14:11:17,533 | 430 | 1 | 30 | No error
| 0
+2024-06-20 11:50:09,496 | 460 | 1 | 75654 | No error
| 0
+2024-07-23 12:02:11,633 | 500 | 1 | 75654 | No error
| 0
+2024-06-20 12:03:24,230 | 505 | 2 | 83751 | No error
| 0
+2024-07-10 04:00:33,131 | 540 | 2 | 83751 | No error
| 0
+2024-07-10 04:00:46,825 | 595 | 2 | 83751 | No error
| 0
+```
+
+## NOTE
+
+- This tool assumes that all dn-container log files are already extracted from
the Datanodes and placed into a directory.
+- For the parsing command, if the DB path is not provided, a new database will
be created in the current working directory with the default filename
`container_datanode.db`.
+- For all other commands, if the DB path is not provided, it will look for a
default database (`container_datanode.db`) file in the current directory. If it
doesn’t exist, it will throw an error asking to provide a valid path.
diff --git
a/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/README.mdx
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/README.mdx
new file mode 100644
index 000000000..54761fc0a
--- /dev/null
+++
b/docs/05-administrator-guide/03-operations/11-tools/03-ozone-debug/README.mdx
@@ -0,0 +1,26 @@
+---
+sidebar_label: Ozone Debug
+---
+
+import DocCardList from "@theme/DocCardList";
+
+# Debug Tools
+
+Ozone Debug command (`ozone debug`) is a collection of developer tools
intended to help in debugging and get more information of various components of
ozone.
+
+It includes the following tools:
+
+- **ldb** - Tools to debug RocksDB related issues.
+- **om** - Debug commands related to OM.
+- **datanode** - Debug commands related to Datanode.
+- **replicas** - Debug commands for key replica related issues.
+- **ratis** - Debug commands related to Ratis.
+- **auditparser** - A tool to parse and query Ozone audit logs.
+- **log** - A tool to parse and provide insights on logs, currently supports
only the datanode’s container logs.
+- **checknative** - Checks if native libraries are loaded
+- **version** - Show internal version of Ozone components
+
+For more information see the following subpages:
+
+<!-- prettier-ignore -->
+<DocCardList/>
diff --git a/docs/05-administrator-guide/03-operations/11-tools/README.mdx
b/docs/05-administrator-guide/03-operations/11-tools/README.mdx
new file mode 100644
index 000000000..a4562f274
--- /dev/null
+++ b/docs/05-administrator-guide/03-operations/11-tools/README.mdx
@@ -0,0 +1,55 @@
+---
+sidebar_label: Tools
+---
+
+import DocCardList from "@theme/DocCardList";
+
+# Tools
+
+Ozone provides a set of command-line tools that can be used to manage and
operate an Ozone cluster.
+
+All commands are invoked via the `ozone` script.
+
+---
+
+## Daemon commands
+
+Commands used to start or stop Ozone services.
+
+- **`scm`** – Storage Container Manager service
+- **`om`** – Ozone Manager service
+- **`datanode`** – HDDS DataNode service
+- **`s3g`** – S3-compatible REST gateway
+- **`recon`** – Recon Web UI service
+- **`httpfs`** – HttpFS gateway
+
+---
+
+## Client commands
+
+- **`sh`** – Manage volumes, buckets, and keys
+- **`fs`** – Ozone filesystem commands (similar to `hdfs dfs`)
+- **`version`** – Print Ozone and HDDS version information
+
+---
+
+## Admin commands
+
+- **`admin`** – Admin and developer commands for Ozone components
+- **`insight`** – Display filtered logs, metrics, or configs for debugging
+- **`classpath`** – Print Hadoop classpath
+- **`dtutil`** – Delegation token operations
+- **`envvars`** – Display computed Hadoop environment variables
+- **`getconf`** – Read Ozone configuration values
+- **`genconf`** – Generate minimal `ozone-site.xml`
+
+---
+
+## Test tools
+
+- **`freon`** – Ozone load generator
+
+---
+
+<!-- prettier-ignore -->
+<DocCardList/>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
