This is an automated email from the ASF dual-hosted git repository.
weichiu pushed a commit to branch HDDS-9225-website-v2
in repository https://gitbox.apache.org/repos/asf/ozone-site.git
The following commit(s) were added to refs/heads/HDDS-9225-website-v2 by this
push:
new daa55566c HDDS-14391. [Website v2] [Docs] [Core Concepts] Ozone S3
Gateway (#233)
daa55566c is described below
commit daa55566c359e6ee6333fa0818be1bb22d8eb198
Author: Eric C. Ho <[email protected]>
AuthorDate: Sat Jan 10 05:09:27 2026 +0800
HDDS-14391. [Website v2] [Docs] [Core Concepts] Ozone S3 Gateway (#233)
---
.../01-overview.md} | 8 +-
.../01-architecture/02-s3-gateway.md | 111 +++++++++++++++++++++
.../{ => 01-architecture}/FunctionalOzone.png | Bin
docs/03-core-concepts/01-architecture/README.mdx | 11 ++
.../{ => 01-architecture}/ozoneBlockDiagram.png | Bin
docs/03-core-concepts/02-replication/03-ratis.md | 2 +-
6 files changed, 128 insertions(+), 4 deletions(-)
diff --git a/docs/03-core-concepts/01-architecture.md
b/docs/03-core-concepts/01-architecture/01-overview.md
similarity index 90%
rename from docs/03-core-concepts/01-architecture.md
rename to docs/03-core-concepts/01-architecture/01-overview.md
index 4b3380844..ce40ccdb8 100644
--- a/docs/03-core-concepts/01-architecture.md
+++ b/docs/03-core-concepts/01-architecture/01-overview.md
@@ -1,4 +1,6 @@
-# Architecture
+---
+sidebar_label: Overview
+---
<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
@@ -16,7 +18,7 @@
limitations under the License.
-->
-## Overview
+# Overview
Ozone is a fault-tolerant, distributed object store optimized for Big Data
workloads. Its primary design goal is scalability, aiming to support billions
of objects.
@@ -55,7 +57,7 @@ Another way to understand Ozone is by looking at its
functional layers.
- The **Metadata Management Layer** is composed of the Ozone Manager and
Storage Container Manager.
- The **Data Storage Layer** consists of the Datanodes, which are managed by
SCM. Data is stored in logical blocks, which are grouped into containers for
replication.
-- The **Replication Layer** is powered by [Ratis](replication/ratis), a
Java-based implementation of the Raft consensus protocol. Ratis ensures that
metadata is reliably replicated across OM and SCM instances and maintains data
consistency during write operations on the Datanodes.
+- The **Replication Layer** is powered by
[Ratis](../02-replication/03-ratis.md), a Java-based implementation of the Raft
consensus protocol. Ratis ensures that metadata is reliably replicated across
OM and SCM instances and maintains data consistency during write operations on
the Datanodes.
- The **Protocol Bus** allows Ozone to be extended with different protocols.
Currently, it supports an S3-compatible protocol. This bus allows new object
store or file system protocols to be implemented on top of Ozone's native
protocol.
This layered architecture, combined with a clear separation of metadata and
data management, allows Ozone to scale to handle billions of objects while
maintaining high performance and fault tolerance.
diff --git a/docs/03-core-concepts/01-architecture/02-s3-gateway.md
b/docs/03-core-concepts/01-architecture/02-s3-gateway.md
new file mode 100644
index 000000000..46eca9729
--- /dev/null
+++ b/docs/03-core-concepts/01-architecture/02-s3-gateway.md
@@ -0,0 +1,111 @@
+---
+sidebar_label: Ozone S3 Gateway
+---
+
+# Ozone S3 Gateway
+
+Apache Ozone’s S3 Gateway (often called **S3G**) is a component that provides
an Amazon S3-compatible REST interface for Ozone’s object store.
+In essence, it allows any S3 client or tool to read and write data in Ozone as
if it were talking to AWS S3.
+The S3 Gateway runs as a **separate, stateless service** on top of Ozone,
translating HTTP S3 API calls into Ozone’s native storage operations.
+
+This design lets users leverage the rich ecosystem of S3-compatible
applications with Ozone, without modifying those applications.
+
+## High-Level Overview of the S3 Gateway
+
+Ozone’s S3 Gateway acts as a **REST front-end** that speaks the S3 protocol
and bridges it to the Ozone backend.
+It is **stateless**, meaning it does not persist data or metadata locally
between requests – all state is stored in the Ozone cluster itself.
+
+Being stateless allows multiple S3 Gateway instances to run in parallel behind
a load balancer for scaling and high availability.
+The idea of stateless also allows S3G to be deployed in Kubernetes quite
easily.
+The gateway supports a broad set of S3 operations – create/delete buckets,
PUT/GET objects, list objects, perform multipart uploads – using standard S3
SDKs or CLI tools.
+
+Under the hood, each operation is translated into the appropriate Ozone calls:
+
+- A PUT object request becomes a `putKey` write operation.
+- A GET object becomes a `getKey` read operation.
+
+### Mapping S3 to Ozone Concepts
+
+In Ozone, data is organized into **volumes**, **buckets**, and **keys**. By
default, the S3 Gateway uses a special
+volume named `/s3v` to store all S3 buckets (this is configurable via the
`ozone.s3g.volume.name` property).
+
+Each S3 bucket created is represented as a bucket under the `/s3v` volume in
Ozone. Objects (keys) are stored within those buckets. This mapping is
transparent to the end user.
+
+The default bucket layout for S3 buckets (`OBJECT_STORE`) can be configured
using the `ozone.s3g.default.bucket.layout` property. This property defines the
storage layout for buckets created via the S3 Gateway.
+
+## S3 Gateway in the Ozone Architecture
+
+The S3 Gateway does the following:
+
+- It receives S3 REST API calls from clients.
+- It translates them and forwards metadata operations to OM.
+- It streams data directly to and from Datanodes.
+
+This stateless gateway can be scaled horizontally.
+
+## Internal Components and Design
+
+### HTTP Server and REST API Layer
+
+Runs an embedded HTTP server exposing REST endpoints like:
+
+- `PUT /<bucket>/<key>`
+- `GET /<bucket>/<key>`
+
+Handles authentication headers, content length, and routing.
+
+### Request Handlers / Protocol Translation
+
+Each S3 operation is mapped to Ozone APIs:
+
+- Create Bucket → create in `/s3v` volume
+- List Buckets → query all user-owned buckets under `/s3v`
+- Put/Get Object → `putKey` / `getKey`
+
+It also assembles proper S3-compliant response formats.
+
+### Ozone Client Integration
+
+Uses Ozone client libraries to:
+
+- Call OM for metadata operations
+- Stream data to/from Datanodes
+
+Bulk data does not flow through OM – it goes directly between gateway and
Datanodes.
+
+### Statelessness and Caching
+
+The gateway:
+
+- Does not persist state
+- Uses lightweight in-memory caches
+- Depends on OM for coordination and user/token validation
+
+This simplifies scaling and failure recovery.
+
+### Authentication and Security
+
+Supports AWS Signature-style authentication (if enabled):
+
+- Only AWS Signature V4 is supported (the older AWS Signature V2 is not).
+- Validates access/secret keys via OM
+- Uses OM’s stored credentials
+- Relies on OM for final authorization
+
+Supports Kerberos authentication for secure clusters; In unsecured mode,
allows anonymous or dummy access.
+
+## Summary
+
+The Ozone S3 Gateway is a **protocol adapter**:
+
+- It translates REST S3 operations into native Ozone calls.
+- It doesn’t store or manage data – it delegates to OM and Datanodes.
+- Its stateless, scalable design enables multiple instances behind load
balancers.
+- It enables seamless use of existing S3-compatible clients and tools.
+
+For more details, refer to the other documentation:
+
+- [Ozone S3 Gateway
Interface](../../04-user-guide/01-client-interfaces/03-s3/01-s3-api.md)
+- [S3
Multi-Tenancy](../../05-administrator-guide/03-operations/07-s3-multi-tenancy.md)
+- [Securing
S3](../../04-user-guide/01-client-interfaces/03-s3/02-securing-s3.md)
+- [Network
Ports](../../05-administrator-guide/02-configuration/01-basic/02-network/07-default-ports.md)
diff --git a/docs/03-core-concepts/FunctionalOzone.png
b/docs/03-core-concepts/01-architecture/FunctionalOzone.png
similarity index 100%
rename from docs/03-core-concepts/FunctionalOzone.png
rename to docs/03-core-concepts/01-architecture/FunctionalOzone.png
diff --git a/docs/03-core-concepts/01-architecture/README.mdx
b/docs/03-core-concepts/01-architecture/README.mdx
new file mode 100644
index 000000000..a556727d3
--- /dev/null
+++ b/docs/03-core-concepts/01-architecture/README.mdx
@@ -0,0 +1,11 @@
+---
+sidebar_label: Architecture
+---
+
+# Architecture
+
+import DocCardList from '@theme/DocCardList';
+
+This section describes the architecture and components within Ozone.
+
+<DocCardList/>
diff --git a/docs/03-core-concepts/ozoneBlockDiagram.png
b/docs/03-core-concepts/01-architecture/ozoneBlockDiagram.png
similarity index 100%
rename from docs/03-core-concepts/ozoneBlockDiagram.png
rename to docs/03-core-concepts/01-architecture/ozoneBlockDiagram.png
diff --git a/docs/03-core-concepts/02-replication/03-ratis.md
b/docs/03-core-concepts/02-replication/03-ratis.md
index 547389207..ef4301580 100644
--- a/docs/03-core-concepts/02-replication/03-ratis.md
+++ b/docs/03-core-concepts/02-replication/03-ratis.md
@@ -13,7 +13,7 @@ and integration.
Ozone leverages Ratis to replicate system states across multiple nodes,
ensuring high availability and
redundancy. When using Ratis for replication, each piece of data written by
clients is replicated to 3 Datanodes.
Within Ozone, Ratis is employed in critical components such as the [Ozone
Manager](../03-namespace/01-overview.md),
-[Storage Container Manager](../01-architecture.md), and Datanodes. It forms
the central pillar for the
+[Storage Container Manager](../01-architecture/01-overview.md), and Datanodes.
It forms the central pillar for the
High Availability (HA) mechanisms of both the Ozone Manager (OM-HA) and
Storage Container Manager (SCM-HA).
For more detailed information, please visit the [Apache Ratis
website](https://ratis.apache.org/).
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
