This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/ozone-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 8753ca28 [auto] Generated docs from Apache Ozone master
7cf9f655b090d70f5ad680d34955f2ed8eb50c02
8753ca28 is described below
commit 8753ca28fdccd1e2edd1f87e52586cba189e762a
Author: Github Actions <[email protected]>
AuthorDate: Sat Sep 20 12:46:13 2025 +0000
[auto] Generated docs from Apache Ozone master
7cf9f655b090d70f5ad680d34955f2ed8eb50c02
---
docs/edge/en/sitemap.xml | 4 ++--
docs/edge/feature/topology.html | 20 +++++++++++++++++++-
docs/edge/sitemap.xml | 2 +-
docs/edge/start/fromsource.html | 31 ++++++++++++++++++++++++++++++-
4 files changed, 52 insertions(+), 5 deletions(-)
diff --git a/docs/edge/en/sitemap.xml b/docs/edge/en/sitemap.xml
index 1f20ea72..8ab69a34 100644
--- a/docs/edge/en/sitemap.xml
+++ b/docs/edge/en/sitemap.xml
@@ -207,7 +207,7 @@
/>
</url><url>
<loc>/feature/topology.html</loc>
- <lastmod>2025-09-02T11:42:12-07:00</lastmod>
+ <lastmod>2025-09-19T13:18:09-07:00</lastmod>
<xhtml:link
rel="alternate"
hreflang="zh"
@@ -695,7 +695,7 @@
/>
</url><url>
<loc>/start/fromsource.html</loc>
- <lastmod>2025-04-14T11:15:15-07:00</lastmod>
+ <lastmod>2025-09-19T10:28:49-07:00</lastmod>
<xhtml:link
rel="alternate"
hreflang="zh"
diff --git a/docs/edge/feature/topology.html b/docs/edge/feature/topology.html
index 972b100f..5d1a7a3d 100644
--- a/docs/edge/feature/topology.html
+++ b/docs/edge/feature/topology.html
@@ -731,6 +731,21 @@ Example <code>determine_rack.sh</code>:
</li>
</ul>
<p>Note: When configuring these values, include the full class name prefix:
for example,
org.apache.hadoop.hdds.scm.container.placement.algorithms.SCMContainerPlacementCapacity</p>
+<h2 id="container-placement-for-erasure-coded-ec-containers">Container
Placement for Erasure Coded (EC) Containers</h2>
+<p>For Erasure Coded (EC) containers, SCM employs a specialized placement
policy to ensure data resilience and availability by distributing data and
parity blocks across multiple racks. This is configured using the
<code>ozone.scm.container.placement.ec.impl.key</code> property in
<code>ozone-site.xml</code>.</p>
+<h3 id="1-scmcontainerplacementrackscatter-default">1.
<code>SCMContainerPlacementRackScatter</code> (Default)</h3>
+<ul>
+<li><strong>Function:</strong> This is the default policy for EC containers.
It attempts to place each block (both data and parity) of an EC container on a
different rack. For example, for an RS-6-3-1024k container (6 data blocks + 3
parity blocks), this policy will try to place the 9 blocks on 9 different
racks. This “scatter” approach maximizes the fault tolerance, as
the loss of a single rack will not impact more than one block of the container.
[5]</li>
+<li><strong>Use Cases:</strong> This policy is highly recommended for
production clusters using Erasure Coding to protect against rack-level
failures.</li>
+<li><strong>Configuration:</strong>
+<div class="highlight"><pre tabindex="0"
style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code
class="language-xml" data-lang="xml"><span style="display:flex;"><span><span
style="color:#f92672"><property></span>
+</span></span><span style="display:flex;"><span> <span
style="color:#f92672"><name></span>ozone.scm.container.placement.ec.impl.key<span
style="color:#f92672"></name></span>
+</span></span><span style="display:flex;"><span> <span
style="color:#f92672"><value></span>org.apache.hadoop.hdds.scm.container.placement.algorithms.SCMContainerPlacementRackScatter<span
style="color:#f92672"></value></span>
+</span></span><span style="display:flex;"><span><span
style="color:#f92672"></property></span>
+</span></span></code></pre></div></li>
+<li><strong>Behavior:</strong> If the number of available racks is less than
the number of blocks in the EC group, the policy will start placing more than
one block on the same rack, while trying to keep the distribution as even as
possible.</li>
+<li><strong>Limitations:</strong> Similar to
<code>SCMContainerPlacementRackAware</code>, this policy is designed for
single-layer rack topologies (e.g., <code>/rack/node</code>) and is not
recommended for multi-layer hierarchies.</li>
+</ul>
<h2 id="optimizing-read-paths">Optimizing Read Paths</h2>
<p>Enable by setting <code>ozone.network.topology.aware.read</code> to
<code>true</code> in <code>ozone-site.xml</code>. [1]</p>
<div class="highlight"><pre tabindex="0"
style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code
class="language-xml" data-lang="xml"><span style="display:flex;"><span><span
style="color:#f92672"><property></span>
@@ -743,6 +758,8 @@ Example <code>determine_rack.sh</code>:
<li><strong>Accurate Topology:</strong> Maintain an accurate, up-to-date
topology map (static or dynamic script); this is foundational.</li>
<li><strong>Pipeline Creation:</strong> For production environments, use the
default <code>PipelinePlacementPolicy</code> for
<code>ozone.scm.pipeline.placement.impl</code> to ensure both rack fault
tolerance and pipeline load balancing.</li>
<li><strong>Pipeline Selection:</strong> The default
<code>RandomPipelineChoosePolicy</code> for
<code>hdds.scm.pipeline.choose.policy.impl</code> is suitable for general load
balancing.</li>
+<li><strong>Replicated (RATIS) Containers:</strong> For production, use
<code>SCMContainerPlacementRackAware</code> (mindful of its single-layer
topology limitation) or <code>SCMContainerPlacementCapacity</code> (balanced
disk usage) over <code>SCMContainerPlacementRandom</code>.</li>
+<li><strong>Erasure Coded (EC) Containers:</strong> For production rack fault
tolerance, use <code>SCMContainerPlacementRackScatter</code>.</li>
<li><strong>Read Operations:</strong> Enable
<code>ozone.network.topology.aware.read</code> with accurate topology.</li>
<li><strong>Monitor & Validate:</strong> Regularly monitor placement and
balance; use tools like Recon to verify topology awareness.</li>
</ul>
@@ -752,6 +769,7 @@ Example <code>determine_rack.sh</code>:
<li><a
href="https://github.com/apache/ozone/tree/master/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms";>Ozone
Source Code: container placement policies</a>. (For implementations of
pluggable placement policies).</li>
<li><a
href="https://github.com/apache/ozone/blob/master/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/SCMContainerPlacementRandom.java";>Ozone
Source Code: SCMContainerPlacementRandom.java</a>.</li>
<li><a
href="https://github.com/apache/ozone/blob/master/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/SCMContainerPlacementCapacity.java";>Ozone
Source Code: SCMContainerPlacementCapacity.java</a>.</li>
+<li><a
href="https://github.com/apache/ozone/blob/master/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/SCMContainerPlacementRackScatter.java";>Ozone
Source Code: SCMContainerPlacementRackScatter.java</a>.</li>
</ol>
@@ -772,7 +790,7 @@ Example <code>determine_rack.sh</code>:
<footer class="footer">
<div class="container">
<span class="small text-muted">
- Version: 2.1.0-SNAPSHOT, Last Modified: September 2, 2025 <a
class="hide-child link primary-color"
href="https://github.com/apache/ozone/commit/3948ca052d32071da8f10765fefdc39824d94342";>3948ca052d</a>
+ Version: 2.1.0-SNAPSHOT, Last Modified: September 19, 2025 <a
class="hide-child link primary-color"
href="https://github.com/apache/ozone/commit/7cf9f655b090d70f5ad680d34955f2ed8eb50c02";>7cf9f655b0</a>
</span>
</div>
</footer>
diff --git a/docs/edge/sitemap.xml b/docs/edge/sitemap.xml
index d25fd267..adc4f9ed 100644
--- a/docs/edge/sitemap.xml
+++ b/docs/edge/sitemap.xml
@@ -4,7 +4,7 @@
<sitemap>
<loc>/en/sitemap.xml</loc>
- <lastmod>2025-09-16T14:47:35-07:00</lastmod>
+ <lastmod>2025-09-19T13:18:09-07:00</lastmod>
</sitemap>
diff --git a/docs/edge/start/fromsource.html b/docs/edge/start/fromsource.html
index 0b875d85..700775ab 100644
--- a/docs/edge/start/fromsource.html
+++ b/docs/edge/start/fromsource.html
@@ -679,6 +679,35 @@ from the <a href="https://github.com/apache/ozone";>git
repository</a>).</p>
</span></span><span style="display:flex;"><span>mvn install:install-file
-DgroupId<span style="color:#f92672">=</span>com.google.protobuf
-DartifactId<span style="color:#f92672">=</span>protoc -Dversion<span
style="color:#f92672">=</span><span
style="color:#e6db74">${</span>PROTOBUF_VERSION<span
style="color:#e6db74">}</span> -Dclassifier<span
style="color:#f92672">=</span>osx-aarch_64 -Dpackaging<span
style="color:#f92672">=</span>exe -Dfile<span
style="color:#f92672">=</span>src/protoc
</span></span><span style="display:flex;"><span><span style="color:#75715e">#
workaround for Maven 3.9.x. Not needed for 3.8.x or earlier</span>
</span></span><span style="display:flex;"><span>cp
$HOME/.m2/repository/com/google/protobuf/protoc/<span
style="color:#e6db74">${</span>PROTOBUF_VERSION<span
style="color:#e6db74">}</span>/protoc-<span
style="color:#e6db74">${</span>PROTOBUF_VERSION<span
style="color:#e6db74">}</span>-osx-aarch_64
$HOME/.m2/repository/com/google/protobuf/protoc/<span
style="color:#e6db74">${</span>PROTOBUF_VERSION<span
style="color:#e6db74">}</span>/protoc-<span
style="color:#e6db74">${</span>PROTOBUF_VE [...]
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>cd ..
+</span></span><span style="display:flex;"><span><span style="color:#75715e">#
Patch protobuf 3.19.6</span>
+</span></span><span style="display:flex;"><span>PROTOBUF_VERSION<span
style="color:#f92672">=</span><span
style="color:#e6db74">"3.19.6"</span>
+</span></span><span style="display:flex;"><span>curl -LO
https://github.com/protocolbuffers/protobuf/releases/download/v<span
style="color:#e6db74">${</span>PROTOBUF_VERSION<span
style="color:#e6db74">}</span>/protobuf-all-<span
style="color:#e6db74">${</span>PROTOBUF_VERSION<span
style="color:#e6db74">}</span>.tar.gz
+</span></span><span style="display:flex;"><span>tar xzf protobuf-all-<span
style="color:#e6db74">${</span>PROTOBUF_VERSION<span
style="color:#e6db74">}</span>.tar.gz
+</span></span><span style="display:flex;"><span>cd protobuf-<span
style="color:#e6db74">${</span>PROTOBUF_VERSION<span
style="color:#e6db74">}</span>
+</span></span><span style="display:flex;"><span>./configure --disable-shared
+</span></span><span style="display:flex;"><span>make -j
+</span></span><span style="display:flex;"><span>mvn install:install-file
-DgroupId<span style="color:#f92672">=</span>com.google.protobuf
-DartifactId<span style="color:#f92672">=</span>protoc -Dversion<span
style="color:#f92672">=</span><span
style="color:#e6db74">${</span>PROTOBUF_VERSION<span
style="color:#e6db74">}</span> -Dclassifier<span
style="color:#f92672">=</span>osx-aarch_64 -Dpackaging<span
style="color:#f92672">=</span>exe -Dfile<span
style="color:#f92672">=</span>src/protoc
+</span></span><span style="display:flex;"><span><span style="color:#75715e">#
workaround for Maven 3.9.x. Not needed for 3.8.x or earlier</span>
+</span></span><span style="display:flex;"><span>cp
$HOME/.m2/repository/com/google/protobuf/protoc/<span
style="color:#e6db74">${</span>PROTOBUF_VERSION<span
style="color:#e6db74">}</span>/protoc-<span
style="color:#e6db74">${</span>PROTOBUF_VERSION<span
style="color:#e6db74">}</span>-osx-aarch_64
$HOME/.m2/repository/com/google/protobuf/protoc/<span
style="color:#e6db74">${</span>PROTOBUF_VERSION<span
style="color:#e6db74">}</span>/protoc-<span
style="color:#e6db74">${</span>PROTOBUF_VE [...]
+</span></span><span style="display:flex;"><span>
+</span></span><span style="display:flex;"><span>cd ..
+</span></span><span style="display:flex;"><span><span style="color:#75715e">#
Compile grpc</span>
+</span></span><span style="display:flex;"><span><span style="color:#75715e">#
Pre-req: Manually install Gradle (version 7.0 or higher) and a compatible JVM
(JDK 8 or higher) as required by gRPC Java v1.71.0. See
https://github.com/grpc/grpc-java/blob/v1.71.0/README.md for details.</span>
+</span></span><span style="display:flex;"><span>git clone
https://github.com/grpc/grpc-java.git
+</span></span><span style="display:flex;"><span>cd grpc-java
+</span></span><span style="display:flex;"><span>git checkout v1.71.0
+</span></span><span style="display:flex;"><span>PARENTDIR<span
style="color:#f92672">=</span><span style="color:#66d9ef">$(</span>cd .. <span
style="color:#f92672">&&</span> pwd<span style="color:#66d9ef">)</span>
+</span></span><span style="display:flex;"><span>export PROTOBUF_ROOT<span
style="color:#f92672">=</span><span
style="color:#e6db74">"</span>$PARENTDIR<span
style="color:#e6db74">/protobuf-</span><span
style="color:#e6db74">${</span>PROTOBUF_VERSION<span
style="color:#e6db74">}</span><span style="color:#e6db74">"</span>
+</span></span><span style="display:flex;"><span>export PATH<span
style="color:#f92672">=</span><span style="color:#e6db74">"</span><span
style="color:#e6db74">${</span>PROTOBUF_ROOT<span
style="color:#e6db74">}</span><span
style="color:#e6db74">/src:</span>$PATH<span style="color:#e6db74">"</span>
+</span></span><span style="display:flex;"><span>export CPPFLAGS<span
style="color:#f92672">=</span><span style="color:#e6db74">"-I</span><span
style="color:#e6db74">${</span>PROTOBUF_ROOT<span
style="color:#e6db74">}</span><span style="color:#e6db74">/src"</span>
+</span></span><span style="display:flex;"><span>export LDFLAGS<span
style="color:#f92672">=</span><span style="color:#e6db74">"-L</span><span
style="color:#e6db74">${</span>PROTOBUF_ROOT<span
style="color:#e6db74">}</span><span style="color:#e6db74">/src/.libs"</span>
+</span></span><span style="display:flex;"><span>./gradlew
:grpc-compiler:java_pluginExecutable -PskipAndroid<span
style="color:#f92672">=</span>true
+</span></span><span style="display:flex;"><span>PLUGIN<span
style="color:#f92672">=</span><span
style="color:#e6db74">"protoc-gen-grpc-java-1.71.0-osx-aarch_64.exe"</span>
+</span></span><span style="display:flex;"><span>cp
compiler/build/exe/java_plugin/protoc-gen-grpc-java $PLUGIN
+</span></span><span style="display:flex;"><span>mvn install:install-file
-DgroupId<span style="color:#f92672">=</span>io.grpc -DartifactId<span
style="color:#f92672">=</span>protoc-gen-grpc-java -Dversion<span
style="color:#f92672">=</span>1.71.0 -Dclassifier<span
style="color:#f92672">=</span>osx-aarch_64 -Dpackaging<span
style="color:#f92672">=</span>exe -Dfile<span
style="color:#f92672">=</span>$PLUGIN
+</span></span><span style="display:flex;"><span>cd ..
</span></span></code></pre></div><h2 id="build-ozone">Build Ozone</h2>
<p>Run the ozone build command.</p>
<div class="highlight"><pre tabindex="0"
style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code
class="language-bash" data-lang="bash"><span style="display:flex;"><span>mvn
clean package -DskipTests<span style="color:#f92672">=</span>true
@@ -719,7 +748,7 @@ provided along with the official release.</p>
<footer class="footer">
<div class="container">
<span class="small text-muted">
- Version: 2.1.0-SNAPSHOT, Last Modified: April 14, 2025 <a
class="hide-child link primary-color"
href="https://github.com/apache/ozone/commit/9e4da5bac25a356ff3704cc7d5f471757fbf08b2";>9e4da5bac2</a>
+ Version: 2.1.0-SNAPSHOT, Last Modified: September 19, 2025 <a
class="hide-child link primary-color"
href="https://github.com/apache/ozone/commit/abb47056d81165a65960570f5ba0c3ea88a1acfd";>abb47056d8</a>
</span>
</div>
</footer>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
