AndrewJSchofield commented on code in PR #20029:
URL: https://github.com/apache/kafka/pull/20029#discussion_r2167632428
##########
docs/streams/developer-guide/config-streams.html:
##########
@@ -775,6 +776,20 @@ <h4><a class="toc-backref"
href="#id46">ensure.explicit.internal.resource.naming
</div>
</blockquote>
</div>
+ <div class="section" id="group-protocol">
+ <h4><a class="toc-backref" href="#id47">group.protocol</a><a
class="headerlink" href="#group-protocol" title="Permalink to this
headline"></a></h4>
+ <blockquote>
+ <div>
+ <p>
+ The group protocol used by the Kafka Streams client used for
coordination.
+ It determines how the client will communicate with the Kafka
brokers and other clients in the same group.
+ The default value is <code class="docutils literal"><span
class="pre">"classic"</span></code>, which is the classic consumer group
protocol.
+ Can be set to <code class="docutils literal"><span
class="pre">"streams"</span></code> (requires broker side enablement) to enable
the new Kafka Streams group protocol.
Review Comment:
nit: "broker-side" please
##########
docs/streams/upgrade-guide.html:
##########
@@ -141,6 +141,91 @@ <h3 class="anchor-heading"><a id="streams_notable_changes"
class="anchor-link"><
<h3><a id="streams_api_changes_410"
href="#streams_api_changes_410">Streams API changes in 4.1.0</a></h3>
+ <h4>Early Access of the Streams Rebalance Protocol</h4>
+
+ <p>
+ The Streams Rebalance Protocol is a broker-driven rebalancing system
designed specifically for Kafka
+ Streams applications. Following the pattern of KIP-848, which moved
rebalance coordination of plain consumers
+ from clients to brokers, KIP-1071 extends this model to Kafka Streams
workloads. Instead of clients
+ computing new assignments on the client during rebalance events
involving all members of the group, assignments are
+ computed continuously on the broker. Instead of using a consumer
group, the streams application registers as a
+ streams group with the broker, which manages and exposes all metadata
required for coordination of the
+ streams application instances.
+ </p>
+
+ <p>
+ This Early Access release covers a subset of the functionality
detailed in
+ <a
href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1071%3A+Streams+Rebalance+Protocol";>KIP-1071</a>.
+ Do not use the new protocol in production. The API is subject to
change in future
+ releases.
+ </p>
+
+ <p><strong>What's Included in Early Access</strong></p>
+
+ <ul>
+ <li><strong>Core Streams Group Rebalance Protocol:</strong> The
<code>group.protocol=streams</code> configuration
+ enables the dedicated streams rebalance protocol. This separates
streams groups from consumer groups and
+ provides a streams-specific group membership lifecycle and
metadata management on the broker.</li>
+ <li><strong>Sticky Task Assignor:</strong> A basic task assignment
strategy that minimizes task movement
+ during rebalances is included.</li>
+ <li><strong>Interactive Query Support:</strong> IQ operations are
compatible with the new streams protocol.</li>
+ <li><strong>New Admin RPC:</strong> The
<code>StreamsGroupDescribe</code> RPC provides streams-specific metadata
+ separate from consumer group information, with corresponding
access via the <code>Admin</code> client.</li>
+ <li><strong>CLI Integration:</strong> You can list, describe, and
delete streams groups via the <code>kafka-streams-groups.sh</code> script.</li>
+ </ul>
+
+ <p><strong>What's Not Included in Early Access</strong></p>
+
+ <ul>
+ <li><strong>Static Membership:</strong> Setting a client `instance.id`
will be rejected.</li>
+ <li><strong>Topology Updates:</strong> If a topology is changed
significantly (e.g., by adding new source topics
+ or changing the number of sub-topologies), a new streams group
must be created.</li>
+ <li><strong>High Availability Assignor:</strong> Only the sticky
assignor is supported.</li>
+ <li><strong>Regular Expressions:</strong> Pattern-based topic
subscription is not supported.</li>
+ <li><strong>Reset Operations:</strong> CLI offset reset operations are
not supported.</li>
+ <li><strong>Protocol Migration:</strong> Group migration is not
available between the classic and new streams protocols.</li>
+ </ul>
+
+ <p><strong>Why Use the Streams Rebalance Protocol?</strong></p>
+
+ <ul>
+ <li>
+ <strong>Broker-Driven Coordination:</strong>
+ Centralizes task assignment logic on brokers instead of the
client. This provides consistent,
+ authoritative task assignment decisions from a single coordination
point and reduces the potential for
+ split-brain scenarios.
+ </li>
+ <li>
+ <strong>Faster, More Stable Rebalances:</strong>
+ Reduces rebalance duration and impact by removing the global
synchronization point. This minimizes
+ application downtime during membership changes or failures.
+ </li>
+ <li>
+ <strong>Better Observability:</strong>
+ Provides dedicated metrics and admin interfaces that separate
streams from consumer groups, leading to
+ clearer troubleshooting with broker-side observability.
+ </li>
+ </ul>
+
+ <p>
+ To enable the protocol, set
<code>unstable.feature.versions.enable=true</code> for controllers and brokers,
and
+ set <code>unstable.api.versions.enable=true</code> on the brokers as
well. In your Kafka Streams application
+ configuration, set <code>group.protocol=streams</code>.
Review Comment:
This isn't quite enough I think. For a new cluster, setting these configs is
sufficient. For an existing cluster, if you set these configs, you would also
need to set `streams.version=1`. Maybe you should tell people to make a new
cluster for testing only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
