This is an automated email from the ASF dual-hosted git repository.
ashrigondekar pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
from 412ce481a667 [SPARK-54646][BUILD] Upgrade `gcs-connector` to 2.2.29
add dc9b1d53b241 [SPARK-54443][SS] Partition key extraction for all
streaming stateful operators
No new revisions were added by this update.
Summary of changes:
.../src/main/resources/error/error-conditions.json | 11 ++
.../FlatMapGroupsInPandasWithStateExec.scala | 3 +-
.../StatePartitionKeyExtractorFactory.scala | 84 ++++++++++++
.../StreamingAggregationStateManager.scala | 8 +-
.../StreamingSessionWindowStateManager.scala | 9 +-
.../FlatMapGroupsWithStateExec.scala | 4 +-
.../FlatMapGroupsWithStateExecHelper.scala | 9 +-
.../join/SymmetricHashJoinStateManager.scala | 48 +++++--
.../operators/stateful/statefulOperators.scala | 31 ++++-
.../StateStoreColumnFamilySchemaUtils.scala | 41 +++++-
.../StateTypesEncoderUtils.scala | 127 +++++++++++++++++-
.../TransformWithStateVariableUtils.scala | 8 +-
.../transformwithstate/timers/TimerStateImpl.scala | 10 ++
.../state/OfflineStateRepartitionErrors.scala | 36 +++++
.../streaming/state/RocksDBStateEncoder.scala | 2 +-
.../state/StatePartitionKeyExtractor.scala | 105 +++++++++++++++
.../execution/streaming/state/ListStateSuite.scala | 26 ++++
.../execution/streaming/state/MapStateSuite.scala | 27 ++++
.../StreamingAggregationStateManagerSuite.scala | 32 ++++-
.../StreamingSessionWindowStateManagerSuite.scala | 42 +++++-
.../state/SymmetricHashJoinStateManagerSuite.scala | 95 +++++++++++++-
.../sql/execution/streaming/state/TimerSuite.scala | 21 +++
.../streaming/state/ValueStateSuite.scala | 107 ++++++++++++++-
.../streaming/FlatMapGroupsWithStateSuite.scala | 145 ++++++++++++++++++++-
.../streaming/StreamingDeduplicationSuite.scala | 111 +++++++++++++++-
...treamingDeduplicationWithinWatermarkSuite.scala | 25 +++-
26 files changed, 1128 insertions(+), 39 deletions(-)
create mode 100644
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/operators/stateful/StatePartitionKeyExtractorFactory.scala
create mode 100644
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StatePartitionKeyExtractor.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
