This is an automated email from the ASF dual-hosted git repository.
ruifengz pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
from b005f5692842 [SPARK-53689][BUILD][FOLLOW-UP] Check if RELEASE_VERSION
is already set properly
add 4418d6ec567e [SPARK-53562][PYTHON] Limit Arrow batch sizes in
`applyInArrow` and `applyInPandas`
No new revisions were added by this update.
Summary of changes:
python/pyspark/sql/pandas/serializers.py | 87 ++++++++++++++
.../sql/tests/arrow/test_arrow_grouped_map.py | 40 ++++++-
.../connect/arrow/test_parity_arrow_grouped_map.py | 4 +-
.../pandas/test_parity_pandas_grouped_map.py | 4 +-
.../sql/tests/pandas/test_pandas_grouped_map.py | 50 ++++++--
python/pyspark/worker.py | 13 +-
.../sql/execution/arrow/ArrowWriterWrapper.scala | 84 +++++++++++++
.../python/FlatMapGroupsInBatchExec.scala | 2 +-
.../sql/execution/python/PythonArrowInput.scala | 132 ++++++++++++++++++---
9 files changed, 379 insertions(+), 37 deletions(-)
create mode 100644
sql/catalyst/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriterWrapper.scala
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
