This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new ac9b01ecff6f [SPARK-55020][PYTHON][FOLLOW-UP] Move release into
disable gc protection to prevent deadlock
ac9b01ecff6f is described below
commit ac9b01ecff6f26d6f06e97afa059cee0d075ebab
Author: Tian Gao <[email protected]>
AuthorDate: Fri Feb 13 06:40:37 2026 +0900
[SPARK-55020][PYTHON][FOLLOW-UP] Move release into disable gc protection to
prevent deadlock
### What changes were proposed in this pull request?
Move release process into `disable_gc()` in
`ExecutePlanResponseReattachableIterator`.
### Why are the changes needed?
`_release_all()` also has some non-reentriable code. For example,
submitting a task.
https://github.com/gaogaotiantian/spark/actions/runs/21920616983/job/63299737303
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Still need a few days of CI.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #54275 from gaogaotiantian/fix-deadlock-threadpool.
Authored-by: Tian Gao <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
python/pyspark/sql/connect/client/reattach.py | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/python/pyspark/sql/connect/client/reattach.py
b/python/pyspark/sql/connect/client/reattach.py
index dd16b80b82b6..62593384c7d0 100644
--- a/python/pyspark/sql/connect/client/reattach.py
+++ b/python/pyspark/sql/connect/client/reattach.py
@@ -148,15 +148,15 @@ class ExecutePlanResponseReattachableIterator(Generator):
if not self._has_next():
raise StopIteration()
- ret = self._current
- assert ret is not None
+ ret = self._current
+ assert ret is not None
- self._last_returned_response_id = ret.response_id
- if ret.HasField("result_complete"):
- self._release_all()
- else:
- self._release_until(self._last_returned_response_id)
- self._current = None
+ self._last_returned_response_id = ret.response_id
+ if ret.HasField("result_complete"):
+ self._release_all()
+ else:
+ self._release_until(self._last_returned_response_id)
+ self._current = None
return ret
def _has_next(self) -> bool:
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
