This is an automated email from the ASF dual-hosted git repository.
peter-toth pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 24eb02c54691 [SPARK-56888][PYTHON][DOCS] Simplify
`get_caller_source_code_location` docstring
24eb02c54691 is described below
commit 24eb02c546912ae0e1c7b9c011dcbae597ad5294
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Sat May 16 09:22:09 2026 +0200
[SPARK-56888][PYTHON][DOCS] Simplify `get_caller_source_code_location`
docstring
### What changes were proposed in this pull request?
Simplify the docstring of `get_caller_source_code_location` in
`python/pyspark/pipelines/source_code_location.py` by removing the
description of
Python < 3.10 decorator line-number behavior.
### Why are the changes needed?
Apache Spark 4.1.0 dropped Python 3.9 support.
- [SPARK-54262 Drop Python 3.9
Support](https://issues.apache.org/jira/browse/SPARK-54262)
So, PySpark 4.1+ requires Python >= 3.10, so the pre-3.10 branch in the
docstring is dead
documentation.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
N/A. Docstring-only change.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 4.7)
Closes #55913 from dongjoon-hyun/SPARK-56888.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Peter Toth <[email protected]>
(cherry picked from commit 9ee53c7b41ac66e64e32963549b064b2f3f78382)
Signed-off-by: Peter Toth <[email protected]>
---
python/pyspark/pipelines/source_code_location.py | 32 +++---------------------
1 file changed, 4 insertions(+), 28 deletions(-)
diff --git a/python/pyspark/pipelines/source_code_location.py
b/python/pyspark/pipelines/source_code_location.py
index cbf4cbe514a6..e4f0e99b1dd4 100644
--- a/python/pyspark/pipelines/source_code_location.py
+++ b/python/pyspark/pipelines/source_code_location.py
@@ -28,35 +28,11 @@ class SourceCodeLocation:
def get_caller_source_code_location(stacklevel: int) -> SourceCodeLocation:
"""
- Returns a SourceCodeLocation object representing the location code that
invokes this function.
+ Returns a SourceCodeLocation object representing the location of the code
that invokes
+ this function.
- If this function is called from a decorator (ex. @sdp.table), note that
the returned line
- number is affected by how the decorator was triggered - i.e. whether
@sdp.table or @sdp.table()
- was called - AND what python version is being used
-
- Case 1:
- |@sdp.table()
- |def fn
-
- @sdp.table() is executed immediately, on line 1. This is true for all
python versions.
-
- Case 2:
- |@sdp.table
- |def fn
-
- In python < 3.10, @sdp.table will expand to fn = sdp.table(fn), replacing
the line that `fn` is
- defined on. This would be line 2. More interestingly, this means:
-
- |@sdp.table
- |
- |
- |def fn
-
- Will expand to fn = sdp.table(fn) on line 4, where `fn` is defined.
-
- However, in python 3.10+, the line number in the stack trace will still be
the line that the
- decorator was defined on. In other words, case 2 will be treated the same
as case 1, and the
- line number will be 1.
+ When called from a decorator (e.g. ``@sdp.table`` or ``@sdp.table()``),
the returned
+ line number is the line on which the decorator was applied.
:param stacklevel: The number of stack frames to go up. 0 means the direct
caller of this
function, 1 means the caller of the caller, and so on.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
