This is an automated email from the ASF dual-hosted git repository.
yao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new e9a6230ae09c [SPARK-44239][SQL][FOLLOWUP] Do not disable vector memory
optimization when hugeVectorThreshold=0 to align its document
e9a6230ae09c is described below
commit e9a6230ae09c05697c0f750f78170f62353e50e9
Author: wankunde <[email protected]>
AuthorDate: Fri Sep 6 14:56:02 2024 +0800
[SPARK-44239][SQL][FOLLOWUP] Do not disable vector memory optimization when
hugeVectorThreshold=0 to align its document
### What changes were proposed in this pull request?
Minor fix: Update default hugeVectorThreshold from 0 to -1 in code to match
the documentation.
Keep consistent with the documentation
### Why are the changes needed?
Keep code and documentation consistent
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Exists UT
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #47988 from wankunde/minor_fix.
Authored-by: wankunde <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
---
.../org/apache/spark/sql/execution/vectorized/WritableColumnVector.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java
b/sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java
index 8b2fc7f5db31..10594d6c5d34 100644
---
a/sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java
+++
b/sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/WritableColumnVector.java
@@ -72,7 +72,7 @@ public abstract class WritableColumnVector extends
ColumnVector {
numNulls = 0;
}
- if (hugeVectorThreshold > 0 && capacity > hugeVectorThreshold) {
+ if (hugeVectorThreshold > -1 && capacity > hugeVectorThreshold) {
capacity = defaultCapacity;
releaseMemory();
reserveInternal(capacity);
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
