This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new bbf31e7 [SPARK-53985] Suppress `StatusRecorder` warning messages
bbf31e7 is described below
commit bbf31e72a9584b58b0496fbfd9da54d7b3208b1d
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Oct 22 11:06:26 2025 -0700
[SPARK-53985] Suppress `StatusRecorder` warning messages
### What changes were proposed in this pull request?
This PR aims to suppress `StatusRecorder` warning messages.
### Why are the changes needed?
Currently, `StatusRecorder` shows `WARN` messages too verbosely at most
`spark.kubernetes.operator.api.statusPatchMaxAttempts` times. Then, it will
show `ERROR` message at the end.
```
25/10/22 17:17:32 WARN test-00672 default o.a.s.k.o.u.StatusRecorder Error
while patching status, retrying 2/3...
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing:
PUT at:
https://xxx:443/apis/spark.apache.org/v1/namespaces/default/sparkapplications/test-00672/status.
Message: Operation cannot be fulfilled on sparkapplications.spark.apache.org
"test-00672": the object has been modified; please apply your changes to the
latest version and try again. Received status: Status(apiVersion=v1, code=409,
details=StatusDetails(causes=[], group=spark.apache.org, kind=sparkapplicati
[...]
at
io.fabric8.kubernetes.client.KubernetesClientException.copyAsCause(KubernetesClientException.java:205)
at
io.fabric8.kubernetes.client.dsl.internal.OperationSupport.waitForResult(OperationSupport.java:507)
at
io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleResponse(OperationSupport.java:524)
at
io.fabric8.kubernetes.client.dsl.internal.OperationSupport.handleUpdate(OperationSupport.java:358)
at
io.fabric8.kubernetes.client.dsl.internal.BaseOperation.handleUpdate(BaseOperation.java:759)
at
io.fabric8.kubernetes.client.dsl.internal.HasMetadataOperation.update(HasMetadataOperation.java:138)
at
io.fabric8.kubernetes.client.dsl.internal.HasMetadataOperation.update(HasMetadataOperation.java:121)
at
io.fabric8.kubernetes.client.dsl.internal.HasMetadataOperation.updateStatus(HasMetadataOperation.java:126)
at
io.fabric8.kubernetes.client.dsl.internal.HasMetadataOperation.updateStatus(HasMetadataOperation.java:44)
at
org.apache.spark.k8s.operator.utils.StatusRecorder.patchAndStatusWithVersionLocked(StatusRecorder.java:101)
```
We can switch `WARN` to `DEBUG` during retrying
`spark.kubernetes.operator.api.statusPatchMaxAttempts` times because it's
recoverable.
### Does this PR introduce _any_ user-facing change?
No. This is a log message change.
### How was this patch tested?
Manual review.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #397 from dongjoon-hyun/SPARK-53985.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../main/java/org/apache/spark/k8s/operator/utils/StatusRecorder.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/spark-operator/src/main/java/org/apache/spark/k8s/operator/utils/StatusRecorder.java
b/spark-operator/src/main/java/org/apache/spark/k8s/operator/utils/StatusRecorder.java
index b41cf99..87fc4a6 100644
---
a/spark-operator/src/main/java/org/apache/spark/k8s/operator/utils/StatusRecorder.java
+++
b/spark-operator/src/main/java/org/apache/spark/k8s/operator/utils/StatusRecorder.java
@@ -96,14 +96,14 @@ public class StatusRecorder<
Exception err = null;
long maxRetry = API_STATUS_PATCH_MAX_ATTEMPTS.getValue();
for (long i = 0; i < maxRetry; i++) {
- // We retry the status update 3 times to avoid some intermittent
connectivity errors
+ // We retry the status update maxRetry times to avoid some intermittent
connectivity errors
try {
CR updated =
client.resource(resource).lockResourceVersion().updateStatus();
resource.getMetadata().setResourceVersion(updated.getMetadata().getResourceVersion());
err = null;
break;
} catch (KubernetesClientException e) {
- log.warn("Error while patching status, retrying {}/{}...", i + 1,
maxRetry, e);
+ log.debug("Error while patching status, retrying {}/{}...", i + 1,
maxRetry, e);
Thread.sleep(TimeUnit.SECONDS.toMillis(API_RETRY_ATTEMPT_AFTER_SECONDS.getValue()));
err = e;
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
