kalluripradeep commented on PR #64391: URL: https://github.com/apache/airflow/pull/64391#issuecomment-4150745779
The `Integration: providers celery` test failure appears to be an unrelated flaky test with the Celery/RabbitMQ Docker infrastructure. This PR only touches the Spark provider codebase (`spark_submit.py`) and has no overlap with Celery integration. Could a maintainer please re-trigger the CI? Thanks! > I do not think it fixes the issue, we have fixed a similar issue differently, by adjusting the driver pod template and using k8s native sidecar containers > > https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/ > > Which is supported in the k8s version airflow requires ( > 1.30) I agree that using native Kubernetes sidecars (K8s 1.28+) is the much better architectural solution here. Unless you think this post_submit_commands parameter still has value for general worker-side teardown tasks (like local metric emission or temp file cleanup), I am happy to close this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
