vandonr-amz commented on code in PR #32414:
URL: https://github.com/apache/airflow/pull/32414#discussion_r1256600883
##########
airflow/providers/amazon/aws/hooks/emr.py:
##########
@@ -284,13 +284,7 @@ def cancel_running_jobs(self, application_id: str,
waiter_config: dict = {}):
)
for job_id in job_ids:
self.conn.cancel_job_run(applicationId=application_id,
jobRunId=job_id)
- if count > 0:
- self.log.info("now waiting for the %s cancelled job(s) to
terminate", count)
- self.get_waiter("no_job_running").wait(
- applicationId=application_id,
-
states=list(self.JOB_INTERMEDIATE_STATES.union({"CANCELLING"})),
- WaiterConfig=waiter_config,
- )
+ return count
Review Comment:
isn't that a breaking change ?
Also, if you were to go with it, you could remove the waiter_config param.
##########
airflow/providers/amazon/aws/operators/emr.py:
##########
@@ -1079,6 +1098,29 @@ def execute(self, context: Context) -> str | None:
)
return application_id
+ def start_application_deferred(self, context, event=None):
+ if event["status"] == "success":
Review Comment:
you could flip this if to reduce indentation
##########
airflow/providers/amazon/aws/operators/emr.py:
##########
@@ -1025,11 +1031,12 @@ def __init__(
self.aws_conn_id = aws_conn_id
self.release_label = release_label
self.job_type = job_type
- self.wait_for_completion = wait_for_completion
+ self.wait_for_completion = False if deferrable else wait_for_completion
Review Comment:
this is not really needed, is it ?
I was actually wondering with the new default if it'd make sense to defer
only if `deferrable && wait_for_completion` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
