o-nikolas commented on code in PR #31071:
URL: https://github.com/apache/airflow/pull/31071#discussion_r1185456796
##########
tests/system/providers/amazon/aws/example_emr.py:
##########
@@ -164,10 +164,14 @@ def get_step_id(step_ids: list):
task_id="add_steps",
job_flow_id=create_job_flow.output,
steps=SPARK_STEPS,
- wait_for_completion=True,
execution_role_arn=execution_role_arn,
)
# [END howto_operator_emr_add_steps]
+ add_steps.wait_for_completion = True
+ # On rare occasion (1 in 50ish?) this system test times out. Extending
+ # the delay and max_attempts to attempt to mitigate the flaky test.
+ add_steps.waiter_delay = 60
+ add_steps.waiter_max_attempts = 90
Review Comment:
The new config means we'll wait an hour and half, which feels a bit too
long? That would block the whole test suite from completing for that long if
this step hung.
If we used to wait 30min total I'd bump it up to 40min or 45min, anything
beyond that we really need to find some other fix I think :grimacing:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
