o-nikolas commented on PR #54523: URL: https://github.com/apache/airflow/pull/54523#issuecomment-3197926664
> > LocalExecutor would only actually begin execution for every second task submitted to it > > That doesn't match up with my experience. I'm able to start up and run a dag with one task perfectly happily. Or I can clear a single task as many times as I like and it runs immediately. > > I'll take a closer look tomorrow > > Edit: Ohhh, once it hits concurrency limits, right. K, that'll teach me for looking at things on the weekend Yeah, to clarify, the dag must have _many_ tasks to see the execution pattern, not just one. It's not possible to run up to the max active tasks or parallelism because only every second call to submit a task to the LocalExecutor from the scheduler results in a running task (the others stay in the queued state). Then we hit the limit of running and queued task and the scheduler no longer runs any more until slots are freed. Run the attached dag and you will see 8 tasks start running and 8 stay queued for the entire duration of the running tasks. Instead of all 16 proceeding to running -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
