dedalozzo commented on issue #65010:
URL: https://github.com/apache/airflow/issues/65010#issuecomment-4232787696
I noticed the same. When I run it locally, on Docker, I use this
configuration:
```yaml
AIRFLOW__CORE__PARALLELISM: 4
AIRFLOW__CORE__MAX_ACTIVE_TASKS_PER_DAG: 2
AIRFLOW__CORE__MAX_ACTIVE_RUNS_PER_DAG: 2
AIRFLOW__API__WORKERS: 1
AIRFLOW__DAG_PROCESSOR__PARSING_PROCESSES: 1
AIRFLOW__CELERY__WORKER_CONCURRENCY: 1
```
With the same configuration, Airflow 3.2.0 is completely unusable. Same
issue on our dev environment with standard configuration and 8 GB of RAM.
Immediately exhausted all the connection from the pool.
Airflow 3.2.0 has critical performance regressions: 14-second DAG parse
times, Task SDK connection pool exhaustion, triggerer burning 100% CPU with 0
triggers, and worker memory bloat (2.5 GB idle)!
Task SDK rewrote the execution model, introducing HTTP API calls for every
task lifecycle event. This saturates DB connection pools and causes cascading
timeouts across all components. No tuning combination resolved the issue.
It is an architectural disgrace. Why did you release something like that?
Release 3.1.9 at least, fix the JWT vulnerability. And then you can take 6
months to rework 3.2.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
