GitHub user potiuk added a comment to the discussion: High CPU and database load caused by Airflow 3 dag-processor and scheduler
> Airflow 3 should cause the same or lower CPU load to Airflow 2. This is by no means guaranteed or expected. The characteristics of Airflow 3 should be different, but "lower" CPU usage in all scenarios is not guaranteed. I think - in order for this one to be actionable, you need to provide a LOT more information. Airflow has never been targetted for Raspbery PI environment, and while it **might ** work there, there might be many reasons why some resources (not necesarily CPU) will not be enough. It could for example mean that you started to run out of memory and this caused CPU usage while some processes are being swapped out. Or accidentally you might have started using (depend how you build it) Amd images and emulation kicked in (that would certainly explain extremely high CPU usage). So in order to be able to help you with your problem you need to do a lot more investigation - there is absolutely no way we can help you with that little information that you provided. You could do a lot of investigation: 1) check other resources (memory especially) - and possibly using memray to see what is using memory 2) check the architecture your images are using 3) run airflow py-spy to see what is **really** taking a lot of time and provide py-spy flamegraphs For now - this is an interesting discussion, so I convert it to a discussion, but if you provide more evidences and details we might turn it into a real issue. GitHub link: https://github.com/apache/airflow/discussions/53177#discussioncomment-13730501 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
