kaxil commented on code in PR #65196:
URL: https://github.com/apache/airflow/pull/65196#discussion_r3079296917
##########
airflow-core/src/airflow/dag_processing/manager.py:
##########
@@ -1221,7 +1221,10 @@ def prepare_file_queue(self, known_files: dict[str,
set[DagFileInfo]]):
if self.log.isEnabledFor(logging.DEBUG):
for path, processor in self._processors.items():
self.log.debug(
- "File path %s is still being processed (started: %s)",
path, processor.start_time
+ "File path %s is still being processed (started at: %.2f,
duration: %.2fs)",
+ path,
+ processor.start_time,
+ time.monotonic() - processor.start_time,
Review Comment:
Nit: `_kill_timed_out_processors` right below (line 1236) captures `now =
time.monotonic()` once before its loop and reuses it. Worth doing the same here
so every processor in the batch is measured against the same instant and you
avoid calling `monotonic()` per iteration:
```python
if self.log.isEnabledFor(logging.DEBUG):
now = time.monotonic()
for path, processor in self._processors.items():
...
now - processor.start_time,
```
##########
airflow-core/src/airflow/dag_processing/manager.py:
##########
@@ -1221,7 +1221,10 @@ def prepare_file_queue(self, known_files: dict[str,
set[DagFileInfo]]):
if self.log.isEnabledFor(logging.DEBUG):
for path, processor in self._processors.items():
self.log.debug(
- "File path %s is still being processed (started: %s)",
path, processor.start_time
+ "File path %s is still being processed (started at: %.2f,
duration: %.2fs)",
+ path,
+ processor.start_time,
Review Comment:
The raw monotonic start value (`started at: 2637.46`) is meaningless without
context -- that's the whole reason for this PR. Nobody should be parsing debug
log output for a monotonic clock value. I'd drop it and just show the duration:
```python
"File path %s is still being processed (duration: %.2fs)",
path,
now - processor.start_time,
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
