Copilot commented on code in PR #64018:
URL: https://github.com/apache/airflow/pull/64018#discussion_r3025334303
##########
airflow-core/docs/howto/create-custom-decorator.rst:
##########
@@ -86,6 +98,65 @@ tasks. The steps to create and register ``@task.foo`` are:
Please note that the ``name`` must be a valid python identifier.
+Alternative: Creating Decorators for Local Projects
+---------------------------------------------------
+
+If you want to create custom decorators for use within a single Airflow
project without creating a full provider package,
+you can use a simpler factory approach. This is useful for project-specific
decorators like Django integration.
+
+Example: Creating a Django task decorator:
+
+.. code-block:: python
+
+ from __future__ import annotations
+ from pathlib import Path
+ from typing import Callable
+ from airflow.sdk import task
+
+ def django_connect(app_path: Path, settings_module: str):
+ """Connect to Django database - implement your Django setup here"""
+ import os
+ import django
+ os.environ.setdefault('DJANGO_SETTINGS_MODULE', settings_module)
+ django.setup()
+
+ def make_django_decorator(app_path: Path, settings_module: str):
+ """Factory function to create a Django-specific task decorator"""
+
+ def django_task(task_id: str = None, *args, **kwargs):
+ def django_task_decorator(fn: Callable):
+ @task(task_id=task_id or fn.__name__, *args, **kwargs)
+ def new_fn(*fn_args, **fn_kwargs):
Review Comment:
The local-project factory example doesn’t support being used as a bare
decorator, but the later example uses it as ``@django_task``. With the current
signature (``task_id`` first), using ``@django_task`` will pass the function
object as ``task_id`` and produce an invalid/non-string task_id. Either update
the usage to ``@django_task()`` (or ``@django_task(task_id=...)``), or adjust
``django_task`` to accept an optional ``python_callable`` first so it works
both with and without parentheses.
##########
airflow-core/docs/howto/create-custom-decorator.rst:
##########
@@ -68,6 +76,10 @@ tasks. The steps to create and register ``@task.foo`` are:
a list with each item containing ``name`` and ``class-name`` keys. When
Airflow starts, the
``ProviderManager`` class will automatically import this value and
``task.foo`` will work as a new decorator!
+ The **ProviderManager** is Airflow's internal system that discovers and
registers task decorators from provider packages.
+ When Airflow starts, it scans all installed providers for entry points,
calls each provider's ``get_provider_info()`` function,
+ and dynamically attaches registered decorators to ``@task`` as
``@task.decorator_name``.
Review Comment:
This section refers to a ``ProviderManager`` class, but in this codebase the
implementation is ``ProvidersManager`` (plural) in
``airflow.providers_manager``. Also, ``@task.<name>`` resolution is lazy at
attribute-access time via ``TaskDecoratorCollection.__getattr__`` and
``ProvidersManagerTaskRuntime().taskflow_decorators``, rather than being
“dynamically attached”/patched during startup—please adjust the wording so it
matches how decorators are discovered and resolved.
```suggestion
``ProvidersManager`` class will automatically import this value and
``task.foo`` will work as a new decorator!
The **ProvidersManager** (from ``airflow.providers_manager``) is
Airflow's internal system that discovers task
decorators from provider packages by reading their
``get_provider_info()`` metadata (including ``task-decorators``).
At runtime, attributes on ``task`` such as ``task.foo`` are resolved
lazily via
``TaskDecoratorCollection.__getattr__`` and
``ProvidersManagerTaskRuntime().taskflow_decorators``, so referencing
``@task.foo`` will look up and return the registered ``foo_task``
decorator when it is first accessed.
```
##########
airflow-core/docs/howto/create-custom-decorator.rst:
##########
@@ -86,6 +98,65 @@ tasks. The steps to create and register ``@task.foo`` are:
Please note that the ``name`` must be a valid python identifier.
+Alternative: Creating Decorators for Local Projects
+---------------------------------------------------
+
+If you want to create custom decorators for use within a single Airflow
project without creating a full provider package,
+you can use a simpler factory approach. This is useful for project-specific
decorators like Django integration.
+
+Example: Creating a Django task decorator:
+
+.. code-block:: python
+
+ from __future__ import annotations
+ from pathlib import Path
+ from typing import Callable
+ from airflow.sdk import task
+
+ def django_connect(app_path: Path, settings_module: str):
+ """Connect to Django database - implement your Django setup here"""
+ import os
+ import django
+ os.environ.setdefault('DJANGO_SETTINGS_MODULE', settings_module)
+ django.setup()
+
+ def make_django_decorator(app_path: Path, settings_module: str):
Review Comment:
``django_connect`` takes an ``app_path`` parameter but never uses it, and
``make_django_decorator`` passes it through. In a documentation example this is
confusing because it implies the path matters. Either show how ``app_path`` is
used (e.g., adjusting ``sys.path``/working directory as part of the setup) or
drop the parameter from the example to keep it minimal and accurate.
##########
airflow-core/docs/howto/create-custom-decorator.rst:
##########
@@ -86,6 +98,65 @@ tasks. The steps to create and register ``@task.foo`` are:
Please note that the ``name`` must be a valid python identifier.
+Alternative: Creating Decorators for Local Projects
+---------------------------------------------------
+
+If you want to create custom decorators for use within a single Airflow
project without creating a full provider package,
+you can use a simpler factory approach. This is useful for project-specific
decorators like Django integration.
+
+Example: Creating a Django task decorator:
+
+.. code-block:: python
+
+ from __future__ import annotations
+ from pathlib import Path
+ from typing import Callable
+ from airflow.sdk import task
+
+ def django_connect(app_path: Path, settings_module: str):
+ """Connect to Django database - implement your Django setup here"""
+ import os
+ import django
+ os.environ.setdefault('DJANGO_SETTINGS_MODULE', settings_module)
+ django.setup()
+
+ def make_django_decorator(app_path: Path, settings_module: str):
+ """Factory function to create a Django-specific task decorator"""
+
+ def django_task(task_id: str = None, *args, **kwargs):
+ def django_task_decorator(fn: Callable):
+ @task(task_id=task_id or fn.__name__, *args, **kwargs)
+ def new_fn(*fn_args, **fn_kwargs):
+ # Connect to Django database before running the function
+ django_connect(app_path, settings_module)
+ # Now the function can use Django models
+ return fn(*fn_args, **fn_kwargs)
+ return new_fn
+ return django_task_decorator
+ return django_task
+
+Usage in your DAG:
+
+.. code-block:: python
+
+ from plugins.django_decorator import make_django_decorator
+ from pathlib import Path
Review Comment:
The import ``from plugins.django_decorator import make_django_decorator`` is
likely incorrect/misleading for most setups: modules in
``$AIRFLOW_HOME/plugins`` are typically imported directly by filename (e.g.
``from django_decorator import ...``), and many users will instead place helper
modules alongside their Dag files. Consider rephrasing this to a more generic
import (e.g. ``from django_decorator import ...``) and explicitly stating where
the helper module should live so the import works.
##########
airflow-core/docs/howto/create-custom-decorator.rst:
##########
@@ -86,6 +98,65 @@ tasks. The steps to create and register ``@task.foo`` are:
Please note that the ``name`` must be a valid python identifier.
+Alternative: Creating Decorators for Local Projects
+---------------------------------------------------
+
+If you want to create custom decorators for use within a single Airflow
project without creating a full provider package,
+you can use a simpler factory approach. This is useful for project-specific
decorators like Django integration.
+
+Example: Creating a Django task decorator:
+
+.. code-block:: python
+
+ from __future__ import annotations
+ from pathlib import Path
+ from typing import Callable
+ from airflow.sdk import task
+
+ def django_connect(app_path: Path, settings_module: str):
+ """Connect to Django database - implement your Django setup here"""
+ import os
+ import django
+ os.environ.setdefault('DJANGO_SETTINGS_MODULE', settings_module)
+ django.setup()
+
+ def make_django_decorator(app_path: Path, settings_module: str):
+ """Factory function to create a Django-specific task decorator"""
+
Review Comment:
The code sample includes blank lines that contain trailing spaces (e.g.,
right after imports and before/after code blocks). The repo runs the pre-commit
``trailing-whitespace`` hook, so these spaces will fail checks—please remove
the whitespace on otherwise-empty lines in this section.
```suggestion
def django_connect(app_path: Path, settings_module: str):
"""Connect to Django database - implement your Django setup here"""
import os
import django
os.environ.setdefault('DJANGO_SETTINGS_MODULE', settings_module)
django.setup()
def make_django_decorator(app_path: Path, settings_module: str):
"""Factory function to create a Django-specific task decorator"""
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
