potiuk commented on code in PR #64760: URL: https://github.com/apache/airflow/pull/64760#discussion_r3044908769
########## airflow-core/docs/security/jwt_token_authentication.rst: ########## @@ -0,0 +1,453 @@ + .. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + .. http://www.apache.org/licenses/LICENSE-2.0 + + .. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +JWT Token Authentication +======================== + +This document describes how JWT (JSON Web Token) authentication works in Apache Airflow +for both the public REST API (Core API) and the internal Execution API used by workers. + +.. contents:: + :local: + :depth: 2 + +Overview +-------- + +Airflow uses JWT tokens as the primary authentication mechanism for its APIs. There are two +distinct JWT authentication flows: + +1. **REST API (Core API)** — used by UI users, CLI tools, and external clients to interact + with the Airflow public API. +2. **Execution API** — used internally by workers, the Dag File Processor, and the Triggerer + to communicate task state and retrieve runtime data (connections, variables, XComs). + +Both flows share the same underlying JWT infrastructure (``JWTGenerator`` and ``JWTValidator`` +classes in ``airflow.api_fastapi.auth.tokens``) but differ in audience, token lifetime, subject +claims, and scope semantics. + + +Signing and Cryptography +------------------------ + +Airflow supports two mutually exclusive signing modes: + +**Symmetric (shared secret)** + Uses a pre-shared secret key (``[api_auth] jwt_secret``) with the **HS512** algorithm. + All components that generate or validate tokens must share the same secret. If no secret + is configured, Airflow auto-generates a random 16-byte key at startup — but this key is + ephemeral and different across processes, which will cause authentication failures in + multi-component deployments. Deployment Managers must explicitly configure this value. + +**Asymmetric (public/private key pair)** + Uses a PEM-encoded private key (``[api_auth] jwt_private_key_path``) for signing and + the corresponding public key for validation. Supported algorithms: **RS256** (RSA) and + **EdDSA** (Ed25519). The algorithm is auto-detected from the key type when + ``[api_auth] jwt_algorithm`` is set to ``GUESS`` (the default). + + Validation can use either: + + - A JWKS (JSON Web Key Set) endpoint configured via ``[api_auth] trusted_jwks_url`` + (local file or remote HTTP/HTTPS URL, polled periodically for updates). + - The public key derived from the configured private key (automatic fallback when + ``trusted_jwks_url`` is not set). + +The asymmetric mode is recommended for production deployments where you want workers +and the API server to operate with different credentials (workers only need the private key for +token generation; the API server only needs the JWKS for validation). + + +REST API Authentication Flow +----------------------------- + +Token acquisition +^^^^^^^^^^^^^^^^^ + +1. A client sends a ``POST`` request to ``/auth/token`` with credentials (e.g., username + and password in JSON body). +2. The auth manager validates the credentials and creates a user object. +3. The auth manager serializes the user into JWT claims and calls ``JWTGenerator.generate()``. +4. The generated token is returned in the response as ``access_token``. + +For UI-based authentication, the token is stored in a secure, HTTP-only cookie (``_token``) +with ``SameSite=Lax``. + +The CLI uses a separate endpoint (``/auth/token/cli``) with a different (shorter) expiration +time. + +Token structure (REST API) +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. list-table:: + :header-rows: 1 + :widths: 15 85 + + * - Claim + - Description + * - ``jti`` + - Unique token identifier (UUID4 hex). Used for token revocation. + * - ``iss`` + - Issuer (from ``[api_auth] jwt_issuer``). Optional but recommended. + * - ``aud`` + - Audience (from ``[api_auth] jwt_audience``). Optional but recommended. + * - ``sub`` + - User identifier (serialized by the auth manager). + * - ``iat`` + - Issued-at timestamp (Unix epoch seconds). + * - ``nbf`` + - Not-before timestamp (same as ``iat``). + * - ``exp`` + - Expiration timestamp (``iat + jwt_expiration_time``). + +Token validation (REST API) +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +On each API request, the token is extracted in this order of precedence: + +1. ``Authorization: Bearer <token>`` header. +2. OAuth2 query parameter. +3. ``_token`` cookie. + +The ``JWTValidator`` verifies the signature, expiry (``exp``), not-before (``nbf``), +issued-at (``iat``), audience, and issuer claims. A configurable leeway +(``[api_auth] jwt_leeway``, default 10 seconds) accounts for clock skew. + +Token revocation (REST API only) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Token revocation applies only to REST API and UI tokens — it is **not** used for Execution API +tokens issued to workers. + +Revoked tokens are tracked in the ``revoked_token`` database table by their ``jti`` claim. +On logout or explicit revocation, the token's ``jti`` and ``exp`` are inserted into this +table. Expired entries are automatically cleaned up at a cadence of ``2× jwt_expiration_time``. + +Execution API tokens are not subject to revocation. They are short-lived (default 10 minutes) +and automatically refreshed by the ``JWTReissueMiddleware``, so revocation is not part of the +Execution API security model. Once an Execution API token is issued to a worker, it remains +valid until it expires. + +Token refresh (REST API) +^^^^^^^^^^^^^^^^^^^^^^^^ + +The ``JWTRefreshMiddleware`` runs on UI requests. When the middleware detects that the +current token's ``_token`` cookie is approaching expiry, it calls +``auth_manager.refresh_user()`` to generate a new token and sets it as the updated cookie. + +Default timings (REST API) +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. list-table:: + :header-rows: 1 + :widths: 50 50 + + * - Setting + - Default + * - ``[api_auth] jwt_expiration_time`` + - 86400 seconds (24 hours) + * - ``[api_auth] jwt_cli_expiration_time`` + - 3600 seconds (1 hour) + * - ``[api_auth] jwt_leeway`` + - 10 seconds + + +Execution API Authentication Flow +---------------------------------- + +The Execution API is an internal API used by workers to report task state transitions, +heartbeats, and to retrieve connections, variables, and XComs at task runtime. + +Token generation (Execution API) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +1. The **Scheduler** (via the executor) generates a JWT for each task instance before + dispatching it to a worker. The executor's ``jwt_generator`` property creates a + ``JWTGenerator`` configured with the ``[execution_api]`` settings. +2. The token's ``sub`` (subject) claim is set to the **task instance UUID**. +3. The token is embedded in the workload JSON payload (``BaseWorkloadSchema.token`` field) + that is sent to the worker process. + +Token structure (Execution API) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. list-table:: + :header-rows: 1 + :widths: 15 85 + + * - Claim + - Description + * - ``jti`` + - Unique token identifier (UUID4 hex). + * - ``iss`` + - Issuer (from ``[api_auth] jwt_issuer``). Optional. + * - ``aud`` + - Audience (from ``[execution_api] jwt_audience``, default: ``urn:airflow.apache.org:task``). + * - ``sub`` + - Task instance UUID — the identity of the workload. + * - ``scope`` + - Token scope: ``"execution"`` (default) or ``"workload"`` (restricted). + * - ``iat`` + - Issued-at timestamp. + * - ``nbf`` + - Not-before timestamp. + * - ``exp`` + - Expiration timestamp (``iat + [execution_api] jwt_expiration_time``). + +Token scopes (Execution API) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The Execution API defines two token scopes: + +**execution** (default) + Accepted by all Execution API endpoints. This is the standard scope for worker + communication. + +**workload** + A restricted scope accepted only on endpoints that explicitly opt in via + ``Security(require_auth, scopes=["token:workload"])``. Used for endpoints that + manage task state transitions. + +Tokens without a ``scope`` claim default to ``"execution"`` for backwards compatibility. + +Token delivery to workers +^^^^^^^^^^^^^^^^^^^^^^^^^ + +The token flows through the execution stack as follows: + +1. **Executor** generates the token and embeds it in the workload JSON payload. +2. The workload JSON is passed to the worker process (via the executor-specific mechanism: + Celery message, Kubernetes Pod spec, local subprocess arguments, etc.). +3. The worker's ``execute_workload()`` function reads the workload JSON and extracts the token. +4. The ``supervise()`` function receives the token and creates an ``httpx.Client`` instance + with ``BearerAuth(token)`` for all Execution API HTTP requests. +5. The token is included in the ``Authorization: Bearer <token>`` header of every request. + +Token validation (Execution API) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The ``JWTBearer`` security dependency validates the token once per request: + +1. Extracts the token from the ``Authorization: Bearer`` header. +2. Performs cryptographic signature validation via ``JWTValidator``. +3. Verifies standard claims (``exp``, ``iat``, ``aud`` — ``nbf`` and ``iss`` if configured). +4. Defaults the ``scope`` claim to ``"execution"`` if absent. +5. Creates a ``TIToken`` object with the task instance ID and claims. +6. Caches the validated token on the ASGI request scope for the duration of the request. + +Route-level enforcement is handled by ``require_auth``: + +- Checks the token's ``scope`` against the route's ``allowed_token_types`` (precomputed + by ``ExecutionAPIRoute`` from ``token:*`` Security scopes at route registration time). +- Enforces ``ti:self`` scope — verifies that the token's ``sub`` claim matches the + ``{task_instance_id}`` path parameter, preventing a worker from accessing another task's + endpoints. + +Token refresh (Execution API) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The ``JWTReissueMiddleware`` automatically refreshes tokens that are approaching expiry: + +1. After each response, the middleware checks the token's remaining validity. +2. If less than **20%** of the total validity remains (minimum 30 seconds), the server + generates a new token preserving all original claims (including ``scope`` and ``sub``). +3. The refreshed token is returned in the ``Refreshed-API-Token`` response header. +4. The client's ``_update_auth()`` hook detects this header and transparently updates + the ``BearerAuth`` instance for subsequent requests. + +This mechanism ensures long-running tasks do not lose API access due to token expiry, +without requiring the worker to re-authenticate. + +Default timings (Execution API) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. list-table:: + :header-rows: 1 + :widths: 50 50 + + * - Setting + - Default + * - ``[execution_api] jwt_expiration_time`` + - 600 seconds (10 minutes) + * - ``[execution_api] jwt_audience`` + - ``urn:airflow.apache.org:task`` + * - Token refresh threshold + - 20% of validity remaining (minimum 30 seconds, i.e., at ~120 seconds before expiry + with the default 600-second token lifetime) + + +Dag File Processor and Triggerer +--------------------------------- + +The **Dag File Processor** and **Triggerer** are internal Airflow components that also +interact with the Execution API, but they do so via an **in-process** transport +(``InProcessExecutionAPI``) rather than over the network. This in-process API: + +- Runs the Execution API application directly within the same process, using an ASGI/WSGI + bridge. +- **Potentially bypasses JWT authentication** — the JWT bearer dependency is overridden to + always return a synthetic ``TIToken`` with the ``"execution"`` scope, effectively bypassing + token validation. +- Also potentially bypasses per-resource access controls (connection, variable, and XCom access + checks are overridden to always allow). + +Airflow implements software guards that prevent accidental direct database access from Dag +author code in these components. However, because the child processes that parse Dag files and +execute trigger code run as the **same Unix user** as their parent processes, these guards do +not protect against intentional access. A deliberately malicious Dag author can potentially +retrieve the parent process's database credentials (via ``/proc/<PID>/environ``, configuration +files, or secrets manager access) and gain full read/write access to the metadata database and +all Execution API operations — without needing a valid JWT token. + +This is in contrast to workers, where the isolation is genuine: worker processes do not receive +database credentials at all and communicate exclusively through the Execution API. Review Comment: Updated: > This is in contrast to workers/task execution, where the isolation is implemented ad deployment level - where sensitive configuration of database credentials is not available to Airflow processes because they are not set in their deployment configuration at all, and communicate exclusively through the Execution API. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
