potiuk commented on code in PR #848: URL: https://github.com/apache/airflow-site/pull/848#discussion_r1298055550
########## landing-pages/site/content/en/blog/airflow-2.7.0/index.md: ########## @@ -0,0 +1,84 @@ +--- +title: "Apache Airflow 2.7.0 is here" +linkTitle: "Apache Airflow 2.7.0 is here" +author: "Jed Cunningham" +github: "jedcunningham" +linkedin: "jedidiah-cunningham" +description: "Apache Airflow 2.7.0 has been released!" +tags: [Release] +date: "2023-08-18" +--- + +I’m happy to announce that Apache Airflow 2.7.0 has been released! Some notable features have been added that we are excited for the community to use. + +Apache Airflow 2.7.0 contains over 500 commits, which include 40 new features, 49 improvements, 53 bug fixes, and 15 documentation changes. + +**Details**: + +📦 PyPI: https://pypi.org/project/apache-airflow/2.7.0/ \ +📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.7.0/ \ +🛠 Release Notes: https://airflow.apache.org/docs/apache-airflow/2.7.0/release_notes.html \ +🐳 Docker Image: "docker pull apache/airflow:2.7.0" \ +🚏 Constraints: https://github.com/apache/airflow/tree/constraints-2.7.0 + +## Setup and Teardown (AIP-52) + +Airflow now has first class support for the concept of setup and teardown tasks. These tasks have special behavior in that: + +* Teardown tasks will still run, no matter what state the upstream tasks end up in +* Teardown tasks failing won’t, by default, cause the DAG run to fail +* Automatically clear setup/teardown tasks when clearing a dependent task + +You can read more about setup and teardown in the [Introducing Setup and Teardown tasks blog post]({{< ref "blog/introducing_setup_teardown/index.md" >}}), or in the [setup and teardown docs](https://airflow.apache.org/docs/apache-airflow/2.7.0/howto/setup-and-teardown.html). + +## Cluster Activity UI + +There is a new top level page in Airflow, the Cluster Activity page. This gives an overview of the cluster, including component health, dag and task state counts, and more! + + + +## Graph and gantt views moved into the Grid view UI + +The graph and gantt views have been rewritten and moved into the now familiar grid view. This makes it easier to jump between task details, logs, graph, and gantt views without losing your place in a complicated DAG. + + + +## Enable deferrable mode for all deferable tasks with 1 config setting + +Airflow 2.7.0 comes with a new config option, `default_deferrable`, which allows admins to enable deferrable mode for all deferrable tasks without requiring any DAG modifications. Simply set it in your config and enjoy async tasks! + +## OpenLineage provider + +[OpenLineage](https://openlineage.io/) provides a spec standardizing operational lineage collection and distribution across the data ecosystem that projects – open source or proprietary – implement. + +With 2.7.0, OpenLineage changes from a plugin implementation maintained in the OpenLineage project to a built-in feature of Airflow. As a plugin, OpenLineage depended on Airflow and operators’ internals, making it brittle. Built-in OpenLineage support in Airflow makes publishing operational lineage through the OpenLineage ecosystem easier and more reliable. It has been implemented by moving the [openlineage-airflow](https://github.com/OpenLineage/OpenLineage/tree/main/integration/airflow) package from the OpenLineage project to an `apache-airflow-providers-openlineage` provider in the base Airflow Docker image, where it can be easily enabled by configuration. Also, lineage extraction logic that was included in [Extractors](https://github.com/OpenLineage/OpenLineage/tree/main/integration/airflow/openlineage/airflow/extractors) in that package has been moved into each corresponding provider package along with unit tests, eliminating the need for Extractors in most cases. For this pu rpose, a new optional API for Operators (`get_openlineage_facets_on_{start(), complete(ti), failure(ti)}`, documented [here](https://openlineage.io/docs/integrations/airflow/default-extractors)) can be used. Having the extraction logic in each provider ensures the stability of the lineage contract in each operator and makes adding lineage coverage to custom operators easier. + +## Some executors moved into providers + +Some of the executors that were shipped in core Airflow have moved into their respective providers for Airflow 2.7.0. + +The following providers have been moved and require certain minimum providers versions: + +* In order to use Celery executors, install the celery provider version 3.3.0+ +* In order to use the Kubernetes executor, install the kubernetes provider version 7.4.0+ +* In order to use the Dask executor, install any version of the daskexecutor provider + +If you use the official docker images, all of these providers come preinstalled. + +## Additional new features + +Here are just a few interesting new features, since there are too many to list in full: + +* Pools can now consider tasks in the deferred state as running (#32709) +* chain_linear, like chain but allowing sequential tasks (#31927) +* Grid view now supports keyboard shortcuts! (#30950) +* Mark task groups as success or failed (#30478) +* fail_stop, allowing all remaining and running tasks to be failed on the first failure in a DAG (#29406) + Review Comment: I think there are two more things worth mentioning here: * this is the first release of airflow that dropped support for (end of life) Python 3.7 and mentioning that it follows the community and allows to use new features available in 3.8. * I was wondering if we should mention security model and disabling test connection by default. I think it could be acompanied by explaining that we are strenghtening the focus of Airflow on security and having it displayed here as deliberate action might help to fend-off people who will complain (inevitably). While both are kind of "take away" things, they are also explaining that we are looking forward and improving the "infrastructure" of Airflow - and both have quite an impact on airflow "overall". -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
