potiuk commented on code in PR #36287: URL: https://github.com/apache/airflow/pull/36287#discussion_r1430680869
########## docs/README.rst: ########## @@ -20,40 +20,107 @@ Documentation This directory contains documentation for the Apache Airflow project and other packages that are closely related to it ie. providers packages. Documentation is built using `Sphinx <https://www.sphinx-doc.org/>`__. +Development documentation preview +================================== +Footer +© 2023 GitHub, Inc. +Footer navigation +Terms + Development documentation preview ================================== Documentation from the development version is built and automatically published: `s.apache.org/airflow-docs <https://s.apache.org/airflow-docs>`_ +Working with Sphinx +=================== + +Airflow Documentation uses `Sphinx <https://www.sphinx-doc.org/>`__, a reStructure Text (.rst) markup language that was developed to document Python and Python projects, to build the docs site. + +For most Docs writing purposes, the `reStructured Text Primer <https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html>`__ provides a quick reference of common formatting and syntax. + +A general docs workflow +----------------------- +When you make changes to the docs, it follows roughly the same process as creating and testing code changes. However, for docs, instead of unit tests and integration tests, you run pre-commit checks, spell checks, and visual inspection of your changes in a documentation build. + +1. **Decide to edit in GitHub UI or locally** - Depending on the size and type of docs update you want to make, it might be easier to work in the UI or to make your edits in a local fork. +2. **Find the source files to edit** - While you can access most of the docs source files using the **Suggest a change on this page** button or by searching for a string in the ``/docs/`` file directory, in some cases, the source strings might be located in different provider docs or in the source code itself. +3. **If editing locally, run spellcheck and the build to identify any blocking errors** - Docs require build, spellcheck, and precommit CI/CD tests to pass before they can merge. This means that if you have a pull request with docs changes, a docs build error can prevent your code merge. Checking the build and spelling locally first can help speed up reviews. If you made formatting changes, checking a local build of your docs allows you to make sure you correctly formatted elements like tables, text styling, and line breaks. +4. **Make your pull request** - When you make a PR, Github automatically assigns reviewers and runs CI/CD tests. +5. **Fix any build errors or spelling mistakes** - Your PR can't be merged if there are any spelling or build errors. Check to see which builds are failing and click **Show details**. The output of the tests share the errors, location of the problems, and suggest resolutions. Common Docs failures occur due to incorrect formatting and whitespace. + +Editing in GitHub or locally +---------------------------- + +You have two options for editing Airflow docs: + +1. Through the online GitHub Editor by clicking **Suggest a change on this page** in the `docs <https://airflow.apache.org/docs/>`_, or by selecting a file in `GitHub <https://github.com/apache/airflow/tree/main/docs>`__. + +2. Locally with a forked copy of the Airflow repo, where you can run local builds and tests prior to making a pull request. + ++--------------------------------------+------------------+-------------------------------------------------+ +| Type of Docs update | Suggested Editor | Explanation | ++======================================+==================+=================================================+ +| I need to edit multiple files. | Local Editor | It's easier to batch-edit files in an editor, | +| | | than make multiple PRs or changes to individual | +| | | files in a GitHub editor. | ++--------------------------------------+------------------+-------------------------------------------------+ +| I want to fix a quick typo or a | GitHub Editor | Allows you to quickly edit without any local | +| broken link. | | installation or build required. | ++--------------------------------------+------------------+-------------------------------------------------+ +| My edits contain tables or | Local Editor | GitHub can provide Markdown previews, but might | +| other formatting changes. | | change ``.rst``` styling. Use a local build. | ++--------------------------------------+------------------+-------------------------------------------------+ +| I want to make a new page/ | Local Editor | Will need a local build to check navigation and | +| delete a page. | | link redirects. | ++--------------------------------------+------------------+-------------------------------------------------+ +| I want to edit autogenerated content | Either, probably | Allows you to easily find the correct file and | +| on a page. | Local Editor | generate a preview before creating the PR. | ++--------------------------------------+------------------+-------------------------------------------------+ + +Finding source content to edit +------------------------------ + +Sphinx has _roles_ and _directives_, where Markdown docs frameworks often do not have similar functionality. This means that Airflow uses directives +to pull code examples, autogenerate indexes and tables of contents, and reference from resources across the codebase and across documentation provider packages. +This can make it confusing to find the content source on certain types of reference pages. + +For example, in `Command Line Interface and Environment Variables Reference <https://airflow.apache.org/docs/apache-airflow/stable/cli-and-env-variables-ref.html#environment-variables>`__, the CLI reference is `autogenerated <https://github.com/apache/airflow/blob/main/docs/apache-airflow/cli-and-env-variables-ref.rst?plain=1#L44>`__, +and requires more complex scripting. While the `Environment Variables <https://github.com/apache/airflow/blob/main/docs/apache-airflow/cli-and-env-variables-ref.rst?plain=1#L51>`__ are explicitly written. + +To make an edit to an autogenerated doc, you need to make changes to a string in the Python source file. In the previous example, to make edits to the CLI command reference text, you must edit the `cli_config.py <https://github.com/apache/airflow/blob/main/airflow/cli/cli_config.py#L1861>`__ source file. + Building documentation ====================== -To generate a local version you can use `<../BREEZE.rst>`_. +To generate a local version of the docs you can use `<../BREEZE.rst>`_. The documentation build consists of verifying consistency of documentation and two steps: * spell checking * building documentation -You can only run one of the steps via ``--spellcheck-only`` or ``--docs-only``. +You can choose to run the complete build, to build all the docs and run spellcheck. Or, you can use the optional flags, ``--spellcheck-only``, ``--docs-only``, or ``--package-filter`` to choose the scope of the build. + +Build all docs and spell check them: .. code-block:: bash breeze build-docs -or just to run spell-check +Just run spellcheck: .. code-block:: bash breeze build-docs --spellcheck-only -or just to run documentation building +Build docs without checking spelling: .. code-block:: bash breeze build-docs --docs-only -Also, you can only build one documentation via ``--package-filter``. +Build documentation of just one provider package: Review Comment: The example below can be changed to: ``` Example below can be changed this way: ``` breeze build-docs PACKAGE_ID ``` Where package id is a short id of the package ("apache-aifflow" , "docker-stack", "helm-chart", "google", "apache.beam" etc.) And likely a good idea to add another example: ``` breeze build-docs PACKAGE1_ID PACKAGE2_ID ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
