This is really cool, thanks for setting all of this up! I just took a look at the dashboard and it is really helpful :)
Le lun. 23 janv. 2023 à 16:55, Beck, Vincent <[email protected]> a écrit : > The AWS Airflow team is happy to announce that the dashboard we have been > building which summarizes the health status of system tests within the > Amazon provider package is now available here: > https://aws-mwaa.github.io/open-source/system-tests/dashboard.html. The > purpose is to share with anyone who is interested in the results of the > latest executions of our system tests. These system tests are executed > frequently internally (every hour if new commits have arrived during that > hour) using the last main version and this dashboard shares the results of > these executions in a transparent way. > > > > In this dashboard you can find the list of all system tests within the > Amazon provider package and some statistics about the last executions, such > as: number of invocations, number of successes, number of failures and > average duration. You can also see the status (succeed/failed) of the last > 10 executions for each system test. > > > > *What is the use?* > > > > This dashboard can be used in multiple different scenarios. Here are some > examples: > > > > * When releasing new Airflow provider packages, this dashboard can be used > to check the Amazon provider package health status > > * Being able to detect early some failures or bugs in Amazon provider > package because of new code changes > > > > *Why some system tests are not listed in this dashboard?* > > > > If a system test in the Amazon provider package system test directory ( > https://github.com/apache/airflow/tree/main/tests/system/providers/amazon/aws) > is not listed in this dashboard, it can be due to two reasons: > > > > * It is a relatively new system test and we need internally to configure > it to run in our testing pipeline. Once this configuration is done, it will > show up in the dashboard > > * We cannot support this system test for various reasons (e.g. it needs a > third party account/resource). This is decided case by case so if you ever > wonder why a given system test is not listed in the dashboard, feel free to > ask in #airflow-aws, we’ll happily answer > > > > *How to write a system test myself?* > > > > First of all, any contribution is welcome so please add system tests where > applicable. When doing so, please follow the guidelines defined here > https://github.com/apache/airflow/tree/main/tests/system/providers/amazon. > Once the new system test is merged, as mentioned above, the AWS Airflow > team will write the configuration needed to run it. Once this is done, the > system test you wrote will appear in the dashboard. > > > > *A system test is failing, does it mean something is broken?* > > > > Not necessarily, it can also be due to an issue in our internal > infrastructure which runs the system tests. In such a situation, out team > will investigate the system test failure and act upon it. If it is a > legitimate regression in Airflow/Amazon Provider Package we may work on the > fix ourselves or create a GitHub issue in the Airflow repository to track > the fix. This allows the community to get involved. > > > > This is just a first step and we will try to improve it over time. Any > suggestions/recommendations/improvements are very welcome so please do > reach out (through the different medias) if you have any. >
