Jarek Potiuk created ATTIC-218:
----------------------------------
Summary: Moving one of mny packages to attic (or just removing it
from the code)?
Key: ATTIC-218
URL: https://issues.apache.org/jira/browse/ATTIC-218
Project: Attic
Issue Type: Task
Reporter: Jarek Potiuk
Hello,
We have an interesting situation in Apache Airflow that I would like to have an
advice on.
When we want to remove some small (but with separate release pacakge) part of
the PMC project, should we (somehow?) move the code we remove to attic, or is
it just fine to remove it from sources?
A bit of context:
Apache Airflow is a huge and rather successful PMC (>2600 contributors, tens of
thousands of users) but it also rather complex one. Since Airflow is an
orchestrator and interacts with multiple eservices, instead of a single or few
package we have 1 core, 1 python client and ~ 90 provider packages - each
provider is a separately released convenience package, separately released
source package. Every two weeks or so we release and vote on between few to all
90 provider packages (there is a well established process of bulk voting.
Example one here:
[https://lists.apache.org/thread/j4b5851g9lqg0jm8yb68f83vqfxzqy14]
We released all providers that are "active" - because we bumped the minimum
version of Airflow according to our policies.
All those providers are managed together with Airflow in a single monorepo
[https://github.com/apache/airflow/tree/main/airflow/providers]
We currently have also well established process of suspending providers that
are - for some reason holding us back (mainly due to dependencies) - we stop
releasing them, we stop updating the code and the code in the repo gets
efffectively ignored by tests/CI/release process. The process, voting, resuming
is well established and being followed (we already suspended - and later
resumed yandex, and currently we suspended qubole provider). This is really
nice for temporary suspensions as it does not hold the development, but it
makes it really easy (just a PR) to fix and resume such provider (worked very
nicely in case of yandex who got resumed after upstream libraries fixed some
dependency issues)
https://github.com/apache/airflow/blob/main/airflow/providers/SUSPENDING_AND_RESUMING_PROVIDERS.rst
However we eventually would like to remove qubole. It's pretty useless now -
because the service (qubole) that it connected to had been acquired and
discontinued
The website is still there but see the note here:
[https://github.com/qubole/qds-sdk-py#where-are-the-maintainers-] (note added
7 months ago) - there is basically no way any of their libraries will get
updated, and the service does not work any more. Maybe someone would like to
use the code we have there but for us it is a burden (dependencies, refactors
etc. etc.) and we would very much like to just remove it. from `main` and
simply mark the "apache-airflow-providers-qubole" package as "gone".
So we would like to remove the code and somehow let the users know that the
package we released in the past is "discontinued" in PyPI
[https://pypi.org/project/apache-airflow-providers-qubole/]
So I have a few questions:
1) should attic PMC be involved when we remove the code or are we fine to
**just** remove it.
2) any advice on communication / informing our users?
3) should we do anything about the package we release in PyPI?
Would love to hear what you think.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
