[
https://issues.apache.org/jira/browse/HADOOP-19880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-19880:
------------------------------------
Description:
My {{[https://github.com/steveloughran/cloudstore]}} repo contains
* diagnostics tools for cloud storage (credentials, bandwidth and more)
* utility commands, including PoC of a fast cloud upload (a single process can
saturate most remote cloud links)
* some version checks
([https://github.com/steveloughran/cloudstore/blob/main/src/main/site/safeprefetch.md],
[https://github.com/steveloughran/cloudstore/blob/main/src/main/site/tarhardened.md]
) which probe for a release having a specific bug.
This has always had o.a.h packaging, but I never got around to submitting it,
as having a fast release cycle (hours to minutes, sometimes), was convenient.
i now do want to add to the hadoop codebase as a separate repo.
* supports builds against older releases
* supports a rapid release cycle where it's easy to have a release as a
snapshot artifact on github
* allows for a commit-then-review review process for what is a low-risk project
Status
* repository is [https://github.com/apache/hadoop-cloudstore]
* merge in progress, after squashing everything down
was:
My {{https://github.com/steveloughran/cloudstore}} repo contains
* diagnostics tools for cloud storage (credentials, bandwidth and more)
* utility commands, including PoC of a fast cloud upload (a single process can
saturate most remote cloud links)
* some version checks
(https://github.com/steveloughran/cloudstore/blob/main/src/main/site/safeprefetch.md,
https://github.com/steveloughran/cloudstore/blob/main/src/main/site/tarhardened.md
) which probe for a release having a specific bug.
This has always had o.a.h packaging, but I never got around to submitting it,
as having a fast release cycle (hours to minutes, sometimes), was convenient.
i now do want to add to the hadoop codebase as a separate repo.
* supports builds against older releases
* supports a rapid release cycle where it's easy to have a release as a
snapshot artifact on github
* allows for a commit-then-review review process for what is a low-risk project
> Add cloudstore to new hadoop repository
> ---------------------------------------
>
> Key: HADOOP-19880
> URL: https://issues.apache.org/jira/browse/HADOOP-19880
> Project: Hadoop Common
> Issue Type: New Feature
> Components: cloud-storage, fs/azure, fs/gcs, fs/s3
> Affects Versions: 3.4.3
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
>
> My {{[https://github.com/steveloughran/cloudstore]}} repo contains
> * diagnostics tools for cloud storage (credentials, bandwidth and more)
> * utility commands, including PoC of a fast cloud upload (a single process
> can saturate most remote cloud links)
> * some version checks
> ([https://github.com/steveloughran/cloudstore/blob/main/src/main/site/safeprefetch.md],
>
> [https://github.com/steveloughran/cloudstore/blob/main/src/main/site/tarhardened.md]
> ) which probe for a release having a specific bug.
> This has always had o.a.h packaging, but I never got around to submitting it,
> as having a fast release cycle (hours to minutes, sometimes), was convenient.
> i now do want to add to the hadoop codebase as a separate repo.
> * supports builds against older releases
> * supports a rapid release cycle where it's easy to have a release as a
> snapshot artifact on github
> * allows for a commit-then-review review process for what is a low-risk
> project
> Status
> * repository is [https://github.com/apache/hadoop-cloudstore]
> * merge in progress, after squashing everything down
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
