[
https://issues.apache.org/jira/browse/HDFS-11161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18065579#comment-18065579
]
ASF GitHub Bot commented on HDFS-11161:
---------------------------------------
yangdong2398 opened a new pull request, #8343:
URL: https://github.com/apache/hadoop/pull/8343
## Summary
- Add `hadoop-bos` module under `hadoop-cloud-storage-project` providing
Baidu Object Storage (BOS) FileSystem implementation (`bos://` scheme)
- Support both flat and hierarchical namespace BOS buckets
- Shade BOS SDK and transitive dependencies under `bfs.*` prefix to avoid
classpath conflicts
- Integrate into parent POMs (`hadoop-project`, `hadoop-cloud-storage`,
`hadoop-cloud-storage-dist`)
- Follow Hadoop community conventions: ASF license headers,
package-info.java, auth-keys.xml test credential pattern, tests-off/tests-on
profiles
## Changes
- **New module**: `hadoop-cloud-storage-project/hadoop-bos` with BOS
FileSystem, credentials providers, shaded jar packaging
- **POM integration**: Added to `hadoop-project/pom.xml` (dependency
management), `hadoop-cloud-storage-project/pom.xml` (module),
`hadoop-cloud-storage/pom.xml` (umbrella dependency with `*:*` exclusion),
`hadoop-cloud-storage-dist/pom.xml` (dist + packaging profile)
- **Shade plugin**: Relocates jackson, httpcomponents, guava,
commons-logging, commons-lang, commons-codec, BOS SDK under `bfs.*`
- **Test config**: Standard `core-site.xml` + XInclude `auth-keys.xml`
pattern; tests auto-skip when credentials absent
## Test plan
- [ ] `mvn compile -pl hadoop-cloud-storage-project/hadoop-bos` builds
successfully
- [ ] `mvn package -pl hadoop-cloud-storage-project/hadoop-bos -DskipTests`
produces shaded jar
- [ ] Verify shaded jar contains relocated classes under `bfs.*` prefix
- [ ] Contract tests pass with valid BOS credentials in `auth-keys.xml`
- [ ] Build skips tests when `auth-keys.xml` is absent
- [ ] `mvn package -Pdist -DskipTests -Dtar -Dhadoop-bos-package` includes
hadoop-bos in distribution
🤖 Generated with [Claude Code](https://claude.com/claude-code)
> Incorporate Baidu Yun BOS file system implementation
> ----------------------------------------------------
>
> Key: HDFS-11161
> URL: https://issues.apache.org/jira/browse/HDFS-11161
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: fs
> Reporter: Faen Zhang
> Priority: Major
> Original Estimate: 840h
> Remaining Estimate: 840h
>
> Baidu Yun ( https://cloud.baidu.com/ ) is one of top tier cloud computing
> provider. Baidu Yun BOS is widely used among China's cloud users, but
> currently it is not easy to access data laid on BOS storage from user's
> Hadoop/Spark application, because of no original support for BOS in Hadoop.
> This work aims to integrate Baidu Yun BOS with Hadoop. By simple
> configuration, Spark/Hadoop applications can read/write data from BOS without
> any code change. Narrowing the gap between user's APP and data storage, like
> what have been done for S3 and Aliyun OSS in Hadoop.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
