I've never tried, but it seems possible to start a Httpfs server with fs.defaultFS = s3a://your-bucket Httpfs server speaks WebHDFS protocol so your webhdfs client can use webhdfs. And then for each webhdfs request, httpfs server translates that into the corresponding FileSystem API call. If the fs.defaultFS is the s3a:// URI, it may be able to talk to s3.
On Wed, May 22, 2019 at 3:29 PM Joseph Henry <[email protected]> wrote: > Hey, > > > > I am not sure if this is the correct mailing list for this question, but I > will start here. > > > > Our client application needs to support accessing S3 buckets from hdfs. We > can do this with the Java API using the s3a:// scheme, but also need a way > to access the same files in S3 via the HDFS REST API. > > > > Is there a way to access the data stored in S3 via WEBHDFS? > > > > Thanks, > > Joseph Henry. > > >
