oven-yang commented on issue #1130: URL: https://github.com/apache/iceberg-rust/issues/1130#issuecomment-2757182202
> > Have you considered including libhdfs + JNI in this design? Are we intentionally leaving it out for now, or could it be explored as a separate approach? > > Yep, I intentionally leaving it out for now since I don't want to play with java runtime for now. And I'm guessing adding it doesn't need another approach. We can just follow the same way to handle it. I actually tested accessing HDFS using libhdfs + JNI myself, and it worked! The main issue I ran into was linking the JVM library. It’d be awesome if support for this could be added here eventually. > > I was wondering if the viewfs:// scheme might be supported in this context? I’ve tested it with OpenDAL and confirmed that it’s already supported there. > > Oh, that's something new to me. We just need tp map `viewfs` to `hdfs`, right? To clarify, I ran my test using libhdfs + JNI, and I’m not entirely sure if `viewfs` is supported in the other two approaches. In my experience with libhdfs + JNI, `viewfs` is handled the same way as `hdfs`, with libhdfs taking care of the differences under the hood. That said, it seems like we’d only need to explicitly pass the scheme to OpenDAL for it to work smoothly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
