847850277 commented on code in PR #23935:
URL: https://github.com/apache/datafusion/pull/23935#discussion_r3746041465
##########
datafusion/execution/src/object_store.rs:
##########
@@ -263,13 +263,20 @@ impl ObjectStoreRegistry for DefaultObjectStoreRegistry {
}
}
-/// Get the key of a url for object store registration.
-/// The credential info will be removed
+/// Get the key of a URL for object store registration.
+///
+/// Userinfo is preserved for ABFS schemes, where it identifies a namespace,
+/// and removed for all other schemes.
fn get_url_key(url: &Url) -> String {
+ let authority_start = match url.scheme() {
+ "abfs" | "abfss" if !url.username().is_empty() =>
url::Position::BeforeUsername,
Review Comment:
> it is quite unfortunate that we need to do something abfs specific -- is
there some way to make this more general (or hide the object store specific
behavior in the object_store crate)?
Thanks. A generic rule cannot distinguish an ABFS container from credential
userinfo.
We could preserve usernames for all schemes while excluding passwords, but
that would change existing behavior. Alternatively, we could move the key
derivation into the object_store crate. Which approach would you prefer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
