sunchao commented on code in PR #5053:
URL: https://github.com/apache/datafusion-comet/pull/5053#discussion_r4043758567
##########
native/core/src/parquet/parquet_support.rs:
##########
@@ -1153,6 +1232,34 @@ mod tests {
object_store_cache().write().unwrap().remove(&key);
}
+ #[test]
+ fn isolates_azure_containers_in_cache_and_shared_runtime() {
+ use datafusion::execution::runtime_env::RuntimeEnvBuilder;
+
+ let options = HashMap::from([("fs.azure.account.key".into(),
"c2VjcmV0".into())]);
+ let runtime = RuntimeEnvBuilder::new()
+
.with_object_store_registry(Arc::new(super::CometObjectStoreRegistry::default()))
+ .build_arc()
+ .unwrap();
+ let register = |container| {
+ let (url, _, _) = prepare_object_store_with_configs(
+ Arc::clone(&runtime),
+
format!("abfss://{container}@account.dfs.core.windows.net/file.parquet"),
+ &options,
+ )
+ .unwrap();
+ (url.clone(), runtime.object_store(&url).unwrap())
+ };
+
+ let (url_a, store_a) = register("container-a");
+ let (same_url_a, same_store_a) = register("container-a");
+ let (url_b, store_b) = register("container-b");
+ assert_eq!(url_a, same_url_a);
+ assert!(Arc::ptr_eq(&store_a, &same_store_a));
+ assert_ne!(url_a, url_b);
+ assert!(!Arc::ptr_eq(&store_a, &store_b));
Review Comment:
**[P2] Resolve both URLs after registering both containers**
Could we look up `url_a` and `url_b` again after registering container B and
assert that each still resolves to its expected store? `register` captures A's
`Arc` before B is inserted, so this pointer comparison only compares handles
captured at different times. I ran all four existing assertions against
`DefaultObjectStoreRegistry`: they still passed for both `abfs` and `abfss`,
while subsequent lookups of A and B both returned B's store. The current
`CometObjectStoreRegistry` handles this correctly, but this regression test
would not detect replacing it with the default registry. Re-fetching both
stores after both registrations would cover the shared-runtime overwrite.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
