andygrove commented on PR #4459: URL: https://github.com/apache/datafusion-comet/pull/4459#issuecomment-5218624968
Filed the follow-ups I referred to, and the code comments now point at them rather than at review threads (1dc4e5c): - #5294 — scope the Rust UDF registry to the session that registered the UDF - #5295 — a Rust UDF silently answers calls to an ordinary Scala UDF registered under the same name (the `ignore`d test in `CometRustUdfSuite` is the reproduction; the issue records why the identity fix isn't available through Spark's public `udf` API and what the `FunctionRegistry` route looks like) - #5296 — the adapter rebuilds the kernel impl and re-resolves the return type on every batch - #5297 — the library cache holds its write lock across `dlopen`, including the `RwLock` poisoning consequence I also corrected the rationale in #5252 ([comment](https://github.com/apache/datafusion-comet/issues/5252#issuecomment-5218594456)): its description said the lock was defensive because DataFusion serializes invocations anyway, which is wrong, so removing the lock on that basis would introduce a data race. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
