xudong963 commented on code in PR #21820:
URL: https://github.com/apache/datafusion/pull/21820#discussion_r3653988167
##########
datafusion/common/src/hash_utils.rs:
##########
@@ -186,13 +188,32 @@ where
}).map_err(|_| _internal_datafusion_err!("with_hashes cannot access
thread-local storage during or after thread destruction"))?
}
+/// Creates hashes for the given arrays using a thread-local buffer and a
custom
+/// hash builder, then calls the provided callback with the computed hashes.
+///
+/// Hash compatibility with [`with_hashes`] follows the rules documented on
+/// [`create_hashes_with_hasher`].
+pub fn with_hashes_with_hasher<I, T, F, R, S>(
+ arrays: I,
+ hash_builder: &S,
+ callback: F,
+) -> Result<R>
+where
+ I: IntoIterator<Item = T>,
+ T: AsDynArray,
+ F: FnOnce(&[u64]) -> Result<R>,
+ S: BuildHasher,
+{
+ build_hasher::with_hashes_with_hasher(arrays, hash_builder, callback)
Review Comment:
`with_hashes` is an existing API and cannot directly delegate to
`with_hashes_with_hasher`: its `HashState` path uses seeded rehashing for some
multi-column values, so delegation would change existing hash results and
bypass the optimized path. However, the thread-local buffer management is
duplicated. I’ll extract that part into a small private helper while keeping
the two hashing implementations separate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
