jayshrivastava commented on code in PR #21807:
URL: https://github.com/apache/datafusion/pull/21807#discussion_r3140807422
##########
datafusion/physical-expr/src/expressions/dynamic_filters.rs:
##########
@@ -76,16 +80,40 @@ pub struct DynamicFilterPhysicalExpr {
nullable: Arc<RwLock<Option<bool>>>,
}
-#[derive(Debug)]
-struct Inner {
+/// Atomic internal state of a [`DynamicFilterPhysicalExpr`].
+///
+/// **Warning:** exposed publicly solely so that proto (de)serialization in
+/// `datafusion-proto` can read and rebuild this state. Do not treat this type
+/// or its layout as a stable API.
+#[derive(Debug, Clone)]
+pub struct Inner {
/// A counter that gets incremented every time the expression is updated
so that we can track changes cheaply.
/// This is used for [`PhysicalExpr::snapshot_generation`] to have a cheap
check for changes.
- generation: u64,
- expr: Arc<dyn PhysicalExpr>,
+ pub generation: u64,
+ pub expr: Arc<dyn PhysicalExpr>,
/// Flag for quick synchronous check if filter is complete.
/// This is redundant with the watch channel state, but allows us to
return immediately
/// from `wait_complete()` without subscribing if already complete.
- is_complete: bool,
+ pub is_complete: bool,
+}
+
+// TODO: Include expression_id in debug output.
Review Comment:
I wanted to keep this PR small, so I hid the `expression_id` to avoid 1 test
that was failing. I have
[code](https://github.com/jayshrivastava/datafusion/pull/2) ready to address
this TODO which I plan to publish as a next step after this PR is merged. This
PR is just an incremental step.
```
// See https://github.com/apache/datafusion/issues/20418. Currently, plan
nodes
// like `HashJoinExec`, `AggregateExec`, `SortExec` do not serialize their
// dynamic filter. This causes round trips to fail on the `expression_id`
// because it is regenerated on deserialization.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
