gstvg commented on code in PR #21193:
URL: https://github.com/apache/datafusion/pull/21193#discussion_r3212663518
##########
datafusion/substrait/src/logical_plan/consumer/substrait_consumer.rs:
##########
@@ -481,6 +526,27 @@ pub trait SubstraitConsumer: Send + Sync + Sized {
};
substrait_err!("Missing handler for user-defined literals {}",
type_ref)
}
+
+ // Lambda related methods
+
+ /// Returns a new instance of this consumer which includes the given
`lambda_parameters` and the names they got assigned
+ ///
+ /// Note for custom implementations it's possible to embed a
[DefaultSubstraitLambdaConsumer] and forward this method to it
+ fn with_lambda_parameters(
+ &self,
+ lambda_parameters: &[Type],
+ input_schema: &DFSchema,
+ ) -> datafusion::common::Result<(Vec<String>, Self)>;
Review Comment:
I'll confirm with the maintainer who ends up reviewing this, but
`SubstraitConsumer` is both `Send + Sync` and all it's methods take `&self`.
Since the default consumer is cheap to create, and I expect most/all custom
ones to be cheap as well, I guess it's mostly due to async and to easily embed
it into other structures which should also implement `Send + Sync` than to
allow efficient concurrent usage.
I won't expect any consumer to be used concurrently *for performance*, but,
since *it can* be used, I think it's possible that it's/will be used
concurrently *incidentally* as the easier/natural way within a given codebase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
