akshaychitneni commented on PR #1568:
URL:
https://github.com/apache/datafusion-ballista/pull/1568#issuecomment-4308001358
> thanks for your contribution @akshaychitneni
>
> FWIW, I believe its possible to get cluster state notifications in current
implementation:
>
> ```rust
> let cluster = BallistaCluster::new_from_config(&config).await?;
>
> // this gets you a stream of `ClusterStateEvent`
> cluster.cluster_state().cluster_state_events()
> ```
>
> where `ClusterStateEvent` is defined as
>
> ```rust
> #[derive(Debug, Clone, PartialEq)]
> pub enum ClusterStateEvent {
> /// An executor has been registered with the cluster.
> RegisteredExecutor {
> /// ID of the registered executor.
> executor_id: String,
> },
> /// An executor has been removed from the cluster.
> RemovedExecutor {
> /// ID of the removed executor.
> executor_id: String,
> },
> }
> ```
>
> Having it as external stream gives possibility to run on different thread
than scheduler, having a listener on configuration might have better API.
>
> Let me know what you think
Thanks for pointing out. I think the listener approach might be helpful when
running in Kubernetes-like environments, as callbacks run inline in the
scheduler's event loop, avoiding silent event loss scenarios that can occur
with a separate consumer thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
