Haiyang Sun created SPARK-56284:
-----------------------------------
Summary: Key worker abstraction for language-agnostic UDF protocol
Key: SPARK-56284
URL: https://issues.apache.org/jira/browse/SPARK-56284
Project: Spark
Issue Type: Sub-task
Components: Connect, Spark Core
Affects Versions: 4.2.0
Reporter: Haiyang Sun
To enable language-agnostic UDF execution in Spark, a key step is to decouple
the Spark engine from language-specific worker runtimes and define a unified
interaction model between Spark and those workers.
As a first step, this ticket focuses on establishing the core abstractions for
describing worker specifications, as well as the primary interfaces Spark will
use to interact with workers.
This initial step will primarily introduce a new build target/module to host
these abstractions. Concrete implementations of the abstractions (e.g., for
specific languages or execution backends) will be introduced in follow-up tasks.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
