Beyond this, I believe a monorepo would be more friendly to coding agents by providing them with a more complete code context. This approach would significantly improve efficiency in cross-language implementations, such as porting features from Java to Rust. I view this transition as a proactive and necessary step for the Fluss project to move fast in the era of AI.
Additionally, if the community considers merging repositories a viable option, this initiative warrants a dedicated DISCUSS and vote thread. Such a change requires at least three binding votes from PPMC members, whereas a FIP only requires three binding votes from committers. Best, Jark On Mon, 9 Mar 2026 at 21:33, yuxia <[email protected]> wrote: > > Thanks David for start the discussion. > > I'm big +1 to use Rust to build the gateway for > - Performance: Rust avoids GC pauses, which is valuable for high-QPS gateway > workloads > - Ecosystem: Rust integrates naturally with Arrow/DataFusion/Flight SQL, and > has mature PostgreSQL protocol libraries. > - Industry momentum: More infrastructure projects are adopting Rust for > gateway layers (e.g., sglang-gateway, a Rust-based AI gateway) > > Also, as one the core maintainers of fluss-rust, I'm also +1 to move > fluss-rust to fluss main repo after the first release of fluss-rust for > - Faster feature delivery: It removes multi-repo release chaining and > shortens time-to-user for fixes and new features. > - More contributors: A unified repo can attract broader contributor > participation across Java/Rust/gateway areas. > - Better developer workflow: Developers can change Java + Rust + gateway code > in one PR, reducing coordination overhead and improving review efficiency. > > Best regards, > Yuxia > > ----- 原始邮件 ----- > 发件人: "Jark Wu" <[email protected]> > 收件人: "dev" <[email protected]> > 发送时间: 星期一, 2026年 3 月 09日 下午 8:50:08 > 主题: Re: [DISCUSS] FIP-32 Multiprotocol Query Gateway for Apache Fluss > > Hi David, > > I really appreciate this proposal. It represents a significant step > forward and opens up new possibilities for the Fluss project. > > I have uploaded the FIP markdown to the Apache cwiki to improve > readability and granted you edit permissions. You can access and > modify the document here: > https://cwiki.apache.org/confluence/display/FLUSS/FIP-32%3A+Multiprotocol+Query+Gateway+for+Apache+Fluss > > Overall, I think the proposal is well structured. Below are some > detailed comments for further consideration: > > **1. Repository Strategy** > I am a strong advocate for Rust and fully support leveraging > DataFusion and fluss-rust to build the gateway. Given that the gateway > will become a critical component of Fluss, handling lightweight query > execution tasks, adopting Rust and DataFusion from the outset provides > a solid foundation for performance and reliability. > > However, we need to carefully consider project dependency management. > The current dependency chain Fluss gateway depends on fluss-rust, > which in turn depends on fluss repo means that releasing a new feature > would require three separate repository releases. This could delay > feature availability to users by nearly a year, which is inefficient. > I suggest consolidating fluss-rust and fluss-gateway into the Fluss > main repository (we can do this after the first release of > fluss-rust). This approach would not only streamline the release > process but also encourage more contributors to participate in > Rust-related development and accelerate community iteration across the > Rust, CLI, and gateway ecosystems. > > **2. Separate into multiple FIPs > I recommend splitting this proposal into multiple FIPs. The primary > FIP should focus on the Gateway architecture, covering design > decisions related to dependencies, language choice, repository > structure, and essential features such as ACL support. Within this > Gateway FIP, we can reference REST, ADBC, and PostgreSQL protocols in > a high-level overview without subjecting them to the formal voting > process. Each protocol should then have its own dedicated FIP to > address protocol-specific complexities, including supported APIs, > implementation details, and versioning strategies. This modular > approach will make the review process more focused and manageable. > > **3. Configuration File Format** > Fluss currently uses YAML for configuration files, which ensures > consistency across different environments such as Flink SQL, shell > commands, and Docker Compose. To maintain this uniformity, I suggest > starting with YAML for the gateway configuration as well. This will > provide a consistent external API for users. We can consider > supporting additional formats like TOML or JSON in the future based on > community feedback. > > **4. Connection Parameter** > We should adopt the standard `bootstrap.servers` parameter for the > gateway to connect to the Fluss cluster, rather than using > `coordinator_address`. The coordinator address may change due to > failover or scaling events, whereas bootstrap servers provide a stable > entry point for client discovery and connection. > > **5. DataFusion as an Optional Component** > While DataFusion is a powerful query engine, it may not be the optimal > choice for all gateway use cases. Some scenarios, such as > high-frequency, lightweight lookup_kv or produce_log operations, could > experience performance bottlenecks due to DataFusion's planning and > conversion overhead. Furthermore, certain Fluss-specific APIs are not > supported by DataFusion. I propose making DataFusion an optional > library that gateway implementations can choose to integrate. This > design would also allow bypassing DataFusion entirely and calling the > fluss-rust client directly when appropriate, providing greater > flexibility and performance optimization opportunities. > > Please let me know your thoughts on these suggestions. I am happy to > discuss further and help refine the proposal. > > Best regards, > Jark > > On Sat, 7 Mar 2026 at 12:08, ForwardXu <[email protected]> wrote: > > > > Hi David, > > > > > > Thanks a lot for sharing the FIP-32 draft and George’s prototype work on > > the multiprotocol query gateway! > > > > > > Supporting multiple protocols (especially Flight SQL) via Apache DataFusion > > is a great direction for Fluss, and I fully support this initiative. > > > > > > I’ve checked the repo and have one quick question: will this gateway be > > designed to be pluggable, so that we can easily add new protocols in the > > future? > > > > > > Looking forward to further discussions, and I’m happy to help with any > > reviews or testing. > > > > > > Best, > > Forward > > > > 原始邮件 > > > > > > 发件人：David Reger <[email protected]> > > 发件时间：2026年3月7日 07:39 > > 收件人：dev <[email protected]> > > 主题：[DISCUSS] FIP-32 Multiprotocol Query Gateway for Apache Fluss > > > > > > > > Hey together, > > > > Thanks to George we have already a nice draft of a query gateway using Apache Datafusion du support multiple protocols especially Flight SQL. > > > > His Repo is here https://github.com/gstamatakis95/fluss/tree/feature/query-gateway. > > > > Based on his work I created a FIP to further discuss how the query gateway fits into the project and how we want to integrate and extend it further. > > > > You can find the FIP Draft here https://drive.google.com/file/d/1CC_uSQjtQmlMNGZyyHXr9K162BpKZvZO/view?usp=drivesdk > > > > Thank you and I appreciate every feedback. > > > > Kind regards > > David
