Let me voice a somewhat-alternative idea: For Calcite-like tools, you essentially have Calcite in Java, and DataFusion, in Rust. The ecosystem of adjacent tooling usually either supports one or the other.
For example -- at my workplace, when they rebuilt our core platform, they adopted Rust and DataFusion for our query engine. (We also evaluated using Substrait as our Query Plan IR, but it's both fairly verbose and lacked some of the nodes we'd need. Plus, being tied to Protobuf (we use JSON) wasn't ideal.) Personally, I feel like there'd be more value in reaching out to Andy (and maybe other owners of tools like Trino, etc) to see whether a Common Interface for these query engines could be agreed upon. Language-agnostic integration with a platform of Query Engine-like tools seems like it would unlock a lot of value. This is quite a large endeavor, though, and potentially not possible. Can I ask: Suppose Calcite had an API defined, something like "class MyLanguageEngine implements CalciteImplementation" Do folks think implementations in other languages will pop up? It seems like quite a hefty/herculean task, unless you only ported parts of it. On Sun, Feb 22, 2026 at 5:48 AM jensen <[email protected]> wrote: > I personally strongly support this idea! It would allow us to fully > leverage Calcite’s strengths in SQL parsing and optimization. I think > Substrait is a relatively good reference at this stage (though there might > be even better alternatives). Currently, substrait-java already uses > Calcite as the reference implementation of the standard, which is a great > starting point. > I’ve previously worked on using Substrait to integrate with third-party > execution engines. As Alessandro mentioned, execution layers vary > significantly, making integration extremely challenging—even if you only > aim to maintain compatibility with just two different execution backends. > Third-party operators often differ substantially from Calcite’s model, so > the primary task would likely be defining a unified specification. > However, should this specification be developed by referencing both > Calcite and several recommended execution engines to ensure broad > applicability? > > > > Best regards, > > Zhen Chen > > ---- Replied Message ---- > | From | Alessandro Solimando<[email protected]> | > | Date | 2/22/2026 17:55 | > | To | <[email protected]> | > | Subject | [DISCUSS] Single specification, multiple implementations | > Following the very interesting take of Julian in [1], I'd like to start a > brainstorming discussion on the opportunity of turning Calcite into a > specification-first project, with the aim of providing multiple > implementations, and what that would look like. > > I am supportive of the initiative, as I am seeing the same trends and > discussions Julian mentions, projects like Substrait allow using Calcite > as-a-service, but lowering the barrier for new database projects in other > languages would surely help adoption and the project to stay relevant in > the future. > > I have witnessed successful rewrites via LLMs as snapshots at time T, but I > don't have any experience in maintaining multiple versions over time, which > is probably the biggest challenge. > > The first step Julian suggests is to turn spec and tests into a language > agnostic format (like Quidem), so that we have both a "description" of what > we want, and a way to verify derived implementations, and I can't agree > more. > > New features and bugs in the specification would be handled as changes to > the specification and tests, which are shared, but how would we ensure > "consistency" for implementation-specific concepts? > > Since those concepts are per-implementation, the risk of drift from the > "snapshot rewrite at T" might become an issue. > > I am not talking about lower-level consistency, as that is impossible to > achieve across radically different languages, I am talking about conceptual > consistency and the capability to match higher-level concepts and > interfaces (e.g., the metadata provider), across different implementations. > > I wonder if a protobuf specification of the internal representation of > Calcite would be a good tool to keep the different implementations from > drifting. > > If we move to multiple implementations, we also need to ensure that the > burden on maintainers doesn't double, as this is already a problem we face. > A "spec-first" approach should ideally automate the validation of all > implementations whenever a shared test changes. > > There are many examples of projects with multiple implementations/bindings > (Substrait, Apache Arrow, to name a few projects which are spec-heavy), so > maybe looking at what those communities do could be useful. > > Looking forward to hearing your thoughts and ideas! > > Best regards, > Alessandro >
