FreeOnePlus opened a new issue, #175:
URL: https://github.com/apache/doris-mcp-server/issues/175
# [Discussion] Defining the Boundary Between Doris MCP Server and
Semantic-Layer Runtimes
## Summary
The `new-mcp-server` branch introduces a valuable semantic-first direction.
It embeds MetricFlow, manages semantic models, exposes metric-oriented MCP
tools, and includes a Web-based model-management workflow. This work
demonstrates that Doris can support a practical semantic-query experience
through MCP.
Before adopting that branch as the main server implementation, we should
align on a more fundamental product question:
> What should Apache Doris MCP Server own, and what should remain an
integration behind the MCP boundary?
Our proposal for the 1.0.0 architecture is to keep Doris MCP Server as a
general, version-aware and provider-neutral capability gateway for Doris.
Semantic capabilities remain first-class, but semantic runtimes such as
MetricFlow should be integrated as providers behind the `doris_semantic` domain
rather than becoming the server core.
This discussion is not intended to reject the semantic work in
`new-mcp-server`. The goal is to preserve that value while keeping the project
boundary coherent, extensible, and suitable for the broader Doris ecosystem.
## Context
The two development directions currently emphasize different product scopes.
The 1.0.0 design treats Doris MCP Server as the northbound capability
interface between MCP Hosts and Doris. Its responsibilities include:
- stable and progressively disclosed tool discovery;
- compact, validated tool schemas;
- Doris version and runtime-capability detection;
- route-aware availability and explicit degraded states;
- authentication, authorization, auditing, and policy enforcement;
- safe and bounded query execution;
- transport and MCP protocol conformance;
- provider-neutral integration with Doris features and adjacent systems.
The `new-mcp-server` branch focuses on a semantic analytics workflow:
- embedded MetricFlow model compilation and query planning;
- metric and dimension discovery;
- semantic model storage in Doris;
- active and staging model lifecycles;
- model validation and reload operations;
- Web UI and CLI management experiences;
- raw Doris discovery and SQL as a fallback.
Both directions contain useful work. They represent different products,
however. The first is a general Doris capability gateway. The second is closer
to a MetricFlow-powered semantic query platform with an MCP interface.
## The Product Boundary We Propose
MCP defines how a Host discovers and calls capabilities. A production MCP
middleware product adds value by making those capabilities stable, safe,
observable, and portable across environments.
For Doris MCP Server, the durable product value should therefore remain
concentrated in:
1. **Capability discovery**
Hosts can determine which Doris capabilities exist, which are callable,
and why a capability is unavailable or degraded.
2. **Progressive disclosure**
A small top-level domain surface keeps Host context bounded while
allowing exact child capability discovery when needed.
3. **Version-aware behavior**
Doris features vary across versions, patch releases, deployment modes,
permissions, plugins, and configured endpoints. The server must report observed
availability instead of exposing optimistic static tools.
4. **Governed execution**
Authentication, route selection, authorization, result limits, query
policy, audit records, and error semantics should remain consistent across
every provider.
5. **Provider neutrality**
Doris-native capabilities, Ossie consumers, MetricFlow, external semantic
services, ADBC, and future integrations should fit behind stable contracts
without redefining the whole server.
A semantic layer has a different ownership boundary. It normally includes
semantic model authoring, metric graphs, compilation, validation, model
storage, publishing workflows, and runtime-specific query planning. An MCP
server can expose those capabilities, but the general Doris MCP core should not
have to become the semantic engine itself.
## Why the Distinction Matters
Embedding a complete semantic runtime directly into the MCP core creates
several long-term constraints.
### 1. The General Doris Surface Becomes Secondary
Doris users also need catalog inspection, query diagnostics, cluster health,
ingestion status, search capabilities, governance, lineage, lakehouse
inspection, Variant support, and runtime capability reporting.
A semantic-first server can serve one important analytics workflow, but it
does not represent the complete operational and analytical surface of Doris.
### 2. The Project Becomes Coupled to One Semantic Implementation
MetricFlow is a useful execution engine, but it is one implementation. Doris
users may use Ossie-compatible models, other semantic layers, external modeling
services, or future Doris-native semantic capabilities.
Making MetricFlow the server core would force the MCP project to inherit its
model assumptions, release lifecycle, dependency graph, and maintenance burden.
A provider boundary allows MetricFlow to remain fully supported without making
every user adopt it.
### 3. MCP Governance Can Be Bypassed by an Embedded Vertical Stack
Semantic compilation and SQL execution still need the same route selection,
permissions, result limits, auditing, timeout handling, and error contracts as
every other Doris query.
If a semantic runtime owns its own end-to-end execution path, those controls
can diverge from the existing Query domain. A shared provider contract keeps
semantic queries inside the same production boundary.
### 4. Management-Plane Concerns Expand the Core Product
Active/staging workspaces, model publishing, Web sessions, file editing, and
administrative UI are valid product capabilities. They belong to a semantic
control plane or companion application.
Placing them inside the default MCP runtime expands the server from a
capability gateway into a semantic development platform. That expansion affects
security, deployment, state management, multi-worker behavior, and release
responsibility.
## Proposed 1.0.0 Integration Model
The 1.0.0 architecture already defines `doris_semantic` as a top-level
read-only domain. We propose integrating MetricFlow behind that domain as an
optional semantic compilation provider.
```mermaid
flowchart TD
H["MCP Host / Agent"] --> M["Doris MCP Server"]
M --> S["doris_semantic"]
M --> Q["doris_query"]
S --> C["Semantic Model Consumer"]
C --> O["Ossie Adapter"]
C --> F["Other Model Adapters"]
S --> P["Semantic Provider Interface"]
P --> MF["MetricFlow Provider"]
P --> ES["External Semantic Service"]
P --> DN["Future Doris-Native Provider"]
MF --> Q
ES --> Q
DN --> Q
Q --> D["Apache Doris"]
```
The important boundaries are:
- `doris_semantic` remains the MCP domain visible to Hosts;
- Ossie is a semantic model contract and consumer adapter;
- MetricFlow is an optional semantic compilation provider;
- every semantic request uses an explicit `model_ref`;
- the server does not guess a model when multiple models are available;
- generated SQL or query plans return to the governed `doris_query`
execution path;
- provider availability is reported dynamically with structured reasons;
- MetricFlow does not register an independent global MCP tool surface;
- MetricFlow is not required for users who only need other Doris domains.
This structure can support the user experience explored by `new-mcp-server`
while preserving the wider 1.0.0 product architecture.
## How the `new-mcp-server` Work Can Be Reused
| Capability in `new-mcp-server` | Proposed 1.0.0 Home |
|---|---|
| MetricFlow Doris SQL rendering | `MetricFlowSemanticProvider` |
| Semantic model parsing and validation | Semantic provider or
model-consumer adapter |
| Metric and dimension discovery | `doris_semantic` model/context operations
|
| Semantic query compilation | MetricFlow provider behind `doris_semantic` |
| Query execution | Existing governed `doris_query` path |
| Raw database and table discovery | Existing `doris_catalog` domain |
| Raw SQL fallback | Existing `doris_query` domain |
| Provider health and readiness | Capability snapshot and structured
availability |
| Active/staging model lifecycle | Optional semantic control plane |
| Web UI and model editor | Companion application, not the default MCP core |
| Model reload or publishing | Future management capability, disabled from
the default read-only 1.0 surface |
This approach preserves the engineering investment without replacing
established MCP protocol, routing, security, availability, and domain-discovery
foundations.
## Proposed Scope for the MetricFlow Adapter
The first MetricFlow integration should be deliberately bounded:
- load or resolve a semantic model through an explicit `model_ref`;
- expose sanitized model, metric, dimension, and mapping metadata;
- validate whether the model can bind to the selected Doris route;
- compile a supported semantic request into a deterministic Doris query plan;
- return provider/version/availability evidence;
- delegate SQL execution to the Query domain;
- preserve the common authorization, audit, timeout, pagination, and
result-limit contracts.
The first adapter should not:
- replace the eight-domain MCP surface;
- make MetricFlow a mandatory runtime dependency;
- create an independent authentication or query-execution path;
- expose semantic model write operations in the default read-only surface;
- move Web UI session or model-publishing state into the MCP protocol core;
- infer a semantic model without an explicit `model_ref`.
## Relationship Between Ossie and MetricFlow
Ossie and MetricFlow solve different problems.
- Ossie defines a semantic model contract and interchange boundary.
- MetricFlow compiles and evaluates semantic models.
- Doris MCP Server exposes, governs, and routes the capabilities.
- Doris remains the underlying query and data platform.
An Ossie model may be consumed and translated into the canonical semantic
representation used by a MetricFlow provider. MetricFlow-native models may also
be supported through a separate adapter. The provider architecture should allow
both paths without treating either format or engine as the MCP core.
## Questions for Discussion
1. Do we agree that Apache Doris MCP Server should remain a general Doris
capability gateway rather than a semantic-only MCP product?
2. Should MetricFlow be integrated through an optional provider interface
under `doris_semantic`?
3. Should semantic query execution always return to the existing
`doris_query` governance boundary?
4. Should Ossie, MetricFlow-native manifests, and future semantic formats be
modeled as separate consumer adapters?
5. Should active/staging model management and the Web UI live in a companion
semantic control plane?
6. Which parts of `new-mcp-server` can be extracted with minimal coupling
and reused in the 1.0.0 implementation?
7. What compatibility and maintenance commitments would vendoring MetricFlow
into the Apache Doris MCP repository create?
## Proposed Direction
We propose continuing the 1.0.0 domain architecture and treating the
`new-mcp-server` branch as a valuable semantic implementation reference.
MetricFlow should become a supported sub-adapter/provider of
`doris_semantic`. It can supply model validation, semantic discovery, Doris SQL
compilation, and related runtime capabilities. The MCP core should continue to
own protocol conformance, progressive discovery, capability availability,
authorization, routing, auditing, and governed execution.
This division allows the project to deliver the semantic workflow
demonstrated by `new-mcp-server` without changing Apache Doris MCP Server into
a wrapper around one semantic product. It also leaves room for Ossie, external
semantic systems, and future Doris-native capabilities to participate through
the same stable integration boundary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
