jxyedw opened a new issue, #253:
URL: https://github.com/apache/doris-mcp-server/issues/253
## Summary
When both optional semantic providers are enabled, `doris_semantic` domain
discovery fails with `DomainManifestBudgetError`.
The `doris_semantic` manifest contains all 12 children (4 Ossie + 8
MetricFlow). With both providers enabled, the canonical JSON manifest is
**16,747 bytes**, which is **363 bytes over** the hard-coded `16 KiB` domain
manifest budget.
As a result, Ossie grounding and MetricFlow query cannot be used from the
same server process, even though the 1.0 design (#175) and the public
documentation describe them as two parts of the same `doris_semantic` domain.
## Environment
- `doris-mcp-server`: `1.0.0` (tag; tested checkout `953c3b6`)
- MCP protocol: `2026-07-28`
- Transport: `stdio`
- Exposure mode: `hierarchical` (default); flat mode also reproduced
- Python: `3.12.13`
- Doris: `4.0.1-rc02` (FE/BE)
- Ossie model: 4 datasets / 7 metrics / 3 relationships,
`mapping_status=resolved`
- MetricFlow: `0.213.0` via the `doris-mcp-metricflow/v1` sidecar
- Auth: local stdio (no OAuth); the issue is about manifest size, not
authorization
## Steps to reproduce
1. Start the server with the Ossie provider enabled:
```bash
OSSIE_ENABLED=true
OSSIE_MODEL_DIRECTORY=<ossie-model-dir>
OSSIE_BINDING_MANIFEST=<ossie-binding-manifest>
METRICFLOW_ENABLED=false
```
2. Call `doris_semantic` with `{}`; discovery succeeds.
3. Restart with both providers enabled:
```bash
OSSIE_ENABLED=true
OSSIE_MODEL_DIRECTORY=<ossie-model-dir>
OSSIE_BINDING_MANIFEST=<ossie-binding-manifest>
METRICFLOW_ENABLED=true
METRICFLOW_PROVIDER_COMMAND_JSON=["<python>","<metricflow-provider.py>"]
METRICFLOW_PROJECT_DIRECTORY=<metricflow-project-dir>
```
4. Call `doris_semantic` with `{}` again. Discovery fails with
`DomainManifestBudgetError`.
The public error message is redacted, but the stack points to:
```text
doris_mcp_server/tools/domain_manifest.py:770
raise DomainManifestBudgetError(...)
```
In flat mode, `tools/list` fails earlier with the same error.
## Measured manifest sizes
The numbers below are canonical JSON sizes, measured as:
```python
len(envelope.to_canonical_json().encode("utf-8"))
```
| Configuration | Children | Manifest size | Result |
|---|---:|---:|---|
| MetricFlow only | 12 (4 Ossie `misconfigured`, 8 MF `degraded`) | **15,747
B** | passes, only 637 B headroom |
| Ossie only | 12 (4 Ossie `available/degraded`, 8 MF `misconfigured`) |
**13,041 B** | passes |
| Both providers enabled | 12 (all `available/degraded`) | **16,747 B** |
**fails, 363 B over the 16,384 B limit** |
For the both-enabled case, I could not capture the exact byte count from the
failing server because the error message is redacted. I reconstructed it by
taking the 8 MetricFlow child entries from the MetricFlow-only manifest and the
4 Ossie child entries from the Ossie-only manifest. The flat-mode run fails
with the same `DomainManifestBudgetError`.
### Where the extra ~1 KiB comes from
The 8 MetricFlow entries are identical in both single-provider runs. The
growth comes from the 4 Ossie entries changing from `misconfigured` to
`available/degraded`:
| Ossie child | MetricFlow-only (`misconfigured`) | Both enabled
(`available/degraded`) | Delta |
|---|---:|---:|---:|
| `list_semantic_models` | 751 B | 884 B | +133 B |
| `get_semantic_model_summary` | 818 B | 1,093 B | +275 B |
| `get_semantic_context` | 1,230 B | 1,519 B | +289 B |
| `get_semantic_mapping_status` | 833 B | 1,136 B | +303 B |
| **Total** | | | **+1,000 B** |
The extra bytes are `availability` metadata (`detected_versions`,
`evidence_sources`, `limitations`, `active_variant`) plus the dynamic
description prefix.
## Root cause
- `MAX_DOMAIN_MANIFEST_BYTES = 16 * 1024` is hard-coded in
`doris_mcp_server/tools/domain_models.py`.
- `DomainDefinition.max_manifest_bytes` is constrained to `<=
MAX_DOMAIN_MANIFEST_BYTES`, so an operator cannot raise it.
- `domain_manifest.py::_build_manifest` raises `DomainManifestBudgetError`
when the serialized envelope exceeds the budget.
- `doris_semantic` has 12 children, which is the maximum allowed by
`DomainDefinition.children` (`Field(max_length=12)`).
- All 12 children are always included; children from a disabled provider are
marked `misconfigured` and remain visible.
- There is no pagination, lazy child schema, or partial manifest. One
oversized entry fails the whole domain.
This is a capacity/packaging issue: the domain is at the maximum child
count, and enabling both optional providers adds enough `availability` metadata
to exceed the budget.
## Impact
- Ossie and MetricFlow cannot be enabled together in one server process;
operators must run two profiles/processes and callers need two endpoints.
- This contradicts the design intent in #175 (`doris_semantic` remains the
domain; Ossie is the model contract; MetricFlow is an optional provider; both
paths are supported).
- It also conflicts with the documented behavior that optional integrations
"stay visible but fail closed when called": here the whole domain fails at
discovery, before the caller can see or call the available children.
- The MetricFlow-only case has only **637 B** headroom, so it is fragile as
well: a small future schema/availability addition can break the single-provider
profile too.
## Suggested fixes
1. **Split the domain (preferred)**
- `doris_semantic_metricflow` (8 children, ~7.4 KiB)
- `doris_semantic_ossie` (4 children, ~3.3 KiB)
- This keeps each manifest comfortably within the budget and makes the
provider boundary explicit.
2. **Make the per-domain manifest budget configurable**
- Default remains 16 KiB for safety.
- Allow trusted deployments to raise it (for example, up to 64 KiB) with
explicit validation and documentation.
- This is the smallest code change, but it needs a decision on the
security/DoS bound.
3. **Add manifest pagination or lazy child schema loading**
- This is a larger protocol change and probably out of scope for 1.0.x.
## Additional context
- The public documentation does not mention the 16 KiB number; it only says
that domain discovery has "manifest byte budgets"
(`docs/operations/reliability.md`). The 16 KiB value is a source constant.
- `master` still contains `MAX_DOMAIN_MANIFEST_BYTES = 16 * 1024` as of
2026-11-09.
- Flat mode is not a workaround: the 12 semantic flat tools alone serialize
to ~35.9 KiB, and `MAX_TOP_LEVEL_TOOL_LIST_BYTES` is 24 KiB.
Could maintainers confirm whether this is a known limitation and which fix
direction is acceptable? I can provide a minimal reproduction harness or
sanitized model/binding files if useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
