hengyuss opened a new issue, #7382:
URL: https://github.com/apache/shenyu/issues/7382
# [Task][AI Proxy] Implement the extensible AI Protocol layer
Parent issue: #7351
Depends on: #7381
## Background
The current AI Proxy supports OpenAI Chat Completions and relies on Spring AI
types to represent protocol requests and responses. This limits protocol
extension to the models and behavior provided by Spring AI.
The Protocol layer should understand AI protocol semantics without depending
on a specific LLM provider or transport implementation.
## Goal
Implement the `ShenyuAiProtocol` extension mechanism and migrate OpenAI Chat
Completions protocol handling to it.
## Responsibilities
The Protocol layer is responsible for:
- identifying and parsing requests and responses;
- converting between protocol data and ShenYu's internal models;
- handling normal and streaming responses;
- extracting text, model information, finish reason, and token usage;
- encoding responses in the client protocol format;
- preserving unknown fields through the internal raw payload or extensions.
## Initial Implementation
Implement:
- `openai-chat`
The design should allow future implementations such as:
- `openai-responses`;
- `anthropic-messages`;
- `openai-embeddings`.
## Boundaries
The Protocol layer must not:
- select an upstream provider;
- apply provider authentication;
- handle provider-specific endpoint or field differences;
- open HTTP connections;
- implement retry or fallback policy.
## Acceptance Criteria
- [ ] OpenAI Chat Completions is implemented through `ShenyuAiProtocol`.
- [ ] The implementation does not depend on Spring AI DTOs or clients.
- [ ] Normal requests and responses can be decoded and encoded.
- [ ] Streaming events can be decoded and encoded without buffering the full
response.
- [ ] Text, model information, finish reason, and token usage can be
extracted.
- [ ] Unknown request and response fields are preserved.
- [ ] Protocol errors can be represented using the shared error model.
- [ ] Adding another Protocol does not require changing Provider or
Transport.
- [ ] Unit tests cover normal responses, streaming responses, usage
extraction,
error responses, and unknown-field preservation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
