Revolyssup opened a new pull request, #12565:
URL: https://github.com/apache/apisix/pull/12565
```bash
❯ curl http://127.0.0.1:9080/v1/chat/completions -XPOST -H "Content-Type:
application/json" -d '{
"messages": [
{ "role": "system", "content": "You are a mathematician" },
{ "role": "user", "a": 1, "content": "Correct the spelling in
lostu" }
], "stream": false
}' | jq
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left Speed
100 1442 0 1231 100 211 212 36 0:00:05 0:00:05 --:--:--
324
{
"choices": [
{
"content_filter_results": {
"hate": {
"filtered": false,
"severity": "safe"
},
"protected_material_code": {
"filtered": false,
"detected": false
},
"protected_material_text": {
"filtered": false,
"detected": false
},
"self_harm": {
"filtered": false,
"severity": "safe"
},
"sexual": {
"filtered": false,
"severity": "safe"
},
"violence": {
"filtered": false,
"severity": "safe"
}
},
"finish_reason": "stop",
"index": 0,
"logprobs": null,
"message": {
"content": "It looks like you meant to write \"lost.\"",
"refusal": null,
"role": "assistant"
}
}
],
"created": 1745912572,
"id": "chatcmpl-BRZu83059FcmuuMYD8mqdDk9kJ4Yv",
"model": "gpt-4o-2024-05-13",
"object": "chat.completion",
"prompt_filter_results": [
{
"prompt_index": 0,
"content_filter_results": {
"hate": {
"filtered": false,
"severity": "safe"
},
"jailbreak": {
"filtered": false,
"detected": false
},
"self_harm": {
"filtered": false,
"severity": "safe"
},
"sexual": {
"filtered": false,
"severity": "safe"
},
"violence": {
"filtered": false,
"severity": "safe"
}
}
}
],
"system_fingerprint": "fp_ee1d74bde0",
"usage": {
"completion_tokens": 11,
"completion_tokens_details": {
"accepted_prediction_tokens": 0,
"audio_tokens": 0,
"reasoning_tokens": 0,
"rejected_prediction_tokens": 0
},
"prompt_tokens": 22,
"prompt_tokens_details": {
"audio_tokens": 0,
"cached_tokens": 0
},
"total_tokens": 33
}
}
❯ echo $body
{
"id": "ai-proxy-multi-route",
"uri": "/v1/chat/completions",
"methods": ["POST"],
"plugins": {
"ai-proxy-multi": {
"instances": [
{
"name": "azure-openai",
"provider": "azure-openai",
"weight": 100,
"auth": {
"header": {
"Authorization":
"Bearer
9m7VYroxITMDqOk0y1DEDEqKKEnpOknn1rV7QNQT7DrIBApcwMLYJQQJ99ALACYeBjFXJ3w3AAABACOGXGcd"
}
},
"options": {
"model": "gpt-4o",
"max_tokens": 512,
"temperature": 1.0,
"stream": false
}, "override":
{"endpoint":"https://ai-plugin-developer.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2025-01-01-preview"}
}
],
"logging": {
"summaries": true,
"payloads": true
}
},
"file-logger": {
"path": "logs/file2.log"
}
}
}
```
### Checklist
- [ ] I have explained the need for this PR and the problem it solves
- [ ] I have explained the changes or the new features added to this PR
- [ ] I have added tests corresponding to this change
- [ ] I have updated the documentation to reflect this change
- [ ] I have verified that this change is backward compatible (If not,
please discuss on the [APISIX mailing
list](https://github.com/apache/apisix/tree/master#community) first)
<!--
Note
1. Mark the PR as draft until it's ready to be reviewed.
2. Always add/update tests for any changes unless you have a good reason.
3. Always update the documentation to reflect the changes made in the PR.
4. Make a new commit to resolve conversations instead of `push -f`.
5. To resolve merge conflicts, merge master instead of rebasing.
6. Use "request review" to notify the reviewer after making changes.
7. Only a reviewer can mark a conversation as resolved.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
