branch: externals/minuet
commit 330804f9e947847f92d7883901963115e6d05930
Author: Milan Glacier <d...@milanglacier.com>
Commit: Milan Glacier <d...@milanglacier.com>
doc: update special tokens instruction for llama.cpp.
---
README.md | 2 +-
recipes.md | 10 +++++++++-
2 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/README.md b/README.md
index 46b7514686..f97ae24481 100644
--- a/README.md
+++ b/README.md
@@ -187,7 +187,7 @@ llama-server \
minuet-openai-fim-compatible-options
:prompt
(defun minuet-llama-cpp-fim-qwen-prompt-function (ctx)
- (format "<fim_prefix|>%s\n%s<|fim_suffix|>%s<|fim_middle|>"
+ (format "<|fim_prefix|>%s\n%s<|fim_suffix|>%s<|fim_middle|>"
(plist-get ctx :language-and-tab)
(plist-get ctx :before-cursor)
(plist-get ctx :after-cursor)))
diff --git a/recipes.md b/recipes.md
index 8e11099309..9b088d4c90 100644
--- a/recipes.md
+++ b/recipes.md
@@ -59,7 +59,7 @@ llama-server \
minuet-openai-fim-compatible-options
:prompt
(defun minuet-llama-cpp-fim-qwen-prompt-function (ctx)
- (format "<fim_prefix|>%s\n%s<|fim_suffix|>%s<|fim_middle|>"
+ (format "<|fim_prefix|>%s\n%s<|fim_suffix|>%s<|fim_middle|>"
(plist-get ctx :language-and-tab)
(plist-get ctx :before-cursor)
(plist-get ctx :after-cursor)))
@@ -68,6 +68,14 @@ llama-server \
(minuet-set-optional-options minuet-openai-fim-compatible-options
:max_tokens 56))
```
+> [!NOTE]
+> Symbols like `<|fim_begin|>` and `<|fim_suffix|>` are special tokens
+> that serve as prompt boundaries. Some LLMs, like Qwen2.5-Coder, have
+> been trained with specific tokens to better understand prompt
+> composition. Different LLMs use different special tokens during
+> training, so you should adjust these tokens according to your
+> preferred LLM.
+
## **Acknowledgment**
- [llama.vim](https://github.com/ggml-org/llama.vim): A reference for CLI
parameters used in launching the `llama.cpp` server.
