branch: externals/minuet
commit face48fd9a6054217c52f80b9cd384c21706b930
Author: Milan Glacier <d...@milanglacier.com>
Commit: Milan Glacier <d...@milanglacier.com>
docs: add experimental Gemini configuration setup
Add documentation for an experimental prompt configuration specific to
Gemini LLM that uses a Prefix-Suffix structure. The new configuration
suggests using Before-Cursor -> Cursor-Pos -> After-Cursor order, which
appears to perform better than the standard approach used for other
chat-based LLMs
---
README.md | 8 +++++
prompt.md | 119 +++++++++++++++++++++++++++++++++++++++++++++-----------------
2 files changed, 95 insertions(+), 32 deletions(-)
diff --git a/README.md b/README.md
index 7763b0f57e..b5a1945142 100644
--- a/README.md
+++ b/README.md
@@ -413,6 +413,14 @@ settings following the example:
:threshold "BLOCK_NONE")])
```
+### Experimental Configuration
+
+Gemini appears to perform better with an alternative input structure, unlike
+other chat-based LLMs. This observation is currently experimental and requires
+further validation. For details on the experimental prompt setup currently in
+use by the maintainer, please refer to the
+[prompt
documentation](./prompt.md#an-experimental-configuration-setup-for-gemini).
+
</details>
## OpenAI-compatible
diff --git a/prompt.md b/prompt.md
index f4bc81326f..63e06975ce 100644
--- a/prompt.md
+++ b/prompt.md
@@ -1,3 +1,14 @@
+- [FIM LLM Prompt Structure](#fim-llm-prompt-structure)
+- [Chat LLM Prompt Structure](#chat-llm-prompt-structure)
+ - [Default Template](#default-template)
+ - [Default Prompt](#default-prompt)
+ - [Default Guidelines](#default-guidelines)
+ - [Default `:n_completions` template](#default---n-completions--template)
+ - [Default Few Shots Examples](#default-few-shots-examples)
+ - [Default Chat Input Example](#default-chat-input-example)
+ - [Customization](#customization)
+ - [An Experimental Configuration Setup for
Gemini](#an-experimental-configuration-setup-for-gemini)
+
# FIM LLM Prompt Structure
The prompt sent to the FIM LLM follows this structure:
@@ -119,9 +130,9 @@ Components:
- `:context-after-cursor`: Contains the text content following the cursor
position
-Implementation requires each component to be defined by a function
-that accepts a single parameter `context` and returns a string. This
-context parameter is a plist containing the following values:
+Implementation requires each component to be defined by a function that accepts
+a single parameter `context` and returns a string. This context parameter is a
+plist containing the following values:
- `:before-cursor`
- `:after-cursor`
@@ -155,35 +166,7 @@ Note that `:n_completion_template` is a special
placeholder as it contains one
this template, make sure your prompt also contains only one `%d`.
Similarly, `:fewshots` can be a plist in the following form or a function that
-takes no argument and returns a plist in the following form:
-
-```lisp
-`((:role "user"
- :content "# language: python
-<contextAfterCursor>
-
-fib(5)
-<contextBeforeCursor>
-def fibonacci(n):
- <cursorPosition>")
- (:role "assistant"
- :content " '''
- Recursive Fibonacci implementation
- '''
- if n < 2:
- return n
- return fib(n - 1) + fib(n - 2)
-<endCompletion>
- '''
- Iterative Fibonacci implementation
- '''
- a, b = 0, 1
- for _ in range(n):
- a, b = b, a + b
- return a
-<endCompletion>
-"))
-```
+takes no argument and returns a plist.
Below is an example to configure the prompt based on major mode:
@@ -217,3 +200,75 @@ function fibonacci(n) {
(plist-put minuet-openai-options :fewshots #'my-minuet-few-shots)
```
+
+## An Experimental Configuration Setup for Gemini
+
+Some observations suggest that Gemini might perform better with a
+`Prefix-Suffix` structured input format, specifically
+`Before-Cursor -> Cursor-Pos -> After-Cursor`.
+
+This contrasts with other chat-based LLMs, which may yield better results with
+the inverse structure: `After-Cursor -> Before-Cursor -> Cursor-Pos`.
+
+This finding remains experimental and requires further validation.
+
+Below is the current configuration used by the maintainer for Gemini:
+
+```lisp
+(use-package minuet
+ :config
+ (setq minuet-provider 'gemini)
+
+ (defvar mg-minuet-gemini-prompt
+ "You are the backend of an AI-powered code completion engine. Your
task is to
+provide code suggestions based on the user's input. The user's code will be
+enclosed in markers:
+
+- `<contextAfterCursor>`: Code context after the cursor
+- `<cursorPosition>`: Current cursor location
+- `<contextBeforeCursor>`: Code context before the cursor
+")
+
+ (defvar mg-minuet-gemini-chat-input-template
+ "{{{:language-and-tab}}}
+<contextBeforeCursor>
+{{{:context-before-cursor}}}<cursorPosition>
+<contextAfterCursor>
+{{{:context-after-cursor}}}")
+
+ (defvar mg-minuet-gemini-fewshots
+ `((:role "user"
+ :content "# language: python
+<contextBeforeCursor>
+def fibonacci(n):
+ <cursorPosition>
+<contextAfterCursor>
+
+fib(5)")
+ ,(cadr minuet-default-fewshots)))
+
+ (minuet-set-optional-options minuet-gemini-options
+ :prompt 'mg-minuet-gemini-prompt
+ :system)
+ (minuet-set-optional-options minuet-gemini-options
+ :template
'mg-minuet-gemini-chat-input-template
+ :chat-input)
+ (plist-put minuet-gemini-options :fewshots 'mg-minuet-gemini-fewshots)
+
+ (minuet-set-optional-options minuet-gemini-options
+ :generationConfig
+ '(:maxOutputTokens 256
+ :topP 0.9))
+ (minuet-set-optional-options minuet-gemini-options
+ :safetySettings
+ [(:category "HARM_CATEGORY_DANGEROUS_CONTENT"
+ :threshold "BLOCK_NONE")
+ (:category "HARM_CATEGORY_HATE_SPEECH"
+ :threshold "BLOCK_NONE")
+ (:category "HARM_CATEGORY_HARASSMENT"
+ :threshold "BLOCK_NONE")
+ (:category "HARM_CATEGORY_SEXUALLY_EXPLICIT"
+ :threshold "BLOCK_NONE")])
+
+ )
+```
