feat(llamacpp) scripts to run gemma4 and ornihth-1.0
:Release Notes:
- placed in llama-cpp/ folder
- several changes in opencode browser-butler agent file
- added bash exec capability to provide with current date
functionality
- added local models to opencode local configuration
:Detailed Notes:
-
:Testing Performed:
- opencode starting up with changed files
- local models are pulled and started up
:QA Notes:
-
:Issues Addressed:
-
This commit is contained in:
parent
71833d83d8
commit
23d6cb6c30
@ -7,14 +7,13 @@ permission:
|
|||||||
glob: allow
|
glob: allow
|
||||||
grep: allow
|
grep: allow
|
||||||
list: deny
|
list: deny
|
||||||
bash: deny
|
bash: allow
|
||||||
task: deny
|
task: deny
|
||||||
todowrite: deny
|
todowrite: deny
|
||||||
question: allow
|
question: allow
|
||||||
webfetch: allow
|
webfetch: allow
|
||||||
websearch: allow
|
websearch: allow
|
||||||
skill: allow
|
skill: allow
|
||||||
doom_loop: deny
|
|
||||||
external_directory: deny
|
external_directory: deny
|
||||||
lsp: deny
|
lsp: deny
|
||||||
"chrome-devtools_*": allow
|
"chrome-devtools_*": allow
|
||||||
@ -29,13 +28,12 @@ You may use ONLY the following tools. Any other tool is forbidden and there are
|
|||||||
1. **`chrome-devtools_*`** — all tools from the Chrome DevTools MCP server (navigation, snapshots, clicks, fills, forms, screenshots, console reads,
|
1. **`chrome-devtools_*`** — all tools from the Chrome DevTools MCP server (navigation, snapshots, clicks, fills, forms, screenshots, console reads,
|
||||||
performance traces, heap snapshots, lighthouse, evaluate_script, etc.).
|
performance traces, heap snapshots, lighthouse, evaluate_script, etc.).
|
||||||
2. **`websearch`** — for resolving unknowns the browser cannot answer itself.
|
2. **`websearch`** — for resolving unknowns the browser cannot answer itself.
|
||||||
3. **`webfetch`** — to read a URL's content as markdown when a snapshot is
|
3. **`webfetch`** — to read a URL's content as markdown when the page is not already open in the browser.
|
||||||
overkill or the page is not already open in the browser.
|
4. **`question`** — to ask the user for clarification when the request is genuinely ambiguous.
|
||||||
4. **`question`** — to ask the user for clarification when the request is
|
|
||||||
genuinely ambiguous.
|
|
||||||
5. **`skill`** — to load any opencode skill that is available.
|
5. **`skill`** — to load any opencode skill that is available.
|
||||||
|
6. **`bash`** — for read-only commands, like current date.
|
||||||
|
|
||||||
Do NOT attempt to use `edit`, `bash`, `task`, `list`, `todowrite`, `lsp`. They are disabled at the
|
Do NOT attempt to use `edit`, `task`, `list`, `todowrite`, `lsp`. They are disabled at the
|
||||||
permission layer. Treat filesystem operations as out of scope — if the user
|
permission layer. Treat filesystem operations as out of scope — if the user
|
||||||
asks for one, tell them to use a different agent.
|
asks for one, tell them to use a different agent.
|
||||||
|
|
||||||
@ -209,10 +207,7 @@ default would risk doing the wrong thing. Examples:
|
|||||||
- A destructive action (payment, deletion, send) is one click away — confirm
|
- A destructive action (payment, deletion, send) is one click away — confirm
|
||||||
before submitting.
|
before submitting.
|
||||||
|
|
||||||
Do NOT ask a question for things you can reasonably infer: if the user says
|
Do NOT ask a question for things you can reasonably infer.
|
||||||
"watch Сокровище Нации" and only one full film with that title exists, open it.
|
|
||||||
If a form field is obviously derivable (today's date, a default city, the
|
|
||||||
named item in the request), fill it.
|
|
||||||
|
|
||||||
## When to use websearch vs. in-app search
|
## When to use websearch vs. in-app search
|
||||||
|
|
||||||
@ -226,15 +221,14 @@ Prefer **websearch** when:
|
|||||||
the movie with the guy" — find the song title first, then drive the music
|
the movie with the guy" — find the song title first, then drive the music
|
||||||
app).
|
app).
|
||||||
- You need to know whether an item exists on a given app before committing to
|
- You need to know whether an item exists on a given app before committing to
|
||||||
navigating there ("is Сокровище Нации on rutube?").
|
navigating there ("is this movie available on this platform?").
|
||||||
- The latest information matters (today's match result, a news headline, a
|
- The latest information matters (today's news headline, a
|
||||||
current price) and the app's own search is not date-aware.
|
current price) and the app's own search is not date-aware.
|
||||||
- The user is asking a factual question ("what's the capital of X",
|
- The user is asking a factual question ("what's the capital of X",
|
||||||
"who won the match today") that a search engine answers directly.
|
"who won the match today") that a search engine answers directly.
|
||||||
|
|
||||||
Use `webfetch` only to read a specific URL's content as markdown — for
|
Use `webfetch` only to read a specific URL's content as markdown — for
|
||||||
example, to read a reference doc when you need to recall exact API parameters,
|
example, to read a reference doc when you need to recall exact API parameters.
|
||||||
or to read a search-result page's content without opening it in the browser.
|
|
||||||
Do not use `webfetch` as a search engine; that is what `websearch` is for.
|
Do not use `webfetch` as a search engine; that is what `websearch` is for.
|
||||||
|
|
||||||
## Concurrency and ordering
|
## Concurrency and ordering
|
||||||
|
|||||||
@ -5,5 +5,31 @@
|
|||||||
"type": "local",
|
"type": "local",
|
||||||
"command": ["npx", "-y", "chrome-devtools-mcp@latest", "--browser-url=http://127.0.0.1:9222"]
|
"command": ["npx", "-y", "chrome-devtools-mcp@latest", "--browser-url=http://127.0.0.1:9222"]
|
||||||
}
|
}
|
||||||
|
},
|
||||||
|
"provider": {
|
||||||
|
"llama.cpp": {
|
||||||
|
"npm": "@ai-sdk/openai-compatible",
|
||||||
|
"options": {
|
||||||
|
"baseURL": "http://localhost:8080/v1"
|
||||||
|
},
|
||||||
|
"models": {
|
||||||
|
"ornith-1.0": {
|
||||||
|
"tool_call": true,
|
||||||
|
"reasoning": true,
|
||||||
|
"limit": {
|
||||||
|
"context": 65536,
|
||||||
|
"output": 8192
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"gemma-4": {
|
||||||
|
"tool_call": true,
|
||||||
|
"reasoning": true,
|
||||||
|
"limit": {
|
||||||
|
"context": 131072,
|
||||||
|
"output": 8192
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
52
llama-cpp/run_gemma4.sh
Executable file
52
llama-cpp/run_gemma4.sh
Executable file
@ -0,0 +1,52 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
model_postfix="Q8_0"
|
||||||
|
model_addr="unsloth/gemma-4-12b-it-GGUF"
|
||||||
|
draft_mtp=""
|
||||||
|
quant=$1
|
||||||
|
cache_type="q8_0"
|
||||||
|
fa="on"
|
||||||
|
|
||||||
|
if [[ ${quant} = "q4_xl" ]]; then
|
||||||
|
model_postfix="UD-Q4_K_XL"
|
||||||
|
cache_type="q4_0"
|
||||||
|
model_addr="unsloth/gemma-4-12B-it-qat-GGUF"
|
||||||
|
draft_mtp="--spec-type draft-mtp --spec-draft-n-max 4"
|
||||||
|
elif [[ ${quant} = "q4" ]]; then
|
||||||
|
model_postfix="Q4_K_M"
|
||||||
|
cache_type="q4_0"
|
||||||
|
model_addr="unsloth/gemma-4-12B-it-GGUF"
|
||||||
|
elif [[ ${quant} = "q5" ]]; then
|
||||||
|
model_postfix="Q5_K_M"
|
||||||
|
cache_type="q5_0"
|
||||||
|
elif [[ ${quant} = "q8_small" ]]; then
|
||||||
|
model_postfix="Q8_0"
|
||||||
|
model_addr="unsloth/gemma-4-E4B-it-GGUF"
|
||||||
|
cache_type="q8_0"
|
||||||
|
fa="on"
|
||||||
|
elif [[ ${quant} = "q4_small" ]]; then
|
||||||
|
model_postfix="Q4_K_M"
|
||||||
|
model_addr="unsloth/gemma-4-E4B-it-GGUF"
|
||||||
|
cache_type="q4_0"
|
||||||
|
fa="on"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "cache_type:${cache_type}; model_postfix:${model_postfix}; draft_mtp:${draft_mtp}"
|
||||||
|
|
||||||
|
llama-server \
|
||||||
|
-hf ${model_addr}:${model_postfix} \
|
||||||
|
--ctx-size 131072 \
|
||||||
|
--alias gemma-4 \
|
||||||
|
--jinja \
|
||||||
|
${draft_mtp} \
|
||||||
|
-ngl 999 \
|
||||||
|
-fa ${fa} \
|
||||||
|
-b 1024 \
|
||||||
|
-ub 512 \
|
||||||
|
--no-ui \
|
||||||
|
--cache-ram 2048 \
|
||||||
|
--kv-unified \
|
||||||
|
--cache-type-k ${cache_type} \
|
||||||
|
--cache-type-v ${cache_type} \
|
||||||
|
--repeat-penalty 1.05 \
|
||||||
|
--host 0.0.0.0
|
||||||
19
llama-cpp/run_ornith.sh
Executable file
19
llama-cpp/run_ornith.sh
Executable file
@ -0,0 +1,19 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
|
||||||
|
llama-server \
|
||||||
|
-hf deepreinforce-ai/Ornith-1.0-9B-GGUF \
|
||||||
|
--ctx-size 65536 \
|
||||||
|
--alias ornith-1.0 \
|
||||||
|
--jinja \
|
||||||
|
--chat-template-file ~/.config/llama.cpp/ornith-toolfix.jinja \
|
||||||
|
-ngl 99 \
|
||||||
|
-fa on \
|
||||||
|
-b 2048 \
|
||||||
|
-ub 1024 \
|
||||||
|
--cache-ram 2048 \
|
||||||
|
--kv-unified \
|
||||||
|
--cache-type-k q8_0 \
|
||||||
|
--cache-type-v q8_0 \
|
||||||
|
--presence-penalty 1.5 \
|
||||||
|
--repeat-penalty 1.15 \
|
||||||
|
--host 0.0.0.0
|
||||||
Loading…
x
Reference in New Issue
Block a user