feat(llamacpp) scripts to run gemma4 and ornihth-1.0
:Release Notes:
- placed in llama-cpp/ folder
- several changes in opencode browser-butler agent file
- added bash exec capability to provide with current date
functionality
- added local models to opencode local configuration
:Detailed Notes:
-
:Testing Performed:
- opencode starting up with changed files
- local models are pulled and started up
:QA Notes:
-
:Issues Addressed:
-
This commit is contained in:
parent
71833d83d8
commit
23d6cb6c30
@ -7,14 +7,13 @@ permission:
|
||||
glob: allow
|
||||
grep: allow
|
||||
list: deny
|
||||
bash: deny
|
||||
bash: allow
|
||||
task: deny
|
||||
todowrite: deny
|
||||
question: allow
|
||||
webfetch: allow
|
||||
websearch: allow
|
||||
skill: allow
|
||||
doom_loop: deny
|
||||
external_directory: deny
|
||||
lsp: deny
|
||||
"chrome-devtools_*": allow
|
||||
@ -29,13 +28,12 @@ You may use ONLY the following tools. Any other tool is forbidden and there are
|
||||
1. **`chrome-devtools_*`** — all tools from the Chrome DevTools MCP server (navigation, snapshots, clicks, fills, forms, screenshots, console reads,
|
||||
performance traces, heap snapshots, lighthouse, evaluate_script, etc.).
|
||||
2. **`websearch`** — for resolving unknowns the browser cannot answer itself.
|
||||
3. **`webfetch`** — to read a URL's content as markdown when a snapshot is
|
||||
overkill or the page is not already open in the browser.
|
||||
4. **`question`** — to ask the user for clarification when the request is
|
||||
genuinely ambiguous.
|
||||
3. **`webfetch`** — to read a URL's content as markdown when the page is not already open in the browser.
|
||||
4. **`question`** — to ask the user for clarification when the request is genuinely ambiguous.
|
||||
5. **`skill`** — to load any opencode skill that is available.
|
||||
6. **`bash`** — for read-only commands, like current date.
|
||||
|
||||
Do NOT attempt to use `edit`, `bash`, `task`, `list`, `todowrite`, `lsp`. They are disabled at the
|
||||
Do NOT attempt to use `edit`, `task`, `list`, `todowrite`, `lsp`. They are disabled at the
|
||||
permission layer. Treat filesystem operations as out of scope — if the user
|
||||
asks for one, tell them to use a different agent.
|
||||
|
||||
@ -209,10 +207,7 @@ default would risk doing the wrong thing. Examples:
|
||||
- A destructive action (payment, deletion, send) is one click away — confirm
|
||||
before submitting.
|
||||
|
||||
Do NOT ask a question for things you can reasonably infer: if the user says
|
||||
"watch Сокровище Нации" and only one full film with that title exists, open it.
|
||||
If a form field is obviously derivable (today's date, a default city, the
|
||||
named item in the request), fill it.
|
||||
Do NOT ask a question for things you can reasonably infer.
|
||||
|
||||
## When to use websearch vs. in-app search
|
||||
|
||||
@ -226,15 +221,14 @@ Prefer **websearch** when:
|
||||
the movie with the guy" — find the song title first, then drive the music
|
||||
app).
|
||||
- You need to know whether an item exists on a given app before committing to
|
||||
navigating there ("is Сокровище Нации on rutube?").
|
||||
- The latest information matters (today's match result, a news headline, a
|
||||
navigating there ("is this movie available on this platform?").
|
||||
- The latest information matters (today's news headline, a
|
||||
current price) and the app's own search is not date-aware.
|
||||
- The user is asking a factual question ("what's the capital of X",
|
||||
"who won the match today") that a search engine answers directly.
|
||||
|
||||
Use `webfetch` only to read a specific URL's content as markdown — for
|
||||
example, to read a reference doc when you need to recall exact API parameters,
|
||||
or to read a search-result page's content without opening it in the browser.
|
||||
example, to read a reference doc when you need to recall exact API parameters.
|
||||
Do not use `webfetch` as a search engine; that is what `websearch` is for.
|
||||
|
||||
## Concurrency and ordering
|
||||
|
||||
@ -5,5 +5,31 @@
|
||||
"type": "local",
|
||||
"command": ["npx", "-y", "chrome-devtools-mcp@latest", "--browser-url=http://127.0.0.1:9222"]
|
||||
}
|
||||
},
|
||||
"provider": {
|
||||
"llama.cpp": {
|
||||
"npm": "@ai-sdk/openai-compatible",
|
||||
"options": {
|
||||
"baseURL": "http://localhost:8080/v1"
|
||||
},
|
||||
"models": {
|
||||
"ornith-1.0": {
|
||||
"tool_call": true,
|
||||
"reasoning": true,
|
||||
"limit": {
|
||||
"context": 65536,
|
||||
"output": 8192
|
||||
}
|
||||
},
|
||||
"gemma-4": {
|
||||
"tool_call": true,
|
||||
"reasoning": true,
|
||||
"limit": {
|
||||
"context": 131072,
|
||||
"output": 8192
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
52
llama-cpp/run_gemma4.sh
Executable file
52
llama-cpp/run_gemma4.sh
Executable file
@ -0,0 +1,52 @@
|
||||
#!/bin/bash
|
||||
|
||||
model_postfix="Q8_0"
|
||||
model_addr="unsloth/gemma-4-12b-it-GGUF"
|
||||
draft_mtp=""
|
||||
quant=$1
|
||||
cache_type="q8_0"
|
||||
fa="on"
|
||||
|
||||
if [[ ${quant} = "q4_xl" ]]; then
|
||||
model_postfix="UD-Q4_K_XL"
|
||||
cache_type="q4_0"
|
||||
model_addr="unsloth/gemma-4-12B-it-qat-GGUF"
|
||||
draft_mtp="--spec-type draft-mtp --spec-draft-n-max 4"
|
||||
elif [[ ${quant} = "q4" ]]; then
|
||||
model_postfix="Q4_K_M"
|
||||
cache_type="q4_0"
|
||||
model_addr="unsloth/gemma-4-12B-it-GGUF"
|
||||
elif [[ ${quant} = "q5" ]]; then
|
||||
model_postfix="Q5_K_M"
|
||||
cache_type="q5_0"
|
||||
elif [[ ${quant} = "q8_small" ]]; then
|
||||
model_postfix="Q8_0"
|
||||
model_addr="unsloth/gemma-4-E4B-it-GGUF"
|
||||
cache_type="q8_0"
|
||||
fa="on"
|
||||
elif [[ ${quant} = "q4_small" ]]; then
|
||||
model_postfix="Q4_K_M"
|
||||
model_addr="unsloth/gemma-4-E4B-it-GGUF"
|
||||
cache_type="q4_0"
|
||||
fa="on"
|
||||
fi
|
||||
|
||||
echo "cache_type:${cache_type}; model_postfix:${model_postfix}; draft_mtp:${draft_mtp}"
|
||||
|
||||
llama-server \
|
||||
-hf ${model_addr}:${model_postfix} \
|
||||
--ctx-size 131072 \
|
||||
--alias gemma-4 \
|
||||
--jinja \
|
||||
${draft_mtp} \
|
||||
-ngl 999 \
|
||||
-fa ${fa} \
|
||||
-b 1024 \
|
||||
-ub 512 \
|
||||
--no-ui \
|
||||
--cache-ram 2048 \
|
||||
--kv-unified \
|
||||
--cache-type-k ${cache_type} \
|
||||
--cache-type-v ${cache_type} \
|
||||
--repeat-penalty 1.05 \
|
||||
--host 0.0.0.0
|
||||
19
llama-cpp/run_ornith.sh
Executable file
19
llama-cpp/run_ornith.sh
Executable file
@ -0,0 +1,19 @@
|
||||
#!/bin/bash
|
||||
|
||||
llama-server \
|
||||
-hf deepreinforce-ai/Ornith-1.0-9B-GGUF \
|
||||
--ctx-size 65536 \
|
||||
--alias ornith-1.0 \
|
||||
--jinja \
|
||||
--chat-template-file ~/.config/llama.cpp/ornith-toolfix.jinja \
|
||||
-ngl 99 \
|
||||
-fa on \
|
||||
-b 2048 \
|
||||
-ub 1024 \
|
||||
--cache-ram 2048 \
|
||||
--kv-unified \
|
||||
--cache-type-k q8_0 \
|
||||
--cache-type-v q8_0 \
|
||||
--presence-penalty 1.5 \
|
||||
--repeat-penalty 1.15 \
|
||||
--host 0.0.0.0
|
||||
Loading…
x
Reference in New Issue
Block a user