backend: per-query call budgets configurable from the environment
The 100 Confluence / 50 model call caps per query were fixed Settings defaults. CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS and CONFLUENCE_WEB_MAX_MODEL_CALLS now set them (1-1000 each, validated at startup). Cache hits stay free. The request history still keeps at most 100 tool entries, so a Confluence budget above 100 drops later entries with the existing history_overflow warning; documented next to the setting.
This commit is contained in:
parent
1a8c220ca3
commit
a647406729
@ -113,7 +113,7 @@ make run-dev
|
||||
- **Model key**: backend environment only (`CONFLUENCE_WEB_MODEL_API_KEY`). For a local llama.cpp server any placeholder works.
|
||||
- **Session cookie** `cw_session`: HttpOnly, SameSite=Strict, marks artifact ownership only. It is not authentication.
|
||||
- **Artifacts**: at most 20 files, 10 MiB each, 50 MiB per query, 500 MiB total; downloadable for 15 minutes after a successful query, then deleted. Failed or cancelled queries keep nothing.
|
||||
- **Query**: 180 s total deadline by default (`CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS`, up to `CONFLUENCE_WEB_MAX_DEADLINE_SECONDS`, default 900 s) plus 10 s cleanup, prompt up to 16 MiB, answer up to 128 MiB, 100 Confluence calls and 50 model calls per query, container limited to 1 GiB RAM, 1 CPU, 128 processes, no network.
|
||||
- **Query**: 180 s total deadline by default (`CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS`, up to `CONFLUENCE_WEB_MAX_DEADLINE_SECONDS`, default 900 s) plus 10 s cleanup, prompt up to 16 MiB, answer up to 128 MiB, 100 Confluence calls and 50 model calls per query by default (`CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS`, `CONFLUENCE_WEB_MAX_MODEL_CALLS`), container limited to 1 GiB RAM, 1 CPU, 128 processes, no network.
|
||||
- **Model tokens**: `CONFLUENCE_WEB_MODEL_CONTEXT_WINDOW_TOKENS` and `..._MAX_OUTPUT_TOKENS` describe the provider; they are independent of the byte limits above. Large pages or answers can exceed the model's context before the application limits; the UI then shows `model_context_exceeded` or `model_output_limit`.
|
||||
|
||||
## Checks
|
||||
|
||||
@ -48,6 +48,8 @@ backend/
|
||||
| `CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS` | `180.0` | Total query execution deadline; must not exceed `CONFLUENCE_WEB_MAX_DEADLINE_SECONDS`. |
|
||||
| `CONFLUENCE_WEB_MAX_DEADLINE_SECONDS` | `900.0` | Protocol maximum for one query (60–3600). Passed into the agent container so the supervisor and bridge enforce the same bound. |
|
||||
| `CONFLUENCE_WEB_CLEANUP_TIMEOUT_SECONDS` | `10.0` | Dedicated cleanup timeout. |
|
||||
| `CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS` | `100` | Confluence tool calls allowed per query (1–1000); cache hits are free. The history keeps at most 100 tool entries, so budgets above 100 lose later entries. |
|
||||
| `CONFLUENCE_WEB_MAX_MODEL_CALLS` | `50` | Model requests allowed per query (1–1000). |
|
||||
| `CONFLUENCE_WEB_QUEUE_RESERVATION_SECONDS` | `45` | Admission queue reservation window after promotion; allowed 30-60. See [QUEUE_SPECIFICATION.md](../docs/QUEUE_SPECIFICATION.md). |
|
||||
| `CONFLUENCE_WEB_QUEUE_HEARTBEAT_SECONDS` | `15` | Admission queue heartbeat timeout for queued (not yet reserved) tickets; allowed 5-60. |
|
||||
| `CONFLUENCE_WEB_QUEUE_MAX_LENGTH` | `20` | Maximum queued tickets, excluding the reserved and running sessions; allowed 1-100. |
|
||||
@ -95,4 +97,4 @@ Bare pytest deselects the existing `live` crawler marker. The crawler configurat
|
||||
CONFLUENCE_PAT=backend-synthetic-token-no-network CONFLUENCE_URL=https://approved.example.com .venv/bin/python -m pytest
|
||||
```
|
||||
|
||||
Resource contracts remain 16 MiB decoded prompt, 128 MiB decoded answer, 128 KiB verify body and `6 * 16 MiB + 64 KiB` query body. Request bytes are counted while reading, independent of Content-Length. History is at most 100 entries / 128 MiB, with 64 KiB reserved metadata per entry. Artifact limits are fixed contract values: 20 files, 10 MiB/file, 50 MiB/query, 500 MiB global; default TTL is 900 seconds. Call totals are 100 Confluence / 50 model; these and retention have Python Settings defaults but no additional environment switches.
|
||||
Resource contracts remain 16 MiB decoded prompt, 128 MiB decoded answer, 128 KiB verify body and `6 * 16 MiB + 64 KiB` query body. Request bytes are counted while reading, independent of Content-Length. History is at most 100 entries / 128 MiB, with 64 KiB reserved metadata per entry. Artifact limits are fixed contract values: 20 files, 10 MiB/file, 50 MiB/query, 500 MiB global; default TTL is 900 seconds. Call totals default to 100 Confluence / 50 model per query and are set with `CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS` / `CONFLUENCE_WEB_MAX_MODEL_CALLS`; retention limits have Python Settings defaults but no environment switches.
|
||||
|
||||
@ -63,6 +63,8 @@ class Settings(BaseModel):
|
||||
max_deadline_seconds: float = 900.0
|
||||
cleanup_timeout_seconds: float = 10.0
|
||||
max_inflight_remote_calls: int = 4
|
||||
# Per-query call budgets (1-1000). Cache hits do not count. The request history keeps at
|
||||
# most 100 tool entries, so Confluence budgets above 100 lose later history entries.
|
||||
max_confluence_calls: int = 100
|
||||
max_model_calls: int = 50
|
||||
|
||||
@ -122,6 +124,13 @@ class Settings(BaseModel):
|
||||
raise ValueError("max_deadline_seconds must be between 60 and 3600")
|
||||
return value
|
||||
|
||||
@field_validator("max_confluence_calls", "max_model_calls")
|
||||
@classmethod
|
||||
def validate_call_budgets(cls, value):
|
||||
if not (1 <= value <= 1000):
|
||||
raise ValueError("Call budgets must be between 1 and 1000")
|
||||
return value
|
||||
|
||||
@model_validator(mode="after")
|
||||
def validate_query_timeout_within_max_deadline(self):
|
||||
if self.query_timeout_seconds > self.max_deadline_seconds:
|
||||
@ -233,6 +242,8 @@ class Settings(BaseModel):
|
||||
"CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS", 180.0
|
||||
),
|
||||
max_deadline_seconds=_parse_float("CONFLUENCE_WEB_MAX_DEADLINE_SECONDS", 900.0),
|
||||
max_confluence_calls=_parse_int("CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS", 100),
|
||||
max_model_calls=_parse_int("CONFLUENCE_WEB_MAX_MODEL_CALLS", 50),
|
||||
cleanup_timeout_seconds=_parse_float(
|
||||
"CONFLUENCE_WEB_CLEANUP_TIMEOUT_SECONDS", 10.0
|
||||
),
|
||||
|
||||
@ -50,6 +50,11 @@ CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS=600
|
||||
# Passed into the agent container at start so its supervisor enforces the same bound.
|
||||
CONFLUENCE_WEB_MAX_DEADLINE_SECONDS=900
|
||||
CONFLUENCE_WEB_CLEANUP_TIMEOUT_SECONDS=10
|
||||
# Per-query call budgets (1-1000 each); cache hits are free. Reaching a budget returns an
|
||||
# error to the agent, which then answers from what it has. The request history keeps at most
|
||||
# 100 tool entries, so a Confluence budget above 100 loses later history entries.
|
||||
CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS=100
|
||||
CONFLUENCE_WEB_MAX_MODEL_CALLS=50
|
||||
|
||||
# --- Admission queue (docs/QUEUE_SPECIFICATION.md); defaults shown, allowed ranges in comments ---
|
||||
# Reservation window after promotion, seconds (30-60).
|
||||
|
||||
@ -134,3 +134,33 @@ def test_max_deadline_seconds_from_env(monkeypatch):
|
||||
monkeypatch.setenv("CONFLUENCE_WEB_MAX_DEADLINE_SECONDS", "1200")
|
||||
monkeypatch.setenv("CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS", "1000")
|
||||
assert Settings.from_env().max_deadline_seconds == 1200.0
|
||||
|
||||
|
||||
def test_call_budgets_defaults_and_bounds():
|
||||
settings = Settings()
|
||||
assert settings.max_confluence_calls == 100
|
||||
assert settings.max_model_calls == 50
|
||||
Settings(max_confluence_calls=1, max_model_calls=1)
|
||||
Settings(max_confluence_calls=1000, max_model_calls=1000)
|
||||
with pytest.raises(ValidationError):
|
||||
Settings(max_confluence_calls=0)
|
||||
with pytest.raises(ValidationError):
|
||||
Settings(max_confluence_calls=1001)
|
||||
with pytest.raises(ValidationError):
|
||||
Settings(max_model_calls=0)
|
||||
with pytest.raises(ValidationError):
|
||||
Settings(max_model_calls=1001)
|
||||
|
||||
|
||||
def test_call_budgets_from_env(monkeypatch):
|
||||
monkeypatch.setenv("CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS", "250")
|
||||
monkeypatch.setenv("CONFLUENCE_WEB_MAX_MODEL_CALLS", "120")
|
||||
settings = Settings.from_env()
|
||||
assert settings.max_confluence_calls == 250
|
||||
assert settings.max_model_calls == 120
|
||||
|
||||
|
||||
def test_call_budgets_from_env_reject_non_integer(monkeypatch):
|
||||
monkeypatch.setenv("CONFLUENCE_WEB_MAX_MODEL_CALLS", "many")
|
||||
with pytest.raises(ValueError, match="Invalid integer"):
|
||||
Settings.from_env()
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user