backend: per-query call budgets configurable from the environment

The 100 Confluence / 50 model call caps per query were fixed Settings
defaults. CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS and CONFLUENCE_WEB_MAX_MODEL_CALLS
now set them (1-1000 each, validated at startup). Cache hits stay free. The
request history still keeps at most 100 tool entries, so a Confluence budget
above 100 drops later entries with the existing history_overflow warning;
documented next to the setting.
This commit is contained in:
Artur Mukhamadiev 2026-09-15 14:29:37 +03:00
parent 1a8c220ca3
commit a647406729
5 changed files with 50 additions and 2 deletions

View File

@ -113,7 +113,7 @@ make run-dev
- **Model key**: backend environment only (`CONFLUENCE_WEB_MODEL_API_KEY`). For a local llama.cpp server any placeholder works.
- **Session cookie** `cw_session`: HttpOnly, SameSite=Strict, marks artifact ownership only. It is not authentication.
- **Artifacts**: at most 20 files, 10 MiB each, 50 MiB per query, 500 MiB total; downloadable for 15 minutes after a successful query, then deleted. Failed or cancelled queries keep nothing.
- **Query**: 180 s total deadline by default (`CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS`, up to `CONFLUENCE_WEB_MAX_DEADLINE_SECONDS`, default 900 s) plus 10 s cleanup, prompt up to 16 MiB, answer up to 128 MiB, 100 Confluence calls and 50 model calls per query, container limited to 1 GiB RAM, 1 CPU, 128 processes, no network.
- **Query**: 180 s total deadline by default (`CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS`, up to `CONFLUENCE_WEB_MAX_DEADLINE_SECONDS`, default 900 s) plus 10 s cleanup, prompt up to 16 MiB, answer up to 128 MiB, 100 Confluence calls and 50 model calls per query by default (`CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS`, `CONFLUENCE_WEB_MAX_MODEL_CALLS`), container limited to 1 GiB RAM, 1 CPU, 128 processes, no network.
- **Model tokens**: `CONFLUENCE_WEB_MODEL_CONTEXT_WINDOW_TOKENS` and `..._MAX_OUTPUT_TOKENS` describe the provider; they are independent of the byte limits above. Large pages or answers can exceed the model's context before the application limits; the UI then shows `model_context_exceeded` or `model_output_limit`.
## Checks

View File

@ -48,6 +48,8 @@ backend/
| `CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS` | `180.0` | Total query execution deadline; must not exceed `CONFLUENCE_WEB_MAX_DEADLINE_SECONDS`. |
| `CONFLUENCE_WEB_MAX_DEADLINE_SECONDS` | `900.0` | Protocol maximum for one query (603600). Passed into the agent container so the supervisor and bridge enforce the same bound. |
| `CONFLUENCE_WEB_CLEANUP_TIMEOUT_SECONDS` | `10.0` | Dedicated cleanup timeout. |
| `CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS` | `100` | Confluence tool calls allowed per query (11000); cache hits are free. The history keeps at most 100 tool entries, so budgets above 100 lose later entries. |
| `CONFLUENCE_WEB_MAX_MODEL_CALLS` | `50` | Model requests allowed per query (11000). |
| `CONFLUENCE_WEB_QUEUE_RESERVATION_SECONDS` | `45` | Admission queue reservation window after promotion; allowed 30-60. See [QUEUE_SPECIFICATION.md](../docs/QUEUE_SPECIFICATION.md). |
| `CONFLUENCE_WEB_QUEUE_HEARTBEAT_SECONDS` | `15` | Admission queue heartbeat timeout for queued (not yet reserved) tickets; allowed 5-60. |
| `CONFLUENCE_WEB_QUEUE_MAX_LENGTH` | `20` | Maximum queued tickets, excluding the reserved and running sessions; allowed 1-100. |
@ -95,4 +97,4 @@ Bare pytest deselects the existing `live` crawler marker. The crawler configurat
CONFLUENCE_PAT=backend-synthetic-token-no-network CONFLUENCE_URL=https://approved.example.com .venv/bin/python -m pytest
```
Resource contracts remain 16 MiB decoded prompt, 128 MiB decoded answer, 128 KiB verify body and `6 * 16 MiB + 64 KiB` query body. Request bytes are counted while reading, independent of Content-Length. History is at most 100 entries / 128 MiB, with 64 KiB reserved metadata per entry. Artifact limits are fixed contract values: 20 files, 10 MiB/file, 50 MiB/query, 500 MiB global; default TTL is 900 seconds. Call totals are 100 Confluence / 50 model; these and retention have Python Settings defaults but no additional environment switches.
Resource contracts remain 16 MiB decoded prompt, 128 MiB decoded answer, 128 KiB verify body and `6 * 16 MiB + 64 KiB` query body. Request bytes are counted while reading, independent of Content-Length. History is at most 100 entries / 128 MiB, with 64 KiB reserved metadata per entry. Artifact limits are fixed contract values: 20 files, 10 MiB/file, 50 MiB/query, 500 MiB global; default TTL is 900 seconds. Call totals default to 100 Confluence / 50 model per query and are set with `CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS` / `CONFLUENCE_WEB_MAX_MODEL_CALLS`; retention limits have Python Settings defaults but no environment switches.

View File

@ -63,6 +63,8 @@ class Settings(BaseModel):
max_deadline_seconds: float = 900.0
cleanup_timeout_seconds: float = 10.0
max_inflight_remote_calls: int = 4
# Per-query call budgets (1-1000). Cache hits do not count. The request history keeps at
# most 100 tool entries, so Confluence budgets above 100 lose later history entries.
max_confluence_calls: int = 100
max_model_calls: int = 50
@ -122,6 +124,13 @@ class Settings(BaseModel):
raise ValueError("max_deadline_seconds must be between 60 and 3600")
return value
@field_validator("max_confluence_calls", "max_model_calls")
@classmethod
def validate_call_budgets(cls, value):
if not (1 <= value <= 1000):
raise ValueError("Call budgets must be between 1 and 1000")
return value
@model_validator(mode="after")
def validate_query_timeout_within_max_deadline(self):
if self.query_timeout_seconds > self.max_deadline_seconds:
@ -233,6 +242,8 @@ class Settings(BaseModel):
"CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS", 180.0
),
max_deadline_seconds=_parse_float("CONFLUENCE_WEB_MAX_DEADLINE_SECONDS", 900.0),
max_confluence_calls=_parse_int("CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS", 100),
max_model_calls=_parse_int("CONFLUENCE_WEB_MAX_MODEL_CALLS", 50),
cleanup_timeout_seconds=_parse_float(
"CONFLUENCE_WEB_CLEANUP_TIMEOUT_SECONDS", 10.0
),

View File

@ -50,6 +50,11 @@ CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS=600
# Passed into the agent container at start so its supervisor enforces the same bound.
CONFLUENCE_WEB_MAX_DEADLINE_SECONDS=900
CONFLUENCE_WEB_CLEANUP_TIMEOUT_SECONDS=10
# Per-query call budgets (1-1000 each); cache hits are free. Reaching a budget returns an
# error to the agent, which then answers from what it has. The request history keeps at most
# 100 tool entries, so a Confluence budget above 100 loses later history entries.
CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS=100
CONFLUENCE_WEB_MAX_MODEL_CALLS=50
# --- Admission queue (docs/QUEUE_SPECIFICATION.md); defaults shown, allowed ranges in comments ---
# Reservation window after promotion, seconds (30-60).

View File

@ -134,3 +134,33 @@ def test_max_deadline_seconds_from_env(monkeypatch):
monkeypatch.setenv("CONFLUENCE_WEB_MAX_DEADLINE_SECONDS", "1200")
monkeypatch.setenv("CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS", "1000")
assert Settings.from_env().max_deadline_seconds == 1200.0
def test_call_budgets_defaults_and_bounds():
settings = Settings()
assert settings.max_confluence_calls == 100
assert settings.max_model_calls == 50
Settings(max_confluence_calls=1, max_model_calls=1)
Settings(max_confluence_calls=1000, max_model_calls=1000)
with pytest.raises(ValidationError):
Settings(max_confluence_calls=0)
with pytest.raises(ValidationError):
Settings(max_confluence_calls=1001)
with pytest.raises(ValidationError):
Settings(max_model_calls=0)
with pytest.raises(ValidationError):
Settings(max_model_calls=1001)
def test_call_budgets_from_env(monkeypatch):
monkeypatch.setenv("CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS", "250")
monkeypatch.setenv("CONFLUENCE_WEB_MAX_MODEL_CALLS", "120")
settings = Settings.from_env()
assert settings.max_confluence_calls == 250
assert settings.max_model_calls == 120
def test_call_budgets_from_env_reject_non_integer(monkeypatch):
monkeypatch.setenv("CONFLUENCE_WEB_MAX_MODEL_CALLS", "many")
with pytest.raises(ValueError, match="Invalid integer"):
Settings.from_env()