vptyp/confluence_web
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
confluence_web/backend/README.md

6.7 KiB
Raw Blame History

Confluence Web Backend Service

A FastAPI backend service providing isolated, secure Confluence research orchestration using rootless Docker containers, authoritative tool history, and private artifact retention.

Architecture

backend/
  app.py                 # FastAPI application factory, session cookies, security headers, endpoints
  settings.py            # Validated deployment settings and URL validation/canonicalization
  runner.py              # QueryRunner orchestrating deadlines, gate, disconnect cancellation, cleanup
  transport.py           # Bounded NDJSON bridge reader/writer and protocol state machine (Rev 1)
  containers.py          # Rootless Docker container lifecycle, tmpfs mounts, kill/remove, reconciliation
  confluence.py          # Request-scoped Confluence client, streaming bounds, and tool dispatcher
  model.py               # Neutral model contract adapter (OpenAI-compatible Chat Completions / Fake)
  history.py             # Authoritative in-memory tool execution history and page auditing
  artifacts.py           # Private temporary artifact staging, validation, session binding, downloads
  dev/
    fake_peer.py         # Scripted container peer implementing NDJSON contract
    fake_entrypoint.py   # Container entrypoint for fake peer
    Dockerfile.fake      # Test container image definition
  README.md              # Configuration & documentation

Environment Variables

Variable Default Description
CONFLUENCE_WEB_APPROVED_ORIGINS https://approved.example.com Comma-separated list of approved Confluence base origins and context paths.
CONFLUENCE_WEB_CORPORATE_CA_PATH None Path to corporate CA bundle for TLS verification if needed.
CONFLUENCE_WEB_MODEL_PROVIDER fake Model provider adapter (fake, openai).
CONFLUENCE_WEB_MODEL_NAME fake-model Model name (e.g. gpt-4o).
CONFLUENCE_WEB_MODEL_API_KEY None Backend-held API key for the model provider.
CONFLUENCE_WEB_MODEL_ENDPOINT None Custom model provider endpoint URL.
CONFLUENCE_WEB_MODEL_CONTEXT_WINDOW_TOKENS 128000 Model context window token limit.
CONFLUENCE_WEB_MODEL_MAX_OUTPUT_TOKENS 4096 Model max output tokens limit.
CONFLUENCE_WEB_RUNTIME_IMAGE confluence-agent:latest Pinned Docker container image for the pi agent runtime.
CONFLUENCE_WEB_DOCKER_HOST None Custom Docker host / unix socket path.
CONFLUENCE_WEB_CONTAINER_LABEL_KEY com.confluence_web.app Docker container label namespace key.
CONFLUENCE_WEB_CONTAINER_LABEL_VALUE query-runner Set a unique value for each deployment; reconciliation uses this value.
CONFLUENCE_WEB_CONTAINER_INSTANCE_ID generated UUID Non-secret instance metadata label; orphan discovery also includes previous instances.
CONFLUENCE_WEB_ARTIFACT_DIR /tmp/confluence_web_artifacts Private backend-managed directory for artifact staging and downloads.
CONFLUENCE_WEB_BIND_HOST 127.0.0.1 Bind address for FastAPI service.
CONFLUENCE_WEB_BIND_PORT 8000 Port for FastAPI service.
CONFLUENCE_WEB_FRONTEND_DIST_DIR None Directory containing built frontend static files.
CONFLUENCE_WEB_DEV_MODE false Explicit opt-in development mode with fake test dependencies.
CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS 180.0 Total query execution deadline.
CONFLUENCE_WEB_CLEANUP_TIMEOUT_SECONDS 10.0 Dedicated cleanup timeout.

Running the Service

Use the existing virtual environment (.venv/bin/python -m pip install -r requirements.txt if dependencies need installing).

Version 1 requires one worker process. Multiple workers have separate query gates, sessions and artifact indexes and violate the service assumptions. Do not run concurrent service instances with the same artifact directory or container label value.

Production configuration must set CONFLUENCE_WEB_MODEL_PROVIDER=openai, the provider key/model, approved Confluence origins, runtime image, and a deployment-specific container label value. Unknown providers and missing/blank real-provider keys fail configuration; Docker/provider failures never select a fake automatically. The fake provider is an explicit test configuration.

.venv/bin/uvicorn backend.app:create_app --factory --workers 1 --host 127.0.0.1 --port 8000

Startup requires rootless Docker and cgroup v2 with memory, CPU quota/period and PID enforcement reported by the daemon. It purges prior-run artifacts and confirms removal of all residual containers in the deployment label namespace before accepting work. Maintenance runs every 60 seconds while the query gate is idle. Uncertain cleanup leaves query admission closed until reconciliation succeeds.

For network-free development, all three dependencies are fake unless explicitly injected: an in-process NDJSON peer, a fake model, and an HTTPX Confluence substitute. No Docker or remote provider is required. Use URL https://approved.example.com and PAT dev-pat in the browser or API client; other PATs fail verification.

CONFLUENCE_WEB_DEV_MODE=true .venv/bin/uvicorn backend.app:create_app --factory --workers 1 --host 127.0.0.1 --port 8000

Browser/manual POST clients must send an Origin matching the backend scheme, host and effective port. Bootstrap GET / issues the server-owned session cookie. Downloads require that cookie; query UUIDs are not ownership credentials.

Running Tests

Build the independent fake image before real Docker tests:

docker build -t confluence-fake-agent:test -f backend/dev/Dockerfile.fake .
.venv/bin/python -m pytest tests/backend

Docker checks run against a responsive rootless daemon and the built image; a missing/unresponsive daemon skips those checks, rather than claiming success. To run only deterministic tests:

.venv/bin/python -m pytest tests/backend --ignore=tests/backend/test_docker_live.py

Bare pytest deselects the existing live crawler marker. The crawler configuration tests still require their own environment variables. A completely offline full-suite invocation uses synthetic values:

CONFLUENCE_PAT=backend-synthetic-token-no-network CONFLUENCE_URL=https://approved.example.com .venv/bin/python -m pytest

Resource contracts remain 16 MiB decoded prompt, 128 MiB decoded answer, 128 KiB verify body and 6 * 16 MiB + 64 KiB query body. Request bytes are counted while reading, independent of Content-Length. History is at most 100 entries / 128 MiB, with 64 KiB reserved metadata per entry. Artifact limits are fixed contract values: 20 files, 10 MiB/file, 50 MiB/query, 500 MiB global; default TTL is 900 seconds. Call totals are 100 Confluence / 50 model; these and retention have Python Settings defaults but no additional environment switches.