# Confluence Web Backend Service

A FastAPI backend service providing isolated, secure Confluence research orchestration using rootless Docker containers, authoritative tool history, and private artifact retention.

## Architecture

```text
backend/
  app.py                 # FastAPI application factory, session cookies, security headers, endpoints
  settings.py            # Validated deployment settings and URL validation/canonicalization
  runner.py              # QueryRunner orchestrating deadlines, gate, disconnect cancellation, cleanup
  admission.py           # In-memory admission queue (FIFO tickets, reservation, estimate)
  transport.py           # Bounded NDJSON bridge reader/writer and protocol state machine (Rev 1)
  containers.py          # Rootless Docker container lifecycle, tmpfs mounts, kill/remove, reconciliation
  confluence.py          # Request-scoped Confluence client, streaming bounds, and tool dispatcher
  model.py               # Neutral model contract adapter (OpenAI-compatible Chat Completions / Fake)
  history.py             # Authoritative in-memory tool execution history and page auditing
  artifacts.py           # Private temporary artifact staging, validation, session binding, downloads
  dev/
    fake_peer.py         # Scripted container peer implementing NDJSON contract
    fake_entrypoint.py   # Container entrypoint for fake peer
    Dockerfile.fake      # Test container image definition
  README.md              # Configuration & documentation
```

## Environment Variables

| Variable | Default | Description |
|---|---|---|
| `CONFLUENCE_WEB_APPROVED_ORIGINS` | `https://approved.example.com` | Comma-separated list of approved Confluence base origins and context paths. Exposed in canonical form by `GET /api/v1/config`, which the UI uses to offer the origin as a fixed choice. |
| `CONFLUENCE_WEB_CORPORATE_CA_PATH` | `None` | Path to corporate CA bundle for TLS verification if needed. |
| `CONFLUENCE_WEB_MODEL_PROVIDER` | `fake` | Model provider adapter (`fake`, `openai`). |
| `CONFLUENCE_WEB_MODEL_NAME` | `fake-model` | Model name (e.g. `gpt-4o`). |
| `CONFLUENCE_WEB_MODEL_API_KEY` | `None` | Backend-held API key for the model provider. |
| `CONFLUENCE_WEB_MODEL_ENDPOINT` | `None` | Custom model provider endpoint URL. |
| `CONFLUENCE_WEB_MODEL_CONTEXT_WINDOW_TOKENS` | `128000` | Model context window token limit. |
| `CONFLUENCE_WEB_MODEL_MAX_OUTPUT_TOKENS` | `4096` | Model max output tokens limit. |
| `CONFLUENCE_WEB_RUNTIME_IMAGE` | `confluence-agent:latest` | Pinned Docker container image for the pi agent runtime. |
| `CONFLUENCE_WEB_DOCKER_HOST` | `None` | Custom Docker host / unix socket path. |
| `CONFLUENCE_WEB_CONTAINER_LABEL_KEY` | `com.confluence_web.app` | Docker container label namespace key. |
| `CONFLUENCE_WEB_CONTAINER_LABEL_VALUE` | `query-runner` | Set a unique value for each deployment; reconciliation uses this value. |
| `CONFLUENCE_WEB_CONTAINER_INSTANCE_ID` | generated UUID | Non-secret instance metadata label; orphan discovery also includes previous instances. |
| `CONFLUENCE_WEB_ARTIFACT_DIR` | `/tmp/confluence_web_artifacts` | Private backend-managed directory for artifact staging and downloads. |
| `CONFLUENCE_WEB_BIND_HOST` | `127.0.0.1` | Bind address for FastAPI service. |
| `CONFLUENCE_WEB_BIND_PORT` | `8000` | Port for FastAPI service. |
| `CONFLUENCE_WEB_FRONTEND_DIST_DIR` | `None` | Directory containing built frontend static files. |
| `CONFLUENCE_WEB_DEV_MODE` | `false` | Explicit opt-in development mode with fake test dependencies. |
| `CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS` | `180.0` | Total query execution deadline; must not exceed `CONFLUENCE_WEB_MAX_DEADLINE_SECONDS`. |
| `CONFLUENCE_WEB_MAX_DEADLINE_SECONDS` | `900.0` | Protocol maximum for one query (60–3600). Passed into the agent container so the supervisor and bridge enforce the same bound. |
| `CONFLUENCE_WEB_CLEANUP_TIMEOUT_SECONDS` | `10.0` | Dedicated cleanup timeout. |
| `CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS` | `100` | Confluence tool calls allowed per query (1–1000); cache hits are free on the backend side. Passed into the agent container, whose bridge enforces the same count. The history keeps at most 100 tool entries, so budgets above 100 lose later entries. |
| `CONFLUENCE_WEB_MAX_MODEL_CALLS` | `50` | Model requests allowed per query (1–1000). Passed into the agent container, whose bridge enforces the same count. |
| `CONFLUENCE_WEB_QUEUE_RESERVATION_SECONDS` | `45` | Admission queue reservation window after promotion; allowed 30-60. See [QUEUE_SPECIFICATION.md](../docs/QUEUE_SPECIFICATION.md). |
| `CONFLUENCE_WEB_QUEUE_HEARTBEAT_SECONDS` | `15` | Admission queue heartbeat timeout for queued (not yet reserved) tickets; allowed 5-60. |
| `CONFLUENCE_WEB_QUEUE_MAX_LENGTH` | `20` | Maximum queued tickets, excluding the reserved and running sessions; allowed 1-100. |

## Running the Service

Use the existing virtual environment (`.venv/bin/python -m pip install -r requirements.txt` if dependencies need installing).

Version 1 requires **one worker process**. Multiple workers have separate query gates, sessions and artifact indexes and violate the service assumptions. Do not run concurrent service instances with the same artifact directory or container label value.

Production configuration must set `CONFLUENCE_WEB_MODEL_PROVIDER=openai`, the provider key/model, approved Confluence origins, runtime image, and a deployment-specific container label value. Unknown providers and missing/blank real-provider keys fail configuration; Docker/provider failures never select a fake automatically. The `fake` provider is an explicit test configuration.

```bash
.venv/bin/uvicorn backend.app:create_app --factory --workers 1 --host 127.0.0.1 --port 8000
```

Startup requires rootless Docker and cgroup v2 with memory, CPU quota/period and PID enforcement reported by the daemon. It purges prior-run artifacts and confirms removal of all residual containers in the deployment label namespace before accepting work. Maintenance runs every 60 seconds while the query gate is idle. Uncertain cleanup leaves query admission closed until reconciliation succeeds.

For network-free development, all three dependencies are fake unless explicitly injected: an in-process NDJSON peer, a fake model, and an HTTPX Confluence substitute. No Docker or remote provider is required. Use URL `https://approved.example.com` and PAT `dev-pat` in the browser or API client; other PATs fail verification.

```bash
CONFLUENCE_WEB_DEV_MODE=true .venv/bin/uvicorn backend.app:create_app --factory --workers 1 --host 127.0.0.1 --port 8000
```

Browser/manual POST clients must send an Origin matching the backend scheme, host and effective port. Bootstrap `GET /` issues the server-owned session cookie. Downloads require that cookie; query UUIDs are not ownership credentials.

## Running Tests

Build the independent fake image before real Docker tests:

```bash
docker build -t confluence-fake-agent:test -f backend/dev/Dockerfile.fake .
.venv/bin/python -m pytest tests/backend
```

Docker checks run against a responsive rootless daemon and the built image; a missing/unresponsive daemon skips those checks, rather than claiming success. To run only deterministic tests:

```bash
.venv/bin/python -m pytest tests/backend --ignore=tests/backend/test_docker_live.py
```

Bare pytest deselects the existing `live` crawler marker. The crawler configuration tests still require their own environment variables. A completely offline full-suite invocation uses synthetic values:

```bash
CONFLUENCE_PAT=backend-synthetic-token-no-network CONFLUENCE_URL=https://approved.example.com .venv/bin/python -m pytest
```

### Debugging a failed run

The HTTP error is a fixed sanitized code, so the reason lives in the backend log
(timestamped via `deploy/logging.json`; override with `CONFLUENCE_WEB_LOG_CONFIG`).
A failed query logs two lines: the agent's terminal code with the backend's own
model/Confluence call counters, and the tail of the container's stderr, where the
agent writes `[agent] run failed code=... reason=... model_calls=n/m tool_calls=n/m`.
An exhausted model-call budget and a model that ends its turn with no text both
surface as `execution_failed`; the counters and the reason tell them apart. Raising
`CONFLUENCE_WEB_MAX_MODEL_CALLS` helps only the first.

Resource contracts remain 16 MiB decoded prompt, 128 MiB decoded answer, 128 KiB verify body and `6 * 16 MiB + 64 KiB` query body. Request bytes are counted while reading, independent of Content-Length. History is at most 100 entries / 128 MiB, with 64 KiB reserved metadata per entry. Artifact limits are fixed contract values: 20 files, 10 MiB/file, 50 MiB/query, 500 MiB global; default TTL is 900 seconds. Call totals default to 100 Confluence / 50 model per query and are set with `CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS` / `CONFLUENCE_WEB_MAX_MODEL_CALLS`; retention limits have Python Settings defaults but no environment switches.