Artur Mukhamadiev
d77b52ae86
- tests/integration: frontend served by backend, runtime error mapping, backend + real pi image with scripted model/Confluence (shared example, variants, failure paths, cancellation/busy gate, isolation canaries, HTTP download ownership, 16 MiB prompt round trip), real OpenAI-compatible adapter + real image over a scripted transport, real Chrome against the real backend with a scripted runtime peer, backend crash/restart reconciliation, and an opt-in live model check (marker: live). - backend: map runtime terminal codes (model_output_limit, model_context_exceeded, query_timeout, connectivity_failed) to the contract's HTTP statuses; make the artifact 404 body identical for no-session, wrong-session, unknown and expired IDs. - Makefile, scripts/run-backend.sh, deploy/confluence-web.env.example, root README for the integrated application; integration pytest marker.
116 lines
6.4 KiB
Markdown
116 lines
6.4 KiB
Markdown
# Confluence Research Web UI
|
|
|
|
A minimalist web UI that researches a Confluence Data Center instance with a
|
|
[pi](https://github.com/earendil-works/pi) agent running in a fresh, network-less
|
|
rootless Docker container per query, and returns a cited Markdown answer plus
|
|
optional files the agent produced. The design is in
|
|
[docs/SPECIFICATION.md](docs/SPECIFICATION.md); the wire contracts in
|
|
[docs/implementation/CONTRACTS.md](docs/implementation/CONTRACTS.md); the
|
|
integration results in [docs/INTEGRATION_REPORT.md](docs/INTEGRATION_REPORT.md).
|
|
|
|
The repository also contains the original standalone
|
|
[Confluence crawler](#standalone-crawler), which is unchanged.
|
|
|
|
## Components
|
|
|
|
| Directory | Role | Developer guide |
|
|
| --- | --- | --- |
|
|
| `backend/` | FastAPI service: credentials in memory, Confluence/model upstreams, authoritative tool history, container lifecycle, artifact downloads, serves the frontend | [backend/README.md](backend/README.md) |
|
|
| `agent/` | Container image: pinned pi SDK bridge, Python supervisor, local tools, artifact exporter | [agent/README.md](agent/README.md) |
|
|
| `frontend/` | Static UI (vendored marked + DOMPurify), same-origin mock server for UI development | [frontend/README.md](frontend/README.md) |
|
|
| `tests/integration/` | Integration-stage tests connecting the real components | this file |
|
|
| `deploy/`, `scripts/`, `Makefile` | Deployment configuration and entry points | this file |
|
|
|
|
## Requirements
|
|
|
|
- Linux with a **rootless** Docker daemon (cgroup v2, memory/CPU/PID limits enforced). `docker info` must list `rootless` under Security Options.
|
|
- Python 3.12+ in `.venv` (verified with 3.14.7), Node 22+ (verified with 26.7) and npm.
|
|
- An OpenAI-compatible Chat Completions endpoint the backend can reach (key held by the backend only).
|
|
- Network access from the backend host to the approved Confluence origin, optionally through a SOCKS5/HTTP proxy.
|
|
- For browser checks: Google Chrome with `--remote-debugging-port=9444`.
|
|
|
|
## Setup
|
|
|
|
```bash
|
|
python3 -m venv .venv && .venv/bin/python -m pip install -r requirements.txt
|
|
make install # also installs agent/ and frontend/ dev dependencies
|
|
make build-image # builds confluence-pi-agent:rev1 and prints its image ID
|
|
cp deploy/confluence-web.env.example deploy/confluence-web.env # then edit
|
|
```
|
|
|
|
`deploy/confluence-web.env` is git-ignored. Every variable is documented in the
|
|
example file and in [backend/README.md](backend/README.md). Pin
|
|
`CONFLUENCE_WEB_RUNTIME_IMAGE` to the image ID printed by `make build-image` for
|
|
an exact runtime, set a deployment-unique `CONFLUENCE_WEB_CONTAINER_LABEL_VALUE`,
|
|
and list only approved Confluence origins.
|
|
|
|
## Run
|
|
|
|
```bash
|
|
make run # scripts/run-backend.sh deploy/confluence-web.env
|
|
```
|
|
|
|
The backend binds `127.0.0.1:8000` by default and serves the UI at
|
|
`http://127.0.0.1:8000/`. Open it in a browser, click the key icon, enter the
|
|
Confluence base URL and your personal access token (PAT), test the connection,
|
|
then ask a question. Credentials live only in browser memory and in the backend
|
|
for the duration of a request; a page reload clears them.
|
|
|
|
Version 1 is a single-user deployment: one backend worker, one query at a time
|
|
(a second query gets `busy`), bound to loopback. Do not bind it to a network
|
|
interface without an authenticating reverse proxy in front; if you add one, it
|
|
must accept `6*16 MiB + 64 KiB` request bodies, pass client aborts through
|
|
promptly, avoid buffering bodies to disk, and allow at least 200 s per request.
|
|
|
|
Network-free UI development against explicit fakes (no Docker, PAT `dev-pat`,
|
|
URL `https://approved.example.com`):
|
|
|
|
```bash
|
|
make run-dev
|
|
```
|
|
|
|
## Credentials, expiry and limits
|
|
|
|
- **Confluence PAT**: entered in the browser per session, sent only to the backend over the same origin, never stored, logged, or passed to the container. The token inherits your Confluence permissions.
|
|
- **Model key**: backend environment only (`CONFLUENCE_WEB_MODEL_API_KEY`). For a local llama.cpp server any placeholder works.
|
|
- **Session cookie** `cw_session`: HttpOnly, SameSite=Strict, marks artifact ownership only. It is not authentication.
|
|
- **Artifacts**: at most 20 files, 10 MiB each, 50 MiB per query, 500 MiB total; downloadable for 15 minutes after a successful query, then deleted. Failed or cancelled queries keep nothing.
|
|
- **Query**: 180 s total deadline plus 10 s cleanup, prompt up to 16 MiB, answer up to 128 MiB, 100 Confluence calls and 50 model calls per query, container limited to 1 GiB RAM, 1 CPU, 128 processes, no network.
|
|
- **Model tokens**: `CONFLUENCE_WEB_MODEL_CONTEXT_WINDOW_TOKENS` and `..._MAX_OUTPUT_TOKENS` describe the provider; they are independent of the byte limits above. Large pages or answers can exceed the model's context before the application limits; the UI then shows `model_context_exceeded` or `model_output_limit`.
|
|
|
|
## Checks
|
|
|
|
| Command | What it runs | Needs |
|
|
| --- | --- | --- |
|
|
| `make check` | Crawler + backend + integration (no Docker) pytest, agent host tests, frontend unit/contract tests | nothing external |
|
|
| `make check-docker` | Backend real-Docker checks, backend + real runtime image + scripted peers, real provider adapter + runtime, runtime image/isolation/boundary checks | rootless Docker, built images |
|
|
| `make check-browser` | Real Chrome against the real backend (scripted runtime), frontend's own e2e suite | Chrome on `127.0.0.1:9444`, Node |
|
|
|
|
Each subsystem's own commands are documented in its guide and can be run independently:
|
|
|
|
```bash
|
|
CONFLUENCE_PAT=synthetic-token-no-network CONFLUENCE_URL=https://approved.example.com .venv/bin/python -m pytest tests/backend
|
|
npm --prefix agent test && npm --prefix agent run proof
|
|
npm --prefix frontend test && npm --prefix frontend run test:e2e
|
|
```
|
|
|
|
Fake modes (`CONFLUENCE_WEB_DEV_MODE`, the backend fake image, the frontend mock
|
|
server, the runtime's fake backend peer) are explicit opt-ins and never activate
|
|
on a production failure.
|
|
|
|
## Standalone crawler
|
|
|
|
Crawls Confluence Data Center pages into Markdown files using a PAT from `.env`
|
|
(`CONFLUENCE_URL`, `CONFLUENCE_PAT`; see `.env.example`). PATs require
|
|
Confluence Data Center / Server 7.9+.
|
|
|
|
```bash
|
|
python main.py --space KEY --out output/KEY # one space
|
|
python main.py --all --out output # all visible spaces
|
|
pytest -m live # live crawler tests (uses .env)
|
|
```
|
|
|
|
Each page becomes `output/<space>/<page_id>_<slug>.md` with YAML front matter.
|
|
The web backend imports only the crawler's pure conversion functions; it never
|
|
reads `.env`.
|