vptyp/confluence_web
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
confluence_web/README.md
Artur Mukhamadiev 6743c358f5 diagnostics: explain why a run failed instead of just "execution_failed" 
Every internal failure was mapped to a fixed sanitized code before anything
recorded the cause, so an intermittent execution_failed was undebuggable: an
exhausted model-call budget, a model turn with no text, and a genuine crash all
looked identical.

The bridge now writes one bounded line to container stderr on failure with the
code, the internal reason, the state and both call counters, and the failure
sites pass a reason (budget exhausted, empty final answer with its content block
types, token counts against the limits). The wire error is unchanged.

The backend logs the agent's terminal code together with its own call counters,
and the sanitized tail of container stderr rather than only its byte count.
deploy/logging.json gives every logger a timestamp (uvicorn's default config
leaves non-uvicorn loggers on logging's fallback handler); override with
CONFLUENCE_WEB_LOG_CONFIG.
2026-09-16 19:48:50 +03:00

9.1 KiB
Raw Blame History

Confluence Research Web UI

A minimalist web UI that researches a Confluence Data Center instance with a pi agent running in a fresh, network-less rootless Docker container per query, and returns a cited Markdown answer plus optional files the agent produced. Each component documents its own interface in agent/README.md, backend/README.md and frontend/README.md.

The repository also contains the original standalone Confluence crawler, which is unchanged.

Components

Directory Role Developer guide
backend/ FastAPI service: credentials in memory, Confluence/model upstreams, authoritative tool history, container lifecycle, artifact downloads, serves the frontend backend/README.md
agent/ Container image: pinned pi SDK bridge, Python supervisor, local tools, artifact exporter agent/README.md
frontend/ Static UI (vendored marked, DOMPurify and thinking-orbs engine), same-origin mock server for UI development frontend/README.md
tests/integration/ Integration-stage tests connecting the real components this file
deploy/, scripts/, Makefile Deployment configuration and entry points this file

Requirements

  • Linux with a rootless Docker daemon (cgroup v2, memory/CPU/PID limits enforced). docker info must list rootless under Security Options.
  • Python 3.12+ in .venv (verified with 3.14.7), Node 22+ (verified with 26.7) and npm.
  • An OpenAI-compatible Chat Completions endpoint the backend can reach (key held by the backend only).
  • Network access from the backend host to the approved Confluence origin, optionally through a SOCKS5/HTTP proxy.
  • For browser checks: Google Chrome with --remote-debugging-port=9444.

Setup

python3 -m venv .venv && .venv/bin/python -m pip install -r requirements.txt
make install                 # also installs agent/ and frontend/ dev dependencies
make build-image             # builds confluence-pi-agent:rev1 and prints its image ID
cp deploy/confluence-web.env.example deploy/confluence-web.env   # then edit

deploy/confluence-web.env is git-ignored. Every variable is documented in the example file and in backend/README.md. Pin CONFLUENCE_WEB_RUNTIME_IMAGE to the image ID printed by make build-image for an exact runtime, set a deployment-unique CONFLUENCE_WEB_CONTAINER_LABEL_VALUE, and list only approved Confluence origins.

Run

make run                     # scripts/run-backend.sh deploy/confluence-web.env

The backend binds 127.0.0.1:8000 by default and serves the UI at http://127.0.0.1:8000/. Open it in a browser, click the key icon, pick the Confluence origin (the list comes from CONFLUENCE_WEB_APPROVED_ORIGINS, each entry including its context path, for example https://collab.lge.com/main), enter your personal access token (PAT), test the connection, then ask a question. On instances that allow anonymous REST reads, "Test connection" proves the destination is reachable but cannot prove the PAT is valid; a wrong token then shows up as denied or missing pages. Credentials live only in browser memory and in the backend for the duration of a request; a page reload clears them.

Version 1 is a small-team deployment: one backend worker, one query at a time (a second query gets busy), loopback by default. The application has no login of its own; anyone who can reach the port can run queries with their own PAT. Never bind it to a network interface over plain HTTP, because every request carries the user's PAT. If you put a reverse proxy in front, it must accept 6*16 MiB + 64 KiB request bodies, pass client aborts through promptly, avoid buffering bodies to disk, and allow at least 200 s per request.

Serving colleagues on an internal network (IP only, TLS)

Uvicorn terminates TLS itself; no reverse proxy or DNS name is needed. The certificate comes from a private CA that is name-constrained to the listed IP addresses, so colleagues who import it are trusting it for those addresses only. A certificate issued by the same CA for any hostname or any other address is rejected by browsers and by OpenSSL (permitted subtree violation).

make tls IP=172.26.10.20          # creates deploy/tls/{ca,server}.{key,crt}; git-ignored

Then in deploy/confluence-web.env:

CONFLUENCE_WEB_BIND_HOST=172.26.10.20
CONFLUENCE_WEB_TLS_CERT=./deploy/tls/server.crt
CONFLUENCE_WEB_TLS_KEY=./deploy/tls/server.key

Restart with make run and open https://172.26.10.20:8000/. Under HTTPS the session cookie gains the Secure flag automatically and the same-origin check expects https:// origins.

Hand colleagues only deploy/tls/ca.crt, imported once as a trusted root (Windows: double-click, "Trusted Root Certification Authorities"; macOS: Keychain Access; Chrome on Linux: chrome://settings/certificates; Firefox: Settings, Certificates, Authorities). They can read the constraint themselves in the certificate details ("Name Constraints: Permitted IP ..."). Keep ca.key and server.key on the server host. The server certificate is valid for 825 days (the browser maximum); rotate it with the same CA by deleting deploy/tls/server.* and re-running make tls with the CA files kept, or rotate everything by deleting the directory.

Network-free UI development against explicit fakes (no Docker, PAT dev-pat, URL https://approved.example.com):

make run-dev

Credentials, expiry and limits

  • Confluence PAT: entered in the browser per session, sent only to the backend over the same origin, never stored, logged, or passed to the container. The token inherits your Confluence permissions.
  • Model key: backend environment only (CONFLUENCE_WEB_MODEL_API_KEY). For a local llama.cpp server any placeholder works.
  • Session cookie cw_session: HttpOnly, SameSite=Strict, marks artifact ownership only. It is not authentication.
  • Artifacts: at most 20 files, 10 MiB each, 50 MiB per query, 500 MiB total; downloadable for 15 minutes after a successful query, then deleted. Failed or cancelled queries keep nothing.
  • Query: 180 s total deadline by default (CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS, up to CONFLUENCE_WEB_MAX_DEADLINE_SECONDS, default 900 s) plus 10 s cleanup, prompt up to 16 MiB, answer up to 128 MiB, 100 Confluence calls and 50 model calls per query by default (CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS, CONFLUENCE_WEB_MAX_MODEL_CALLS), container limited to 1 GiB RAM, 1 CPU, 128 processes, no network.
  • Model tokens: CONFLUENCE_WEB_MODEL_CONTEXT_WINDOW_TOKENS and ..._MAX_OUTPUT_TOKENS describe the provider; they are independent of the byte limits above. Large pages or answers can exceed the model's context before the application limits; the UI then shows model_context_exceeded or model_output_limit.

Debugging a failed query

execution_failed in the browser is deliberately vague; the backend log has the reason. Run make run in a terminal you can read (or redirect it to a file) and look for the two lines a failed query emits: the agent's terminal code with the call counters, and [agent] run failed code=... reason=... from the container's stderr. Common causes: the per-query model call budget running out on a long multi-step research prompt (CONFLUENCE_WEB_MAX_MODEL_CALLS), and a model ending its final turn with no text. Neither is a timeout, so raising the deadline does not help either one.

Checks

Command What it runs Needs
make check Crawler + backend + integration (no Docker) pytest, agent host tests, frontend unit/contract tests nothing external
make check-docker Backend real-Docker checks, backend + real runtime image + scripted peers, real provider adapter + runtime, runtime image/isolation/boundary checks rootless Docker, built images
make check-browser Real Chrome against the real backend (scripted runtime), frontend's own e2e suite Chrome on 127.0.0.1:9444, Node

Each subsystem's own commands are documented in its guide and can be run independently:

CONFLUENCE_PAT=synthetic-token-no-network CONFLUENCE_URL=https://approved.example.com .venv/bin/python -m pytest tests/backend
npm --prefix agent test && npm --prefix agent run proof
npm --prefix frontend test && npm --prefix frontend run test:e2e

Fake modes (CONFLUENCE_WEB_DEV_MODE, the backend fake image, the frontend mock server, the runtime's fake backend peer) are explicit opt-ins and never activate on a production failure.

Standalone crawler

Crawls Confluence Data Center pages into Markdown files using a PAT from .env (CONFLUENCE_URL, CONFLUENCE_PAT; see .env.example). PATs require Confluence Data Center / Server 7.9+.

python main.py --space KEY --out output/KEY   # one space
python main.py --all --out output             # all visible spaces
pytest -m live                                # live crawler tests (uses .env)

Each page becomes output/<space>/<page_id>_<slug>.md with YAML front matter. The web backend imports only the crawler's pure conversion functions; it never reads .env.