vptyp/confluence_web
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
confluence_web/README.md
Artur Mukhamadiev 77768ba3ca credentials: offer the approved Confluence origin as a fixed choice 
The origin check is an exact match including the context path, so users had
to type "https://collab.lge.com/main" precisely. GET /api/v1/config now
returns the approved origins in canonical form (non-secret: they are the only
destinations the backend will talk to), and the UI swaps the URL text field
for a select listing them, keeping the element id, focus handling and the
Test connection flow unchanged. The text field remains the fallback when the
fetch fails. Backend validation of the submitted URL is untouched. Mock server
serves the endpoint; contract, API and e2e tests cover it.
2026-09-15 15:24:17 +03:00

154 lines
8.5 KiB
Markdown
Raw Blame History

# Confluence Research Web UI
A minimalist web UI that researches a Confluence Data Center instance with a
[pi](https://github.com/earendil-works/pi) agent running in a fresh, network-less
rootless Docker container per query, and returns a cited Markdown answer plus
optional files the agent produced. Each component documents its own interface
in `agent/README.md`, `backend/README.md` and `frontend/README.md`.
The repository also contains the original standalone
[Confluence crawler](#standalone-crawler), which is unchanged.
## Components
| Directory | Role | Developer guide |
| --- | --- | --- |
| `backend/` | FastAPI service: credentials in memory, Confluence/model upstreams, authoritative tool history, container lifecycle, artifact downloads, serves the frontend | [backend/README.md](backend/README.md) |
| `agent/` | Container image: pinned pi SDK bridge, Python supervisor, local tools, artifact exporter | [agent/README.md](agent/README.md) |
| `frontend/` | Static UI (vendored marked, DOMPurify and thinking-orbs engine), same-origin mock server for UI development | [frontend/README.md](frontend/README.md) |
| `tests/integration/` | Integration-stage tests connecting the real components | this file |
| `deploy/`, `scripts/`, `Makefile` | Deployment configuration and entry points | this file |
## Requirements
- Linux with a **rootless** Docker daemon (cgroup v2, memory/CPU/PID limits enforced). `docker info` must list `rootless` under Security Options.
- Python 3.12+ in `.venv` (verified with 3.14.7), Node 22+ (verified with 26.7) and npm.
- An OpenAI-compatible Chat Completions endpoint the backend can reach (key held by the backend only).
- Network access from the backend host to the approved Confluence origin, optionally through a SOCKS5/HTTP proxy.
- For browser checks: Google Chrome with `--remote-debugging-port=9444`.
## Setup
```bash
python3 -m venv .venv && .venv/bin/python -m pip install -r requirements.txt
make install # also installs agent/ and frontend/ dev dependencies
make build-image # builds confluence-pi-agent:rev1 and prints its image ID
cp deploy/confluence-web.env.example deploy/confluence-web.env # then edit
```
`deploy/confluence-web.env` is git-ignored. Every variable is documented in the
example file and in [backend/README.md](backend/README.md). Pin
`CONFLUENCE_WEB_RUNTIME_IMAGE` to the image ID printed by `make build-image` for
an exact runtime, set a deployment-unique `CONFLUENCE_WEB_CONTAINER_LABEL_VALUE`,
and list only approved Confluence origins.
## Run
```bash
make run # scripts/run-backend.sh deploy/confluence-web.env
```
The backend binds `127.0.0.1:8000` by default and serves the UI at
`http://127.0.0.1:8000/`. Open it in a browser, click the key icon, pick the
Confluence origin (the list comes from `CONFLUENCE_WEB_APPROVED_ORIGINS`, each
entry including its context path, for example `https://collab.lge.com/main`),
enter your personal access token (PAT), test the connection, then ask a question. On
instances that allow anonymous REST reads, "Test connection" proves the
destination is reachable but cannot prove the PAT is valid; a wrong token then
shows up as denied or missing pages. Credentials live only in browser memory and in the backend
for the duration of a request; a page reload clears them.
Version 1 is a small-team deployment: one backend worker, one query at a time
(a second query gets `busy`), loopback by default. The application has no login
of its own; anyone who can reach the port can run queries with their own PAT.
Never bind it to a network interface over plain HTTP, because every request
carries the user's PAT. If you put a reverse proxy in front, it must accept
`6*16 MiB + 64 KiB` request bodies, pass client aborts through promptly, avoid
buffering bodies to disk, and allow at least 200 s per request.
### Serving colleagues on an internal network (IP only, TLS)
Uvicorn terminates TLS itself; no reverse proxy or DNS name is needed. The
certificate comes from a private CA that is **name-constrained** to the listed
IP addresses, so colleagues who import it are trusting it for those addresses
only. A certificate issued by the same CA for any hostname or any other address
is rejected by browsers and by OpenSSL (`permitted subtree violation`).
```bash
make tls IP=172.26.10.20 # creates deploy/tls/{ca,server}.{key,crt}; git-ignored
```
Then in `deploy/confluence-web.env`:
```
CONFLUENCE_WEB_BIND_HOST=172.26.10.20
CONFLUENCE_WEB_TLS_CERT=./deploy/tls/server.crt
CONFLUENCE_WEB_TLS_KEY=./deploy/tls/server.key
```
Restart with `make run` and open `https://172.26.10.20:8000/`. Under HTTPS the
session cookie gains the `Secure` flag automatically and the same-origin check
expects `https://` origins.
Hand colleagues only `deploy/tls/ca.crt`, imported once as a trusted root
(Windows: double-click, "Trusted Root Certification Authorities"; macOS:
Keychain Access; Chrome on Linux: chrome://settings/certificates; Firefox:
Settings, Certificates, Authorities). They can read the constraint themselves in
the certificate details ("Name Constraints: Permitted IP ..."). Keep `ca.key`
and `server.key` on the server host. The server certificate is valid for 825
days (the browser maximum); rotate it with the same CA by deleting
`deploy/tls/server.*` and re-running `make tls` with the CA files kept, or
rotate everything by deleting the directory.
Network-free UI development against explicit fakes (no Docker, PAT `dev-pat`,
URL `https://approved.example.com`):
```bash
make run-dev
```
## Credentials, expiry and limits
- **Confluence PAT**: entered in the browser per session, sent only to the backend over the same origin, never stored, logged, or passed to the container. The token inherits your Confluence permissions.
- **Model key**: backend environment only (`CONFLUENCE_WEB_MODEL_API_KEY`). For a local llama.cpp server any placeholder works.
- **Session cookie** `cw_session`: HttpOnly, SameSite=Strict, marks artifact ownership only. It is not authentication.
- **Artifacts**: at most 20 files, 10 MiB each, 50 MiB per query, 500 MiB total; downloadable for 15 minutes after a successful query, then deleted. Failed or cancelled queries keep nothing.
- **Query**: 180 s total deadline by default (`CONFLUENCE_WEB_QUERY_TIMEOUT_SECONDS`, up to `CONFLUENCE_WEB_MAX_DEADLINE_SECONDS`, default 900 s) plus 10 s cleanup, prompt up to 16 MiB, answer up to 128 MiB, 100 Confluence calls and 50 model calls per query by default (`CONFLUENCE_WEB_MAX_CONFLUENCE_CALLS`, `CONFLUENCE_WEB_MAX_MODEL_CALLS`), container limited to 1 GiB RAM, 1 CPU, 128 processes, no network.
- **Model tokens**: `CONFLUENCE_WEB_MODEL_CONTEXT_WINDOW_TOKENS` and `..._MAX_OUTPUT_TOKENS` describe the provider; they are independent of the byte limits above. Large pages or answers can exceed the model's context before the application limits; the UI then shows `model_context_exceeded` or `model_output_limit`.
## Checks
| Command | What it runs | Needs |
| --- | --- | --- |
| `make check` | Crawler + backend + integration (no Docker) pytest, agent host tests, frontend unit/contract tests | nothing external |
| `make check-docker` | Backend real-Docker checks, backend + real runtime image + scripted peers, real provider adapter + runtime, runtime image/isolation/boundary checks | rootless Docker, built images |
| `make check-browser` | Real Chrome against the real backend (scripted runtime), frontend's own e2e suite | Chrome on `127.0.0.1:9444`, Node |
Each subsystem's own commands are documented in its guide and can be run independently:
```bash
CONFLUENCE_PAT=synthetic-token-no-network CONFLUENCE_URL=https://approved.example.com .venv/bin/python -m pytest tests/backend
npm --prefix agent test && npm --prefix agent run proof
npm --prefix frontend test && npm --prefix frontend run test:e2e
```
Fake modes (`CONFLUENCE_WEB_DEV_MODE`, the backend fake image, the frontend mock
server, the runtime's fake backend peer) are explicit opt-ins and never activate
on a production failure.
## Standalone crawler
Crawls Confluence Data Center pages into Markdown files using a PAT from `.env`
(`CONFLUENCE_URL`, `CONFLUENCE_PAT`; see `.env.example`). PATs require
Confluence Data Center / Server 7.9+.
```bash
python main.py --space KEY --out output/KEY # one space
python main.py --all --out output # all visible spaces
pytest -m live # live crawler tests (uses .env)
```
Each page becomes `output/<space>/<page_id>_<slug>.md` with YAML front matter.
The web backend imports only the crawler's pure conversion functions; it never
reads `.env`.
Reference in New Issue View Git Blame Copy Permalink