docs(openwiki): automate recurring documentation updates
- Add scheduled OpenWiki regeneration and pull-request workflow - Refresh generated wiki metadata, navigation, and source documentation - Add Doxygen configuration and ignore generated documentation output - Publish OpenWiki guidance for Codex and Claude agents TG-3 #ready-for-test
This commit is contained in:
parent
ea99cd62ee
commit
b8d8272f76
51
.github/workflows/openwiki-update.yml
vendored
Normal file
51
.github/workflows/openwiki-update.yml
vendored
Normal file
@ -0,0 +1,51 @@
|
||||
name: OpenWiki Update
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
schedule:
|
||||
- cron: "0 8 * * *"
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
update:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Check out repository
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: "22"
|
||||
|
||||
- name: Install OpenWiki
|
||||
run: npm install --global openwiki
|
||||
|
||||
- name: Run OpenWiki
|
||||
run: openwiki code --update --print
|
||||
env:
|
||||
OPENWIKI_PROVIDER: openrouter
|
||||
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
|
||||
OPENWIKI_MODEL_ID: z-ai/glm-5.2
|
||||
LANGSMITH_API_KEY: ${{ secrets.LANGSMITH_API_KEY }}
|
||||
LANGCHAIN_PROJECT: openwiki
|
||||
LANGCHAIN_TRACING_V2: "true"
|
||||
|
||||
- name: Create OpenWiki update pull request
|
||||
uses: peter-evans/create-pull-request@22a9089034f40e5a961c8808d113e2c98fb63676 # v7
|
||||
with:
|
||||
add-paths: |
|
||||
openwiki
|
||||
AGENTS.md
|
||||
CLAUDE.md
|
||||
.github/workflows/openwiki-update.yml
|
||||
branch: openwiki/update
|
||||
commit-message: "docs: update OpenWiki"
|
||||
title: "docs: update OpenWiki"
|
||||
body: |
|
||||
Automated OpenWiki documentation update.
|
||||
|
||||
This PR was generated by the scheduled OpenWiki workflow.
|
||||
3
.gitignore
vendored
3
.gitignore
vendored
@ -9,6 +9,9 @@ subprojects/yaml-cpp-0.8.0
|
||||
subprojects/base64-0.5.2/
|
||||
subprojects/stdexec/
|
||||
subprojects/.*
|
||||
html/
|
||||
latex/
|
||||
large_tool_results/
|
||||
.venv/
|
||||
.worktrees/
|
||||
|
||||
|
||||
10
AGENTS.md
10
AGENTS.md
@ -161,3 +161,13 @@ Start here:
|
||||
OpenWiki includes repository overview, architecture notes, workflows, domain concepts, operations, integrations, testing guidance, and source maps.
|
||||
|
||||
When working in this repository, read the OpenWiki quickstart first, then follow its links to the relevant architecture, workflow, domain, operation, and testing notes.
|
||||
|
||||
<!-- OPENWIKI:START -->
|
||||
|
||||
## OpenWiki
|
||||
|
||||
This repository uses OpenWiki for recurring code documentation. Start with `openwiki/quickstart.md`, then follow its links to architecture, workflows, domain concepts, operations, integrations, testing guidance, and source maps.
|
||||
|
||||
The scheduled OpenWiki GitHub Actions workflow refreshes the repository wiki. Do not hand-edit generated OpenWiki pages unless explicitly asked; prefer updating source code/docs and letting OpenWiki regenerate.
|
||||
|
||||
<!-- OPENWIKI:END -->
|
||||
|
||||
9
CLAUDE.md
Normal file
9
CLAUDE.md
Normal file
@ -0,0 +1,9 @@
|
||||
<!-- OPENWIKI:START -->
|
||||
|
||||
## OpenWiki
|
||||
|
||||
This repository uses OpenWiki for recurring code documentation. Start with `openwiki/quickstart.md`, then follow its links to architecture, workflows, domain concepts, operations, integrations, testing guidance, and source maps.
|
||||
|
||||
The scheduled OpenWiki GitHub Actions workflow refreshes the repository wiki. Do not hand-edit generated OpenWiki pages unless explicitly asked; prefer updating source code/docs and letting OpenWiki regenerate.
|
||||
|
||||
<!-- OPENWIKI:END -->
|
||||
@ -1,6 +1,6 @@
|
||||
{
|
||||
"updatedAt": "2026-07-02T22:45:41.669Z",
|
||||
"command": "init",
|
||||
"gitHead": "9de6f5a82d5e9cc1c1d52b60a5493f71ede6e700",
|
||||
"updatedAt": "2026-07-16T21:56:19.566Z",
|
||||
"command": "update",
|
||||
"gitHead": "ea99cd62eec0605661c2d8237a16000d0aae7839",
|
||||
"model": "glm-5.2:cloud"
|
||||
}
|
||||
|
||||
1
openwiki/INSTRUCTIONS.md
Normal file
1
openwiki/INSTRUCTIONS.md
Normal file
@ -0,0 +1 @@
|
||||
A code wiki for this local repository. Prioritize a concise quickstart, architecture overview, source map, key workflows, domain concepts, operations/runbook notes, testing guidance, and integration points. Inspect git history to understand reasoning behind code changes and the progression of the repository. Keep pages grounded in the repository structure and recent code changes. Prefer practical navigation for engineers over generic summaries.
|
||||
@ -1,3 +1,10 @@
|
||||
---
|
||||
type: Architecture
|
||||
title: Architecture
|
||||
description: Layered architecture for JSON-RPC 2.0 over TCP, including the OpenCV compute layer, RPC/server/transport layers, threading model, and the SCARED dataset validation server.
|
||||
tags: [architecture, tcp, threading, opencv, stereo-pipeline]
|
||||
---
|
||||
|
||||
# Architecture
|
||||
|
||||
## Overview
|
||||
@ -12,6 +19,8 @@ The system follows a layered architecture for JSON-RPC 2.0 communication over TC
|
||||
│ OpenCV Compute Layer (optional — requires opencv4) │
|
||||
│ CloudPointClient (facade: connect, compute_cloud, │
|
||||
│ write_ply) │
|
||||
│ ScaredDatasetLoader (SCARED stereo calibration + │
|
||||
│ image pair from disk; mm→m translation) │
|
||||
│ StereoRectifier (cv::stereoRectify + remap, Q mat) │
|
||||
│ PointCloudBuilder (SGBM disparity, reproject, NaN │
|
||||
│ filter) │
|
||||
@ -42,7 +51,9 @@ All C++ code lives in the `score` namespace. The `rpc/` git submodule provides `
|
||||
|
||||
## Communication model
|
||||
|
||||
The PlantUML diagram at `docs/communication_model.pu` (rendered as `docs/cm.png`) describes the interaction flow:
|
||||
The PlantUML diagram at `docs/communication_model.pu` describes the interaction flow:
|
||||
|
||||
|
||||
|
||||
1. **Unity side** initializes a `CloudPointServer` via `crpc_init()`, which starts a `TcpServer` awaiting connections.
|
||||
2. **Client side** calls `CloudPointClient::connect()`, which opens a TCP connection and immediately fetches `get-stereo-calibration` once, initialising `StereoRectifier` with the returned intrinsics and geometry.
|
||||
@ -149,12 +160,23 @@ Non-object JSON requests (arrays, strings, numbers, null) throw `nlohmann::json:
|
||||
| `cloud_point_rpc_server` | `src/server_main.cpp` | Standalone server with mock `Service` data |
|
||||
| `cloud_point_rpc_cli` | `src/main.cpp` | Interactive CLI client (menu-driven) |
|
||||
| `minimal_client` | `src/minimal_client.cpp` | Minimal client that sends a hardcoded `ping` request |
|
||||
| `scared_dataset_server` | `src/cloud_point/scared_dataset_server.cpp` | RPC server backed by a SCARED dataset keyframe directory for real-data validation (requires opencv4) |
|
||||
|
||||
### SCARED dataset validation
|
||||
|
||||
`scared_dataset_server` (`src/cloud_point/scared_dataset_server.cpp`) is a standalone RPC server that serves `get-stereo-calibration` and `get-image-pair` from a SCARED [endoscopic stereo dataset](https://huggingface.co/datasets/maxhallan7/scared) keyframe directory, so the full `CloudPointClient` stereo pipeline can be validated against real data instead of mock images. It reuses the standard wire protocol (see [RPC Protocol](rpc-protocol.md)) — no new RPC methods.
|
||||
|
||||
`ScaredDatasetLoader` (`include/cloud_point/scared_dataset_loader.hpp`) reads `endoscope_calibration.yaml` (OpenCV FileStorage with `M1`, `D1`, `M2`, `D2`, `R`, `T`) plus `Left_Image.png` / `Right_Image.png` (1280×1024 RGBA) from a keyframe directory. The YAML `T` is stored in millimetres; the loader divides by 1000 before populating `StereoCalibrationRPC.translation` (metres on the wire). The same images are returned on every `get-image-pair` call (single-keyframe source); the frame counter is `std::atomic<uint64_t>` (incremented per call) so it is safe under TcpServer's per-client handler threads. The constructor throws `std::runtime_error` if the left and right images have different dimensions.
|
||||
|
||||
Usage: `scared_dataset_server <keyframe_dir> [port]` (default port 8080).
|
||||
|
||||
The SCARED rig (fx ≈ 1024 px, baseline ≈ 4.35 mm) produces disparities above 128 px for tissue nearer than ~35 mm. `CloudPointClient` accepts a `num_disparities` constructor parameter (default 128; use 160 for SCARED) which is validated and forwarded to `StereoMatcherFactory::create` — it must be a positive multiple of 16 or `std::invalid_argument` is thrown. See [Build & Testing → SCARED Dataset E2E Test](build-and-testing.md#scared-dataset-e2e-test) for the test and run instructions.
|
||||
|
||||
## Shared libraries
|
||||
|
||||
| Library | Sources | Description |
|
||||
|---|---|---|
|
||||
| `libcloud_point_rpc` | `rpc_coder.cpp`, `rpc_server.cpp`, `server_api.cpp`, `service.cpp` | Core RPC + server + config |
|
||||
| `libcloud_point_rpc` | `rpc_coder.cpp`, `rpc_dto.cpp`, `rpc_server.cpp`, `server_api.cpp`, `service.cpp` | Core RPC + server + config |
|
||||
| `libcloud_point_rpc_cli` | `cli.cpp` | CLI client logic (links against core lib) |
|
||||
| `test_cloud_point` | `test_api.cpp` | Test API library for method scheduling and auto-calling |
|
||||
|
||||
|
||||
@ -1,3 +1,10 @@
|
||||
---
|
||||
type: Reference
|
||||
title: Build & Testing
|
||||
description: Meson build system, dependencies, build targets, Linux/Windows build instructions, Docker, CI pipelines (Gitea + GitHub Actions), Doxygen, test suite overview, and SCARED dataset E2E test guidance.
|
||||
tags: [build, testing, meson, docker, ci, doxygen, scared]
|
||||
---
|
||||
|
||||
# Build & Testing
|
||||
|
||||
## Build system
|
||||
@ -26,9 +33,10 @@ Defined in `src/meson.build`:
|
||||
**Shared libraries:**
|
||||
| Library | Sources | Notes |
|
||||
|---|---|---|
|
||||
| `libcloud_point_rpc` | `rpc_coder.cpp`, `rpc_server.cpp`, `server_api.cpp`, `service.cpp` | Core library, installed with `install_rpath: '$ORIGIN'` |
|
||||
| `libcloud_point_rpc` | `rpc_coder.cpp`, `rpc_dto.cpp`, `rpc_server.cpp`, `server_api.cpp`, `service.cpp` | Core library, installed with `install_rpath: '$ORIGIN'` |
|
||||
| `libcloud_point_rpc_cli` | `cli.cpp` | CLI client logic |
|
||||
| `test_cloud_point` | `test_api.cpp` | Test API library |
|
||||
| `cloud_point_compute` | `src/cloud_point/*.cpp` | OpenCV compute library (optional — only built when `opencv4` is found). Links against `libcloud_point_rpc` + OpenCV. Sources: `image`, `cpu_stereo_matcher`, `gpu_stereo_matcher`, `stereo_matcher_factory`, `stereo_rectifier`, `point_cloud_builder`, `cloud_point_client`, `scared_dataset_loader`. |
|
||||
|
||||
**Executables:**
|
||||
| Executable | Source | Description |
|
||||
@ -36,6 +44,7 @@ Defined in `src/meson.build`:
|
||||
| `cloud_point_rpc_server` | `server_main.cpp` | Standalone mock server |
|
||||
| `cloud_point_rpc_cli` | `main.cpp` | Interactive CLI client |
|
||||
| `minimal_client` | `minimal_client.cpp` | Minimal client sending a hardcoded `ping` |
|
||||
| `scared_dataset_server` | `cloud_point/scared_dataset_server.cpp` | RPC server backed by a SCARED dataset keyframe directory (requires opencv4) |
|
||||
|
||||
### Linux build
|
||||
|
||||
@ -127,6 +136,16 @@ docker run --network=host -it -v $(pwd)/my_config.yaml:/app/config.yaml cloud-po
|
||||
4. `meson setup build && meson compile -C build -j2`
|
||||
5. `meson test -C build`
|
||||
|
||||
`.github/workflows/openwiki-update.yml` defines a GitHub Actions workflow that runs on a daily schedule (`0 8 * * *`) and on manual dispatch. It installs OpenWiki, runs `openwiki code --update --print` using the OpenRouter provider, and opens a pull request with the regenerated `openwiki/` content. This is how the repository wiki stays current without manual intervention.
|
||||
|
||||
## Doxygen
|
||||
|
||||
The `Doxyfile` configures Doxygen to generate API documentation from `openwiki/`, `docs/`, `include/`, `src/`, `README.md`, and `API.md`. HTML output goes to `html/` and LaTeX output to `latex/` (both git-ignored). Run with:
|
||||
|
||||
```bash
|
||||
doxygen Doxyfile
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
All tests are in `tests/` and compiled into a single `unit_tests` executable (defined in `tests/meson.build`) linked against `cloud_point_rpc_dep`, `cloud_point_rpc_cli_dep`, `cloud_point_rpc_test_dep`, and GoogleTest/GMock.
|
||||
@ -154,14 +173,21 @@ meson test -C build unit_tests # explicit
|
||||
| `test_base64.cpp` | Base64 | Encode/decode round-trip |
|
||||
| `test_base64_edge_cases.cpp` | Base64 | Edge cases (empty input, binary with nulls) |
|
||||
| `test_serialize.cpp` | Serialization | `serialize`/`deserialize` for numeric types, `inplace_size_embedding` |
|
||||
| `test_service.cpp` | Service | Default fallbacks, configured data, empty data |
|
||||
| `test_serialize_image.cpp` | Serialization | Image serialization round-trip |
|
||||
| `test_service.cpp` | Service | Default fallbacks, configured data, empty data, stereo calibration + image pair mocks |
|
||||
| `test_stereo_matcher.cpp` | Stereo matching | CPU/GPU stereo matcher factory, disparity output, `num_disparities` validation |
|
||||
| `test_stereo_rectifier.cpp` | Stereo rectification | `StereoRectifier` rectified image pair dimensions and validity |
|
||||
| `test_point_cloud_builder.cpp` | Point cloud | `PointCloudBuilder` SGBM → reproject → NaN filter pipeline |
|
||||
| `test_cloud_point_client.cpp` | CloudPointClient | End-to-end facade: connect, compute_cloud, PLY export |
|
||||
| `test_scared_dataset.cpp` | SCARED E2E | Full stereo pipeline against real SCARED endoscopic data (skipped unless `SCARED_KEYFRAME_DIR` is set) |
|
||||
| `test_image.cpp` | Image | `Image`/`ImageFactory` construction, Mat dimensions, pixel round-trip |
|
||||
|
||||
### Test conventions
|
||||
|
||||
- All test fixtures initialize Google Logging in `SetUp()` with `FLAGS_logtostderr = true`.
|
||||
- Integration tests (`test_integration.cpp`) create a temporary `config.yaml`, start a real `TcpServer` in a thread, and connect via `TCPConnector`/`RpcClient`.
|
||||
- Integration tests (`test_integration.cpp`) create a temporary `config.yaml`, start a real `TcpServer` in a thread, and connect via `TCPConnector`/`RpcClient`. Includes `ClientRetrieveRemoteMethods` which verifies the auto-registered `get-available-methods` method.
|
||||
- C API tests use `crpc_test_init()` / `crpc_test_deinit()` and verify callback invocation via `std::promise`/`std::future`.
|
||||
- The latest commit (`9de6f5a`) added `google::InitGoogleLogging` calls in test files to ensure logging is initialized before glog macros are used.
|
||||
- Stereo pipeline tests (`test_stereo_matcher`, `test_stereo_rectifier`, `test_point_cloud_builder`, `test_cloud_point_client`) require OpenCV and are only compiled when `opencv4` is found.
|
||||
|
||||
### Linting
|
||||
|
||||
@ -173,6 +199,35 @@ ninja -C build clang-format
|
||||
find src include tests -name "*.cpp" -o -name "*.hpp" | xargs clang-format -i
|
||||
```
|
||||
|
||||
## SCARED Dataset E2E Test
|
||||
|
||||
The `ScaredDatasetTest.ComputeCloudFromRealData` test (compiled when OpenCV is
|
||||
found) validates the full stereo pipeline against real endoscopic data from the
|
||||
[SCARED dataset](https://huggingface.co/datasets/maxhallan7/scared).
|
||||
|
||||
The test is **skipped in CI** (no dataset on CI runners). To run it locally:
|
||||
|
||||
```bash
|
||||
export SCARED_KEYFRAME_DIR=/path/to/test_dataset_8/keyframe_0
|
||||
./build/tests/unit_tests '--gtest_filter=ScaredDataset*'
|
||||
# or via meson (the test shows as skipped when env var is absent):
|
||||
meson test -C build -v
|
||||
```
|
||||
|
||||
The test asserts:
|
||||
- Cloud has more than 50,000 valid points.
|
||||
- Median z is in `[0.02, 0.20]` m (20 mm – 200 mm, typical endoscopy range).
|
||||
|
||||
**Important:** the YAML file stores `T` in millimetres (baseline ≈ −4.35 mm).
|
||||
`ScaredDatasetLoader` divides `T` by 1000 before populating
|
||||
`StereoCalibrationRPC.translation` (which is in metres on the wire).
|
||||
If the median depth looks ~1000× too large, the mm→m conversion is missing.
|
||||
|
||||
The SCARED rig has fx ≈ 1024 px and B ≈ 4.35 mm, giving a maximum disparity
|
||||
of only ~127 px at ~35 mm depth. `scared_dataset_server` and the E2E test
|
||||
both use `num_disparities = 160` via the `CloudPointClient` constructor's
|
||||
new `num_disparities` parameter (default 128 — backward-compatible).
|
||||
|
||||
## Source references
|
||||
|
||||
- `meson.build` — Root build config, dependency declarations
|
||||
|
||||
@ -1,3 +1,10 @@
|
||||
---
|
||||
type: API
|
||||
title: C API for Unity Integration
|
||||
description: C API surface for embedding the RPC server in Unity or other native consumers — lifecycle functions, rpc_string memory management, method registration, and the test API for handler testing.
|
||||
tags: [c-api, unity, server-api, test-api, rpc_string, integration]
|
||||
---
|
||||
|
||||
# C API for Unity Integration
|
||||
|
||||
The C API allows Unity (or any C/C++ consumer) to embed the RPC server as a shared library, register custom RPC handlers as C function pointers, and manage the server lifecycle without touching C++ directly.
|
||||
|
||||
13
openwiki/index.md
Normal file
13
openwiki/index.md
Normal file
@ -0,0 +1,13 @@
|
||||
---
|
||||
type: Documentation Index
|
||||
title: "OpenWiki"
|
||||
description: "Files and subdirectories in OpenWiki."
|
||||
---
|
||||
|
||||
# Files
|
||||
|
||||
- [Architecture](architecture.md) - Layered architecture for JSON-RPC 2.0 over TCP, including the OpenCV compute layer, RPC/server/transport layers, threading model, and the SCARED dataset validation server.
|
||||
- [Build & Testing](build-and-testing.md) - Meson build system, dependencies, build targets, Linux/Windows build instructions, Docker, CI pipelines (Gitea + GitHub Actions), Doxygen, test suite overview, and SCARED dataset E2E test guidance.
|
||||
- [C API for Unity Integration](c-api.md) - C API surface for embedding the RPC server in Unity or other native consumers — lifecycle functions, rpc_string memory management, method registration, and the test API for handler testing.
|
||||
- [Cloud Point RPC Quickstart](quickstart.md) - Entry point for the CloudPointRPC code wiki. Covers what the project is, repository layout, build/run instructions, and links to all major documentation sections.
|
||||
- [RPC Protocol](rpc-protocol.md) - JSON-RPC 2.0 wire protocol over TCP — request/response formats, error codes, all RPC methods (get-stereo-calibration, get-image-pair, get-available-methods, legacy methods), and handler registration.
|
||||
@ -1,3 +1,10 @@
|
||||
---
|
||||
type: Quickstart
|
||||
title: Cloud Point RPC Quickstart
|
||||
description: Entry point for the CloudPointRPC code wiki. Covers what the project is, repository layout, build/run instructions, and links to all major documentation sections.
|
||||
tags: [quickstart, overview, navigation]
|
||||
---
|
||||
|
||||
# Cloud Point RPC — Quickstart
|
||||
|
||||
## What is this?
|
||||
@ -24,6 +31,8 @@ The server side with C-API is fully implemented. The C++ stereo point-cloud clie
|
||||
| `config.yml` | Sample server configuration (IP and port) |
|
||||
| `Dockerfile` | Container image for the CLI client |
|
||||
| `.gitea/workflows/test.yaml` | CI pipeline (build + test on push to master) |
|
||||
| `.github/workflows/openwiki-update.yml` | Scheduled GitHub Actions workflow that refreshes OpenWiki docs daily and opens a PR |
|
||||
| `Doxyfile` | Doxygen config — generates HTML/LaTeX API docs from `openwiki/`, `docs/`, `include/`, `src/`, `README.md`, `API.md` (output in `html/` and `latex/`, git-ignored) |
|
||||
|
||||
## Build and run
|
||||
|
||||
|
||||
@ -1,3 +1,10 @@
|
||||
---
|
||||
type: Protocol
|
||||
title: RPC Protocol
|
||||
description: JSON-RPC 2.0 wire protocol over TCP — request/response formats, error codes, all RPC methods (get-stereo-calibration, get-image-pair, get-available-methods, legacy methods), and handler registration.
|
||||
tags: [rpc, json-rpc, protocol, api, wire-format]
|
||||
---
|
||||
|
||||
# RPC Protocol
|
||||
|
||||
## JSON-RPC 2.0
|
||||
@ -107,6 +114,26 @@ DTOs: `StereoCalibrationDto` and `ImagePairDto` in `include/cloud_point_rpc/rpc_
|
||||
|
||||
---
|
||||
|
||||
### `get-available-methods`
|
||||
|
||||
Returns the names of all methods registered on the server. Auto-registered in the `RpcServer` constructor — not added by the application. Useful for client-side discovery.
|
||||
|
||||
**Request:**
|
||||
```json
|
||||
{"jsonrpc": "2.0", "method": "get-available-methods", "id": 5}
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{"jsonrpc": "2.0", "result": ["get-available-methods", "get-stereo-calibration", "get-image-pair"], "id": 5}
|
||||
```
|
||||
|
||||
Result is a JSON array of method-name strings, including `get-available-methods` itself. Tested in `tests/test_integration.cpp` (`ClientRetrieveRemoteMethods`).
|
||||
|
||||
Source: `src/rpc_server.cpp` — `RpcServer::RpcServer()` constructor; `get_method_names()`.
|
||||
|
||||
---
|
||||
|
||||
### `get-intrinsic-params` *(legacy)*
|
||||
|
||||
Retrieves left-camera intrinsic parameters as a flat 3×3 matrix (row-major, 9 doubles). Not used by `CloudPointClient`; kept for backward compatibility.
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user