score-back/openwiki/architecture.md

Architecture

Overview

The system follows a layered architecture for JSON-RPC 2.0 communication over TCP:

┌──────────────────────────────────────────────────────┐
│  Application / Unity                                  │
│  (registers callbacks via C API or server_main.cpp)   │
├──────────────────────────────────────────────────────┤
│  RPC Layer                                            │
│  RpcServer (method dispatch, JSON-RPC 2.0 handling)   │
│  RpcClient (typed method calls via jsonrpccxx)        │
├──────────────────────────────────────────────────────┤
│  Service Layer                                        │
│  Service (mock camera data: intrinsics, extrinsics,   │
│  cloud points)                                        │
├──────────────────────────────────────────────────────┤
│  Transport Layer                                      │
│  TcpServer (accept loop, per-client threads)          │
│  TCPConnector (client-side connector for jsonrpccxx)  │
│  tcp_read (framed read with size prefix)              │
├──────────────────────────────────────────────────────┤
│  Serialization                                        │
│  serialize.hpp (size embedding, numeric (de)serialize)│
│  rpc_coder (Base64 encode/decode)                     │
├──────────────────────────────────────────────────────┤
│  Config                                               │
│  ConfigLoader (YAML → ServerConfig + TestData)        │
└──────────────────────────────────────────────────────┘

All C++ code lives in the score namespace. The rpc/ git submodule provides jsonrpcxx headers used by the client side.

Communication model

The PlantUML diagram at docs/communication_model.pu (rendered as docs/cm.png) describes the interaction flow:

1. Unity side initializes a CloudPointServer which starts a TcpServer awaiting connections.
2. Client side initializes a CloudPointClient which creates a TCPConnector and connects to the server.
3. When a Caller requests data, the client serializes a JSON-RPC request, sends it over TCP, and waits for the response.
4. The server reads the request, dispatches it to RpcServer, which calls the registered handler (either a C++ lambda or a C callback from Unity).
5. The handler result is serialized back as a JSON-RPC response and sent over TCP.
6. The client receives the response and returns typed data to the caller.

The Unity integration model uses a static queue: the server enqueues tasks, and the Unity main loop dequeues and executes them, then sets the return value. This avoids calling Unity APIs from non-main threads.

Wire framing

Every TCP message (both directions) uses a simple length-prefixed framing protocol:

• Header: 8 bytes — a uint64_t in native byte order containing the payload size.
• Payload: exactly payload_size bytes of JSON-RPC text.

This is implemented in:

• inplace_size_embedding() (include/cloud_point_rpc/serialize.hpp): Prepends the serialized uint64_t size to a std::string. Used by TcpServer::handle_client and TCPConnector::Send.
• tcp_read() (include/cloud_point_rpc/tcp_read.hpp): Reads the 8-byte header, deserializes the size, then reads the full payload — continuing to read if the payload arrives in multiple TCP segments.

// Sending: embed size before the JSON payload
std::string response = processor_(payload);
response += "\n";
inplace_size_embedding(response);
asio::write(*socket, asio::buffer(response));

// Receiving: read 8-byte header, then full payload
std::array<char, 8> header;
asio::read(socket, asio::buffer(header, header.size()));
uint64_t packet_size = deserialize<uint64_t>(v);
std::vector<char> payload(packet_size);
asio::read(socket, asio::buffer(payload));

TcpServer

include/cloud_point_rpc/tcp_server.hpp

• Accepts connections on a configurable IP/port using asio.
• Runs an accept thread (std::jthread) that blocks on acceptor_.accept().
• Each client connection is handled in its own std::jthread; finished threads are cleaned up via a std::list<pair<jthread, future<bool>>> with periodic remove_if.
• The RequestProcessor (a std::function<std::string(const std::string&)>) is called for each incoming request — typically RpcServer::process.
• stop() unblocks the accept loop by first connecting a dummy socket to the listening endpoint (to avoid a race with close()), then closing the acceptor.
• Thread safety: cliThrMtx_ guards the client thread list; acceptorMtx_ guards the acceptor.

Key source: include/cloud_point_rpc/tcp_server.hpp

RpcServer

include/cloud_point_rpc/rpc_server.hpp, src/rpc_server.cpp

• Maintains a std::map<std::string, Handler> of registered methods.
• Handler type: std::function<std::variant<nlohmann::json, std::string>(const nlohmann::json&)> — handlers can return either a JSON object or a raw string.
• Supports two registration overloads:
  • register_method(name, Handler) — for C++ lambdas/functors.
  • register_method(name, callback_t) — for C function pointers (rpc_string* (*)(rpc_string*)). The C callback receives the JSON params as a string and returns a string that is parsed as JSON if possible, or kept as a raw string otherwise.
• process(request_str) parses the JSON-RPC 2.0 request, validates required fields (jsonrpc, method, id), dispatches to the handler, and builds the response. Returns JSON-RPC error objects for parse errors (-32700), invalid requests (-32600), method not found (-32601), and server errors (-32000).

Known issue

Non-object JSON requests (arrays, strings, numbers, null) throw nlohmann::json::type_error instead of returning a -32600 Invalid Request error. This is documented in tests/test_rpc_edge_cases.cpp.

RpcClient

include/cloud_point_rpc/rpc_client.hpp

• Extends jsonrpccxx::JsonRpcClient from the json-rpc-cxx submodule.
• Wraps a TCPConnector as the transport connector.
• Provides typed methods: get_intrinsic_params(), get_extrinsic_params(), get_cloud_point().
• Template call<ReturnType>(name) wraps CallMethod with an auto-incrementing request ID.

TCPConnector

include/cloud_point_rpc/tcp_connector.hpp

• Implements jsonrpccxx::IClientConnector.
• On construction, opens a TCP connection to the server.
• Send(request) embeds the size prefix, writes the full message, then calls tcp_read() to receive the response.

Service

include/cloud_point_rpc/service.hpp, src/service.cpp

• Holds TestData (intrinsic params, extrinsic params, cloud point).
• Returns configured data if available, otherwise returns identity-matrix fallbacks.
• Used by server_main.cpp for the standalone mock server. The C API path does not use Service — Unity provides its own handlers via callbacks.

Config

include/cloud_point_rpc/config.hpp

• ConfigLoader::load(path) parses a YAML file into a Config struct.
• Config contains ServerConfig (ip, port) and TestData (camera parameters).
• Falls back to 127.0.0.1:8080 if no server section is present.
• Sample config: config.yml (server IP 127.0.0.1, port 9095).

Executables

Executable	Source	Description
cloud_point_rpc_server	src/server_main.cpp	Standalone server with mock Service data
cloud_point_rpc_cli	src/main.cpp	Interactive CLI client (menu-driven)
minimal_client	src/minimal_client.cpp	Minimal client that sends a hardcoded ping request

Shared libraries

Library	Sources	Description
libcloud_point_rpc	rpc_coder.cpp, rpc_server.cpp, server_api.cpp, service.cpp	Core RPC + server + config
libcloud_point_rpc_cli	cli.cpp	CLI client logic (links against core lib)
test_cloud_point	test_api.cpp	Test API library for method scheduling and auto-calling

Threading model

• Server: Accept thread + per-client handler threads (all std::jthread). Each client handler reads one request, processes it, sends the response, and exits. The server does not keep connections open for multiple round-trips per thread.
• C API server (server_api.cpp): The global RpcServer and TcpServer are guarded by server_mtx. The rpc_string garbage collector is guarded by gc_mtx.
• Test API (test_api.cpp): A TestThread runs a std::jthread with a condition-variable-driven loop that can auto-call registered methods at a configurable interval or process queued one-shot calls.