score-back/API.md

JSON-RPC API Documentation

The Cloud Point RPC server implements the JSON-RPC 2.0 protocol over TCP.

General Format

All requests and responses are JSON objects.

Request

{
  "jsonrpc": "2.0",
  "method": "<method_name>",
  "params": {},
  "id": <integer|string>
}

params is currently ignored by all handlers but is valid per JSON-RPC 2.0.

Response (Success)

{
  "jsonrpc": "2.0",
  "result": <method_specific_result>,
  "id": <matching_request_id>
}

Response (Error)

{
  "jsonrpc": "2.0",
  "error": {
    "code": <integer>,
    "message": "<string>"
  },
  "id": <matching_request_id>
}

---

Conventions

• Matrices: Row-major storage. A 3×3 matrix M with rows [r0, r1, r2] serialises as a flat 9-element JSON array [r0[0], r0[1], r0[2], r1[0], ...].
• Units: Translation in metres. No pixel units unless stated.
• Extrinsics convention (OpenCV): x_right = R · x_left + T. For a parallel rig with baseline b, R = I and T = [-b, 0, 0] (e.g. [-0.06, 0, 0] for a 6 cm baseline).
• Image layout: Top-left origin, row-major, packed channels (BGR order unless otherwise noted). Unity must flip GPU readback vertically before encoding.
• Image data encoding: Raw pixel bytes encoded as Base64 (standard alphabet, no line breaks). The type field indicates channel layout.
• Calibration arrays: Plain JSON double arrays — not base64. Only image pixel data uses base64.

---

Methods Served by the Unity Side

These methods are implemented server-side (in Unity, or in the C++ test mock in src/server_main.cpp).

---

get-available-methods

Returns the list of method names registered on this server instance.

Request:

{ "jsonrpc": "2.0", "method": "get-available-methods", "id": 0 }

Response result: ["method-name-1", "method-name-2", ...]

---

get-stereo-calibration

Returns full stereo rig calibration: intrinsics for both cameras, stereo rotation and translation.

Request:

{ "jsonrpc": "2.0", "method": "get-stereo-calibration", "id": 1 }

Response result:

{
  "left": {
    "camera_matrix": [fx, 0, cx,  0, fy, cy,  0, 0, 1],
    "dist_coeffs":   [k1, k2, p1, p2, k3]
  },
  "right": {
    "camera_matrix": [fx, 0, cx,  0, fy, cy,  0, 0, 1],
    "dist_coeffs":   [k1, k2, p1, p2, k3]
  },
  "rotation":    [r00, r01, r02,  r10, r11, r12,  r20, r21, r22],
  "translation": [tx, ty, tz],
  "image_size":  { "width": 640, "height": 480 }
}

Field details:

Field	Type	Size	Description
camera_matrix	double[]	9	Row-major 3×3 intrinsic matrix: [fx, 0, cx, 0, fy, cy, 0, 0, 1]
dist_coeffs	double[]	5	Radial/tangential coefficients [k1, k2, p1, p2, k3]
rotation	double[]	9	Row-major rotation matrix R (left-to-right frame, OpenCV convention)
translation	double[]	3	Translation vector in metres
image_size	object	—	Sensor resolution before any rectification

Test mock defaults: fx=fy=800, cx=320, cy=240, zero distortion, R=identity, T=[-0.06, 0, 0], 640×480.

---

get-image-pair

Returns a synchronised stereo frame as two base64-encoded images.

Request:

{ "jsonrpc": "2.0", "method": "get-image-pair", "id": 2 }

Response result:

{
  "frame": 42,
  "left":  { "width": 640, "height": 480, "type": "BGR", "data": "<base64>" },
  "right": { "width": 640, "height": 480, "type": "BGR", "data": "<base64>" }
}

Field details:

Field	Description
frame	Monotonically increasing counter per server instance; wraps at uint64_t max.
type	Channel layout: "BGR" (3 ch), "RGBA" (4 ch), or "DEPTH" (1 ch float32).
data	Base64-encoded raw pixel bytes. Size = width × height × channels.

Unity must supply images in top-left-origin row-major order; flip GPU readback vertically before encoding.

---

get-intrinsic-params (legacy)

Returns left-camera intrinsic matrix as a flat 9-element double array.

Request:

{ "jsonrpc": "2.0", "method": "get-intrinsic-params", "id": 3 }

Response result: [fx, 0, cx, 0, fy, cy, 0, 0, 1]

9 plain JSON doubles, row-major 3×3. Not base64.

---

get-extrinsic-params (legacy)

Returns a flat 16-element double array (4×4 row-major extrinsic matrix).

Request:

{ "jsonrpc": "2.0", "method": "get-extrinsic-params", "id": 4 }

Response result: [r00, r01, r02, tx, r10, r11, r12, ty, r20, r21, r22, tz, 0, 0, 0, 1]

16 plain JSON doubles. Not base64.

---

Client-Side Outputs

These are computed locally by the C++ client (RpcClient) from data received via the methods above. They are not JSON-RPC methods callable on the server.

---

get-cloud-point (computed by client — spec in progress)

Reconstructs a dense 3-D point cloud from the rectified stereo pair. The RpcClient::get_cloud_point() method will call get-stereo-calibration and get-image-pair, run stereo rectification and disparity computation locally (OpenCV SGBM or CUDA stereo), and reproject to 3-D.

Expected future result shape (subject to change in Phase 2):

{
  "width":  <int>,
  "height": <int>,
  "data":   "<base64-encoded float32 XYZ triplets, row-major>"
}

data will encode width × height × 3 little-endian float32 values (x, y, z in metres per pixel, NaN for invalid/occluded depth). Full specification and encoding details are deferred to Phase 2.