@startuml comm box ClientProcess #LightBlue Participant Caller Participant CloudPointClient Participant StereoRectifier Participant PointCloudBuilder Participant TCPClient end box box UnityProcess #LightGreen Participant TCPServer Participant CloudPointServer Participant MainThreadQueue Participant UnityWorld end box == Initialization == UnityWorld -> CloudPointServer : Awake() — crpc_init() activate CloudPointServer CloudPointServer -> TCPServer : start (await connections) activate TCPServer Caller -> CloudPointClient : connect() activate CloudPointClient CloudPointClient -> TCPClient : establish TCP connection TCPClient -> TCPServer : TCP handshake TCPServer -> TCPServer : spawn per-client handler thread (loops) CloudPointClient -> TCPClient : get-stereo-calibration TCPClient -> TCPServer : send request TCPServer -> CloudPointServer : dispatch CloudPointServer -> MainThreadQueue : enqueue task MainThreadQueue -> UnityWorld : Update() dequeues activate UnityWorld UnityWorld -> UnityWorld : read camera params UnityWorld -> MainThreadQueue : tcs.SetResult(calibration JSON) deactivate UnityWorld MainThreadQueue -> CloudPointServer : task complete CloudPointServer -> TCPServer : send response TCPServer -> TCPClient : response TCPClient -> CloudPointClient : calibration received CloudPointClient -> StereoRectifier : init(calibration)\ncv::stereoRectify + remap maps deactivate CloudPointClient == Per compute_cloud() call == Caller -> CloudPointClient : compute_cloud() activate CloudPointClient CloudPointClient -> TCPClient : get-image-pair TCPClient -> TCPServer : send request TCPServer -> CloudPointServer : dispatch CloudPointServer -> MainThreadQueue : enqueue task MainThreadQueue -> UnityWorld : Update() dequeues activate UnityWorld UnityWorld -> UnityWorld : Render() both cameras\nReadPixels + vertical flip\nBase64 encode UnityWorld -> MainThreadQueue : tcs.SetResult(image pair JSON) deactivate UnityWorld MainThreadQueue -> CloudPointServer : task complete CloudPointServer -> TCPServer : send response TCPServer -> TCPClient : response (base64 left + right) TCPClient -> CloudPointClient : image pair received CloudPointClient -> StereoRectifier : rectify(left, right) StereoRectifier -> StereoRectifier : cv::remap both images StereoRectifier -> CloudPointClient : rectified pair CloudPointClient -> PointCloudBuilder : build(rectified, Q) PointCloudBuilder -> PointCloudBuilder : cv::StereoSGBM disparity\n(1/16 scale)\ncv::reprojectImageTo3D\nNaN filter PointCloudBuilder -> CloudPointClient : PointCloud CloudPointClient -> Caller : std::expected deactivate CloudPointClient == Teardown == Caller -> CloudPointClient : destruct deactivate CloudPointClient UnityWorld -> CloudPointServer : OnDestroy() — crpc_deinit() deactivate CloudPointServer deactivate TCPServer @enduml