Compare commits

...

4 Commits

Author SHA1 Message Date
14ae0a901f feat(benchmark): summarize SCARED reconstruction depth
All checks were successful
Verification / Is-Buildable (push) Successful in 2m43s
- report valid coverage and depth percentiles even when XYZ ground truth is absent

- accept depth bounds and optional color-mapped PNG output for visual inspection

- retain accuracy metrics when point_cloud.obj is available

- add a batch helper that renders Markdown tables and optional JSON and PNG artifacts

- document benchmark inputs, dataset limits, and visualization tradeoffs
2026-09-14 10:43:29 +03:00
508570cb5c feat(cli): expose point cloud visualization controls
- add block-averaged PLY grid decimation with validity-aware vertex generation

- validate and load ply_stride and wls_filter from cloud_point configuration

- pass matcher tuning through CloudPointClient and apply both settings in CLI exports

- cover stride averaging, sparse blocks, invalid values, and config parsing
2026-09-14 10:43:29 +03:00
96bc82f0fb feat(stereo): add optional disparity post-filters
- add configurable median and edge-aware WLS filtering to the CPU SGBM matcher

- preserve SGBM rejection holes when WLS smooths valid disparities

- detect and link OpenCV ximgproc when available, with a graceful fallback otherwise

- forward CPU tuning through the matcher factory and cover validation and hole preservation
2026-09-14 10:43:29 +03:00
4f21e2ea38 fix(cloud_point): regularise SGBM and export viewable PLY meshes
- Configure StereoSGBM with P1/P2 penalties, 5x5 block, uniqueness ratio,
  speckle filter and left-right check; previously it ran unregularised and
  produced a heavy tail of low-disparity outliers reprojecting metres away
  (SCARED benchmark MAE3D 44 mm -> 0.8 mm, RMSE 289 mm -> 1.4 mm)
- Pass the requested disparity count to the CUDA SGM matcher (rounded to
  64/128/256) instead of a hard-coded 16
- Add optional `cloud_point` config section (algorithm, num_disparities,
  depth range) consumed by CLI options 4/5; ship config.scared.yml
- write_ply now emits a binary PLY with a grid-triangulated mesh so web
  viewers stop fabricating slivers from consecutive vertices
- Add config, matcher and PLY export tests; document in README
2026-09-14 10:43:24 +03:00
23 changed files with 1148 additions and 89 deletions

3
.gitignore vendored
View File

@ -17,3 +17,6 @@ large_tool_results/
# IDE # IDE
.idea/ .idea/
# Point-cloud exports
*.ply

104
README.md
View File

@ -151,38 +151,106 @@ Each keyframe directory contains:
/path/to/test_dataset_8/keyframe_0 8080 /path/to/test_dataset_8/keyframe_0 8080
``` ```
If port 8080 is already taken on your machine (Docker's `rootlesskit`
commonly holds it) pass another port and update `server.port` in the CLI
config accordingly. Connecting the CLI to a foreign service on 8080 shows up
as `invalid JSON response from server` / `std::bad_alloc` errors.
### Connecting with the CLI ### Connecting with the CLI
In a second terminal run the interactive CLI against the same host and port: In a second terminal run the interactive CLI with the SCARED-tuned config:
```bash ```bash
# Adjust ip/port in config.yaml if needed, then: ./build/src/cloud_point_rpc_cli config.scared.yml
./build/src/cloud_point_rpc_cli config.yaml # Option 4 — compute point cloud and print valid point count + bounding box
# Option 4 — compute point cloud and print valid point count # Option 5 — compute point cloud and save a triangulated PLY mesh
# Option 5 — compute point cloud and save to output.ply (inspect in MeshLab)
``` ```
`config.scared.yml` sets the optional `cloud_point` section that options 4/5
honour:
```yaml
cloud_point:
algorithm: cpu # "gpu" falls back to CPU when CUDA is unavailable
num_disparities: 160 # fx ~1024 px, baseline ~4.35 mm -> up to ~160 px
min_depth_m: 0.02 # endoscopic working range: 20 mm .. 300 mm
max_depth_m: 0.30
ply_stride: 1 # option 5: 4 = 16x smaller, block-averaged mesh
wls_filter: false # true = smoother mesh for viewers, less accurate
```
Depth limits are a physical bound on the scene: with this rig depth is
roughly `4.45 m / disparity_px`, so any mismatch with a disparity below
~15 px reprojects metres away. Without the section the CLI falls back to the
generic defaults (GPU, 128 disparities, 0.0110 m). The SGBM matcher itself
is configured with OpenCV's recommended smoothness penalties (P1 = 8·bs²,
P2 = 32·bs²), a 5×5 block, uniqueness ratio 10, speckle filtering and a
left-right consistency check; see `CpuStereoMatcher::Params`.
### Checking the result
Option 4 should report a bounding box with z inside roughly
`[0.03, 0.16]` m for `test_dataset_8` keyframes. Option 5 writes a binary
little-endian PLY containing every valid point **and** a mesh triangulated
from the pixel grid (triangles are dropped where the longest edge exceeds 5 % of the local
depth, so the mesh breaks at occlusions). The mesh is what makes web viewers
usable: viewers such as Meshy's online PLY viewer fabricate a triangle from
every three consecutive vertices of a vertex-only PLY, which draws long
slivers across the surface and makes a correct cloud look like a fan of
rays. Pass `PlyOptions{false, 0.0f, false}` to `write_ply` for a points-only ASCII
file.
**Surface roughness vs accuracy.** SGBM's sub-pixel disparity noise
(~0.2 px, correlated over several pixels) is ~1 mm of depth on this rig,
far more than the 0.07 mm lateral pixel pitch, so a mesh built from the raw
cloud is "hairy": face normals sit a median 45° off the camera axis on
tissue that faces the camera. Colour depth maps hide this; shaded mesh
viewers show it as fuzz. `wls_filter: true` applies OpenCV's edge-aware
WLS disparity filter (needs `opencv_ximgproc`, doubles matching time) and
brings the median normal to ~23° with neighbour depth jumps down from 0.15 mm
to 0.06 mm, but it also costs accuracy on SCARED (keyframe 1: MAE 0.84 →
0.92 mm, within 2 mm 80 → 78 %; dataset_3: MAE 1.75 → 2.20 mm). It is
therefore off by default: use it for pictures, not for measurements. SGBM's
own holes are never filled by the filter.
When the keyframe contains `point_cloud.obj`, run the benchmark to compare the When the keyframe contains `point_cloud.obj`, run the benchmark to compare the
reconstruction with its pixel-aligned XYZ ground truth: reconstruction with its pixel-aligned XYZ ground truth:
```bash ```bash
./build/src/cloud_point/scared_dataset_benchmark \ ./build/src/cloud_point/scared_dataset_benchmark \
/path/to/test_dataset_8/keyframe_0 160 /path/to/dataset_1/keyframe_1 160 [depth.png|-] [min_depth_m max_depth_m]
``` ```
The benchmark reports coverage, component-wise and 3-D errors, threshold The benchmark always reports the valid-point fraction, depth percentiles
accuracy, and matching/reconstruction timings as JSON. OBJ coordinates are and matching/reconstruction timings as JSON, and optionally writes a
converted from millimetres to metres and rectified into the same left-camera colour-mapped depth image (third argument, `-` to skip) for visual
frame as the reconstructed cloud before evaluation. inspection. The optional depth range applies the same filter as the CLI's
`cloud_point` section; without it the generic 0.0110 m defaults are used,
which lets a few residual mismatches at metres of depth inflate RMSE. When
`point_cloud.obj` is present it also reports coverage, component-wise and
3-D errors and threshold accuracy. Note that the `test_dataset_*` archives
ship without `point_cloud.obj`; ground truth is only in the full
`dataset_N.zip` archives (1340 GB each). The zips are served with HTTP
range support, so single keyframes can be extracted remotely with Python's
`zipfile` over a seekable HTTP file object instead of downloading the whole
archive.
**Disparity range caveat:** the CLI constructs `CloudPointClient` with the `scripts/scared_overview.py` runs the benchmark over many keyframes and
default of 128 disparity levels, while this rig (fx ≈ 1024 px, baseline prints a Markdown table:
≈ 4.35 mm) produces disparities above 128 px for tissue nearer than
~35 mm — those pixels silently drop out of the cloud. The E2E test ```bash
(`tests/test_scared_dataset.cpp`) passes `num_disparities = 160` for full scripts/scared_overview.py --png-dir out/depth --depth-range 0.02 0.30 \
coverage; programmatic consumers should do the same via the datasets/scared/dataset_1/keyframe_* datasets/scared/test_dataset_8/keyframe_*
`CloudPointClient` constructor. For the CLI's qualitative check the default ``` OBJ coordinates are
is fine (observed median depth ≈ 115 mm is well within range). converted from millimetres to metres and rectified into the same left-camera
frame as the reconstructed cloud before evaluation. Reference numbers for
`dataset_1/keyframe_1` with the tuned SGBM configuration: coverage ≈ 0.85,
MAE₃D ≈ 0.8 mm, RMSE₃D ≈ 1.4 mm, 80 % of points within 2 mm (the previous
unregularised SGBM gave MAE₃D ≈ 44 mm and RMSE₃D ≈ 289 mm).
The E2E test (`tests/test_scared_dataset.cpp`) exercises the same pipeline
with `num_disparities = 160` and asserts >50 000 valid points and a median
depth in `[0.02, 0.20]` m.
## Communication model ## Communication model

13
config.scared.yml Normal file
View File

@ -0,0 +1,13 @@
# CLI configuration for validating against a SCARED keyframe served by
# scared_dataset_server (default port 8080; pick another if 8080 is busy).
server:
ip: "127.0.0.1"
port: 8080
cloud_point:
algorithm: cpu # "gpu" falls back to CPU when CUDA is unavailable
num_disparities: 160 # fx ~1024 px, baseline ~4.35 mm -> up to ~160 px
min_depth_m: 0.02 # endoscopic working range: 20 mm .. 300 mm
max_depth_m: 0.30
ply_stride: 1 # option 5: 4 = 16x smaller mesh, block-averaged
wls_filter: false # true = smoother mesh for viewers, ~10 % less accurate

View File

@ -1,3 +1,11 @@
server: server:
ip: "127.0.0.1" ip: "127.0.0.1"
port: 9095 port: 9095
# Optional stereo reconstruction settings for CLI options 4/5.
# See config.scared.yml for values tuned to the SCARED endoscopic rig.
# cloud_point:
# algorithm: gpu # or cpu
# num_disparities: 128 # positive multiple of 16
# min_depth_m: 0.01
# max_depth_m: 10.0

View File

@ -29,15 +29,18 @@ class CloudPointClient {
/// @brief Construct client (does not connect). /// @brief Construct client (does not connect).
/// @param ip Server IP address. /// @param ip Server IP address.
/// @param port Server port. /// @param port Server port.
/// @param algo Stereo matching algorithm (GPU falls back to CPU if /// @param algo Stereo matching algorithm (GPU falls back to CPU
/// if
/// unavailable). /// unavailable).
/// @param opts Depth filtering options. /// @param opts Depth filtering options.
/// @param num_disparities SGBM disparity levels (default 128; use 160 for /// @param num_disparities SGBM disparity levels (default 128; use 160 for
/// small-baseline rigs such as SCARED). /// small-baseline rigs such as SCARED).
CloudPointClient(std::string ip, int port, /// @param matcher_params SGBM tuning / post-filters for the CPU matcher.
StereoAlgorithmType algo = StereoAlgorithmType::GPU, CloudPointClient(
PointCloudBuilder::Options opts = {}, std::string ip, int port,
int num_disparities = 128); StereoAlgorithmType algo = StereoAlgorithmType::GPU,
PointCloudBuilder::Options opts = {}, int num_disparities = 128,
CpuStereoMatcher::Params matcher_params = CpuStereoMatcher::Params{});
~CloudPointClient(); ~CloudPointClient();
@ -61,6 +64,7 @@ class CloudPointClient {
StereoAlgorithmType algo_; StereoAlgorithmType algo_;
PointCloudBuilder::Options opts_; PointCloudBuilder::Options opts_;
int num_disparities_; int num_disparities_;
CpuStereoMatcher::Params matcher_params_;
std::unique_ptr<TCPConnector> connector_; std::unique_ptr<TCPConnector> connector_;
std::unique_ptr<RpcClient> client_; std::unique_ptr<RpcClient> client_;
std::unique_ptr<StereoRectifier> rectifier_; std::unique_ptr<StereoRectifier> rectifier_;
@ -68,9 +72,41 @@ class CloudPointClient {
std::unique_ptr<PointCloudBuilder> builder_; std::unique_ptr<PointCloudBuilder> builder_;
}; };
/// @brief Write valid points as ASCII PLY (for MeshLab inspection). /// @brief PLY export settings.
struct PlyOptions {
/// Emit a triangle mesh built from the organised pixel grid. Web viewers
/// (e.g. Meshy) fabricate triangles from consecutive vertices of a
/// vertex-only PLY, which draws long slivers across the surface; a real
/// mesh renders correctly everywhere and still contains every point.
bool triangulate;
/// Reject triangles whose longest edge exceeds this fraction of the
/// triangle's mean depth (breaks the mesh at occlusion boundaries).
float max_edge_depth_ratio;
/// Write binary_little_endian instead of ASCII. A full 1280x1024 mesh
/// is ~40 MB in binary versus ~85 MB in ASCII and parses much faster.
bool binary;
/// Decimate the grid by this factor (1 = full resolution). Each output
/// vertex is the mean of the valid pixels in its stride x stride block
/// (blocks less than half valid are dropped), so stride 4 both shrinks
/// the mesh 16x and cuts per-pixel disparity noise ~4x. Web viewers
/// cope far better with such a mesh than with the raw 1.1 M vertices.
int stride;
PlyOptions() noexcept
: triangulate(true), max_edge_depth_ratio(0.05f), binary(true),
stride(1) {}
PlyOptions(bool tri, float ratio, bool bin = true, int step = 1) noexcept
: triangulate(tri), max_edge_depth_ratio(ratio), binary(bin),
stride(step) {}
};
/// @brief Write valid points (and optionally a grid-triangulated mesh) as
/// PLY for MeshLab / web-viewer inspection.
/// @param cloud Source point cloud. /// @param cloud Source point cloud.
/// @param path Output file path. /// @param path Output file path.
void write_ply(const PointCloud &cloud, const std::string &path); /// @param opts Export settings; defaults to a binary triangulated mesh.
/// @return Number of faces written (0 when not triangulating).
/// @throws std::invalid_argument if opts.stride < 1.
size_t write_ply(const PointCloud &cloud, const std::string &path,
const PlyOptions &opts = PlyOptions{});
} // namespace score } // namespace score

View File

@ -6,10 +6,51 @@
namespace score { namespace score {
/// @brief CPU-based stereo matcher using cv::StereoSGBM. /// @brief CPU-based stereo matcher using cv::StereoSGBM.
///
/// The matcher is configured with the smoothness penalties and post-filters
/// recommended by OpenCV (P1 = 8·bs², P2 = 32·bs², uniqueness ratio,
/// left-right consistency check, speckle filter). Without them SGBM
/// degenerates to unregularised winner-take-all block matching, which
/// produces a heavy tail of low-disparity outliers that reproject metres
/// away from the true surface.
class CpuStereoMatcher : public IStereoMatcher { class CpuStereoMatcher : public IStereoMatcher {
public: public:
/// @brief True when this build can apply the WLS disparity filter.
[[nodiscard]] static bool wls_available() noexcept;
/// @brief SGBM tuning parameters.
struct Params {
int block_size; ///< Odd matching block size (SADWindowSize).
int uniqueness_ratio; ///< Best/second-best cost margin (%).
int speckle_window_size; ///< Max blob size flagged as speckle (0 off).
int speckle_range; ///< Max disparity variation inside a blob.
int disp12_max_diff; ///< Max left-right disparity mismatch (px).
int pre_filter_cap; ///< x-derivative clipping value.
/// Median filter applied to the disparity map (0 disables, else odd
/// 3 or 5). Only trims isolated spikes; SGBM's sub-pixel noise is
/// correlated over several pixels, so prefer the WLS filter.
int median_kernel;
/// Edge-aware weighted-least-squares smoothing of the disparity
/// (cv::ximgproc::DisparityWLSFilter). Off by default: on SCARED it
/// halves the fine-scale surface roughness (mesh normals 45 -> 23
/// deg off-axis) but costs accuracy (MAE 0.84 -> 0.92 mm, within
/// 2 mm 80 -> 78 %) and a second SGBM pass (~2x matching time).
/// Enable for visualisation, keep off for measurement. Ignored,
/// with a warning, when OpenCV was built without ximgproc.
bool wls_filter;
double wls_lambda; ///< Smoothness weight (2000; 8000 over-smooths).
double wls_sigma; ///< Edge sensitivity (typical 0.8-2.0).
// Explicit constructor avoids a GCC limitation with nested-struct
// default-member-initialisers used as default function arguments.
Params() noexcept
: block_size(5), uniqueness_ratio(10), speckle_window_size(100),
speckle_range(2), disp12_max_diff(1), pre_filter_cap(31),
median_kernel(0), wls_filter(false), wls_lambda(2000.0),
wls_sigma(1.5) {}
};
CpuStereoMatcher(int min_disparity = 0, int num_disparities = 128, CpuStereoMatcher(int min_disparity = 0, int num_disparities = 128,
int block_size = 3); Params params = Params{});
~CpuStereoMatcher() override = default; ~CpuStereoMatcher() override = default;
[[nodiscard]] cv::Mat compute(const cv::Mat &left, [[nodiscard]] cv::Mat compute(const cv::Mat &left,
@ -17,6 +58,11 @@ class CpuStereoMatcher : public IStereoMatcher {
private: private:
cv::Ptr<cv::StereoSGBM> sgbm_; cv::Ptr<cv::StereoSGBM> sgbm_;
cv::Ptr<cv::StereoMatcher> right_matcher_; ///< Only with WLS.
cv::Ptr<cv::Algorithm> wls_; ///< DisparityWLSFilter.
int median_kernel_;
double wls_lambda_;
double wls_sigma_;
}; };
} // namespace score } // namespace score

View File

@ -6,15 +6,27 @@ namespace score {
/// @brief GPU-based stereo matcher using cv::cuda::StereoSGM. /// @brief GPU-based stereo matcher using cv::cuda::StereoSGM.
/// Falls back to runtime error if CUDA is unavailable. /// Falls back to runtime error if CUDA is unavailable.
///
/// cv::cuda::StereoSGM only supports 64, 128 or 256 disparity levels; the
/// requested count is rounded up to the next supported value.
class GpuStereoMatcher : public IStereoMatcher { class GpuStereoMatcher : public IStereoMatcher {
public: public:
GpuStereoMatcher(int min_disparity = 0, int num_disparities = 16, /// @param min_disparity Minimum disparity (px).
int block_size = 3); /// @param num_disparities Requested disparity levels (rounded up to
/// 64/128/256).
/// @param uniqueness_ratio Best/second-best cost margin (%).
GpuStereoMatcher(int min_disparity = 0, int num_disparities = 128,
int uniqueness_ratio = 10);
~GpuStereoMatcher() override = default; ~GpuStereoMatcher() override = default;
[[nodiscard]] cv::Mat compute(const cv::Mat &left, [[nodiscard]] cv::Mat compute(const cv::Mat &left,
const cv::Mat &right) override; const cv::Mat &right) override;
/// @brief Round a disparity count up to the nearest value supported by
/// cv::cuda::StereoSGM (64, 128 or 256).
/// @throws std::invalid_argument if num_disparities exceeds 256.
[[nodiscard]] static int supported_num_disparities(int num_disparities);
private: private:
#ifdef HAVE_OPENCV_CUDA #ifdef HAVE_OPENCV_CUDA
cv::Ptr<cv::cuda::StereoSGM> sgm_; cv::Ptr<cv::cuda::StereoSGM> sgm_;

View File

@ -1,5 +1,6 @@
#pragma once #pragma once
#include "cloud_point/cpu_stereo_matcher.hpp"
#include "cloud_point/stereo_matcher.hpp" #include "cloud_point/stereo_matcher.hpp"
#include <memory> #include <memory>
@ -12,12 +13,16 @@ class StereoMatcherFactory {
public: public:
/// @brief Create a stereo matcher of the requested type. /// @brief Create a stereo matcher of the requested type.
/// If GPU is requested but unavailable, falls back to CPU. /// If GPU is requested but unavailable, falls back to CPU.
/// @param num_disparities Number of disparity levels for SGBM (default 128). /// @param num_disparities Number of disparity levels for SGBM (default
/// 128).
/// Must be a positive multiple of 16. /// Must be a positive multiple of 16.
/// @param cpu_params SGBM tuning used for the CPU matcher (and the
/// CPU fallback of the GPU matcher).
/// @throws std::invalid_argument if @p num_disparities is not a positive /// @throws std::invalid_argument if @p num_disparities is not a positive
/// multiple of 16. /// multiple of 16.
[[nodiscard]] static std::unique_ptr<IStereoMatcher> [[nodiscard]] static std::unique_ptr<IStereoMatcher>
create(StereoAlgorithmType type, int num_disparities = 128); create(StereoAlgorithmType type, int num_disparities = 128,
CpuStereoMatcher::Params cpu_params = CpuStereoMatcher::Params{});
}; };
} // namespace score } // namespace score

View File

@ -5,6 +5,21 @@
#include <string> #include <string>
namespace score { namespace score {
/**
* @brief Stereo reconstruction settings for CLI options 4 and 5.
*
* Mirrors CloudPointConfig from config.hpp without pulling OpenCV into the
* CLI interface.
*/
struct CliStereoOptions {
bool use_gpu{true}; ///< GPU matcher (falls back to CPU) or CPU
int num_disparities{128}; ///< SGBM levels, positive multiple of 16
float min_depth_m{0.01f}; ///< Reject points nearer than this (m)
float max_depth_m{10.0f}; ///< Reject points farther than this (m)
int ply_stride{1}; ///< Grid decimation for PLY export
bool wls_filter{false}; ///< WLS disparity smoothing (visualisation)
};
/** /**
* @brief Runs the CLI client. * @brief Runs the CLI client.
* *
@ -12,9 +27,11 @@ namespace score {
* @param output Output stream (usually std::cout) * @param output Output stream (usually std::cout)
* @param ip Server IP * @param ip Server IP
* @param port Server Port * @param port Server Port
* @param stereo Stereo reconstruction settings (options 4/5)
* @return int exit code * @return int exit code
*/ */
int CRPC_EXPORT run_cli(std::istream &input, std::ostream &output, int CRPC_EXPORT run_cli(std::istream &input, std::ostream &output,
const std::string &ip, int port); const std::string &ip, int port,
const CliStereoOptions &stereo = CliStereoOptions{});
} // namespace score } // namespace score

View File

@ -19,9 +19,28 @@ struct TestData {
std::vector<std::vector<double>> cloud_point; std::vector<std::vector<double>> cloud_point;
}; };
/// @brief Stereo reconstruction settings consumed by the CLI (option 4/5).
///
/// Depth limits are physical bounds on the scene: anything reprojected
/// outside [min_depth_m, max_depth_m] is discarded as a mismatch. For the
/// SCARED endoscopic rig use roughly 0.020.3 m; the defaults are wide enough
/// for a generic Unity scene.
struct CloudPointConfig {
std::string algorithm{"gpu"}; ///< "gpu" (falls back to CPU) or "cpu"
int num_disparities{128}; ///< SGBM levels, positive multiple of 16
double min_depth_m{0.01};
double max_depth_m{10.0};
int ply_stride{1}; ///< Grid decimation for PLY export (1 = full res)
/// Edge-aware WLS smoothing of the disparity map (needs opencv
/// ximgproc). Smoother meshes, slightly lower accuracy, ~2x matching
/// time; intended for visualisation.
bool wls_filter{false};
};
struct Config { struct Config {
ServerConfig server; ServerConfig server;
TestData test_data; TestData test_data;
CloudPointConfig cloud_point;
}; };
class ConfigLoader { class ConfigLoader {
@ -61,6 +80,39 @@ class ConfigLoader {
LOG(WARNING) << "No 'test_data' section, using empty/defaults."; LOG(WARNING) << "No 'test_data' section, using empty/defaults.";
} }
// Cloud point (optional)
if (config["cloud_point"]) {
const auto &cp = config["cloud_point"];
CloudPointConfig d;
c.cloud_point.algorithm =
cp["algorithm"].as<std::string>(d.algorithm);
c.cloud_point.num_disparities =
cp["num_disparities"].as<int>(d.num_disparities);
c.cloud_point.min_depth_m =
cp["min_depth_m"].as<double>(d.min_depth_m);
c.cloud_point.max_depth_m =
cp["max_depth_m"].as<double>(d.max_depth_m);
c.cloud_point.ply_stride =
cp["ply_stride"].as<int>(d.ply_stride);
c.cloud_point.wls_filter =
cp["wls_filter"].as<bool>(d.wls_filter);
if (c.cloud_point.ply_stride < 1) {
throw std::runtime_error(
"cloud_point.ply_stride must be >= 1");
}
if (c.cloud_point.algorithm != "gpu" &&
c.cloud_point.algorithm != "cpu") {
throw std::runtime_error(
"cloud_point.algorithm must be \"gpu\" or \"cpu\"");
}
if (c.cloud_point.min_depth_m <= 0.0 ||
c.cloud_point.max_depth_m <= c.cloud_point.min_depth_m) {
throw std::runtime_error(
"cloud_point depth range must satisfy "
"0 < min_depth_m < max_depth_m");
}
}
return c; return c;
} catch (const YAML::Exception &e) { } catch (const YAML::Exception &e) {
LOG(ERROR) << "Failed to load config: " << e.what(); LOG(ERROR) << "Failed to load config: " << e.what();

83
scripts/scared_overview.py Executable file
View File

@ -0,0 +1,83 @@
#!/usr/bin/env python3
"""Run scared_dataset_benchmark over many keyframes and print a Markdown table.
Usage:
scripts/scared_overview.py [--bench PATH] [--disparities N] [--png-dir DIR]
[--json-dir DIR] [--depth-range MIN_M MAX_M]
KEYFRAME_DIR...
Each KEYFRAME_DIR must hold Left_Image.png, Right_Image.png and
endoscope_calibration.yaml. Accuracy columns are filled in only for keyframes
that also contain point_cloud.obj.
"""
import argparse
import json
import os
import subprocess
import sys
def run(bench, kf, disparities, png_dir, json_dir, depth_range):
label = "/".join(kf.rstrip("/").split("/")[-2:])
cmd = [bench, kf, str(disparities)]
if png_dir:
os.makedirs(png_dir, exist_ok=True)
cmd.append(os.path.join(png_dir, label.replace("/", "_") + ".png"))
else:
cmd.append("-")
if depth_range:
cmd += [str(depth_range[0]), str(depth_range[1])]
proc = subprocess.run(cmd, capture_output=True, text=True)
if proc.returncode != 0:
print(f"{label}: benchmark failed\n{proc.stderr}", file=sys.stderr)
return label, None
result = json.loads(proc.stdout)
if json_dir:
os.makedirs(json_dir, exist_ok=True)
with open(os.path.join(json_dir, label.replace("/", "_") + ".json"), "w") as f:
json.dump(result, f, indent=2)
return label, result
def fmt_mm(v):
return "" if v is None else f"{v * 1000:.2f}"
def fmt_pct(v):
return "" if v is None else f"{v * 100:.1f}"
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--bench", default="build-opencv/src/cloud_point/scared_dataset_benchmark")
ap.add_argument("--disparities", type=int, default=160)
ap.add_argument("--png-dir")
ap.add_argument("--json-dir")
ap.add_argument("--depth-range", nargs=2, type=float, metavar=("MIN_M", "MAX_M"),
help="depth filter in metres (default: builder defaults 0.01..10)")
ap.add_argument("keyframes", nargs="+")
args = ap.parse_args()
rows = [run(args.bench, kf, args.disparities, args.png_dir, args.json_dir,
args.depth_range)
for kf in args.keyframes]
print("| keyframe | valid % | z p5 / median / p95 (mm) | match ms | GT | coverage % | MAE3D mm | RMSE3D mm | median mm | <1 mm % | <2 mm % | <5 mm % |")
print("|---|---|---|---|---|---|---|---|---|---|---|---|")
for label, r in rows:
if r is None:
print(f"| {label} | failed | | | | | | | | | | |")
continue
z = f"{r['z_p05_m']*1000:.0f} / {r['z_median_m']*1000:.0f} / {r['z_p95_m']*1000:.0f}"
gt = r.get("has_ground_truth", False)
print("| {} | {} | {} | {:.0f} | {} | {} | {} | {} | {} | {} | {} | {} |".format(
label, fmt_pct(r["valid_fraction"]), z, r["matching_ms"],
"yes" if gt else "no",
fmt_pct(r.get("coverage")), fmt_mm(r.get("mae_3d_m")),
fmt_mm(r.get("rmse_3d_m")), fmt_mm(r.get("median_3d_m")),
fmt_pct(r.get("within_1mm")), fmt_pct(r.get("within_2mm")),
fmt_pct(r.get("within_5mm"))))
if __name__ == "__main__":
main()

View File

@ -41,7 +41,7 @@ std::string vector_to_string(const std::vector<std::vector<T>> &v) {
} }
int run_cli(std::istream &input, std::ostream &output, const std::string &ip, int run_cli(std::istream &input, std::ostream &output, const std::string &ip,
int port) { int port, const CliStereoOptions &stereo) {
try { try {
TCPConnector connector(ip, port); TCPConnector connector(ip, port);
RpcClient client(connector); RpcClient client(connector);
@ -86,7 +86,15 @@ int run_cli(std::istream &input, std::ostream &output, const std::string &ip,
} else if (choice == "4" || choice == "5") { } else if (choice == "4" || choice == "5") {
#ifdef HAVE_CLOUD_POINT_COMPUTE #ifdef HAVE_CLOUD_POINT_COMPUTE
try { try {
CloudPointClient cpc(ip, port, StereoAlgorithmType::GPU); const auto algo = stereo.use_gpu ? StereoAlgorithmType::GPU
: StereoAlgorithmType::CPU;
CpuStereoMatcher::Params matcher;
matcher.wls_filter = stereo.wls_filter;
CloudPointClient cpc(
ip, port, algo,
PointCloudBuilder::Options{stereo.min_depth_m,
stereo.max_depth_m},
stereo.num_disparities, matcher);
cpc.connect(); cpc.connect();
auto result = cpc.compute_cloud(); auto result = cpc.compute_cloud();
@ -127,9 +135,13 @@ int run_cli(std::istream &input, std::ostream &output, const std::string &ip,
output << "PLY output path: "; output << "PLY output path: ";
std::string path; std::string path;
if (input >> path) { if (input >> path) {
write_ply(cloud, path); PlyOptions ply;
output << "Saved " << valid.size() ply.stride = stereo.ply_stride;
<< " points to " << path << "\n"; const auto faces = write_ply(cloud, path, ply);
output << "Saved mesh (stride " << ply.stride
<< ", " << faces << " faces) from "
<< valid.size() << " valid points to "
<< path << "\n";
} }
} }
} }

View File

@ -3,18 +3,25 @@
#include "cloud_point/imageFactory.h" #include "cloud_point/imageFactory.h"
#include "cloud_point_rpc/rpc_client.hpp" #include "cloud_point_rpc/rpc_client.hpp"
#include "cloud_point_rpc/tcp_connector.hpp" #include "cloud_point_rpc/tcp_connector.hpp"
#include <algorithm>
#include <array>
#include <bit>
#include <cmath>
#include <fstream> #include <fstream>
#include <jsonrpccxx/common.hpp> #include <jsonrpccxx/common.hpp>
#include <opencv2/imgproc.hpp> #include <opencv2/imgproc.hpp>
#include <stdexcept>
#include <vector>
namespace score { namespace score {
CloudPointClient::CloudPointClient(std::string ip, int port, CloudPointClient::CloudPointClient(std::string ip, int port,
StereoAlgorithmType algo, StereoAlgorithmType algo,
PointCloudBuilder::Options opts, PointCloudBuilder::Options opts,
int num_disparities) int num_disparities,
CpuStereoMatcher::Params matcher_params)
: ip_(std::move(ip)), port_(port), algo_(algo), opts_(opts), : ip_(std::move(ip)), port_(port), algo_(algo), opts_(opts),
num_disparities_(num_disparities) {} num_disparities_(num_disparities), matcher_params_(matcher_params) {}
CloudPointClient::~CloudPointClient() = default; CloudPointClient::~CloudPointClient() = default;
@ -25,7 +32,8 @@ void CloudPointClient::connect() {
const auto calib_rpc = client_->get_stereo_calibration(); const auto calib_rpc = client_->get_stereo_calibration();
const auto calib = StereoRectifier::Calibration::from_rpc(calib_rpc); const auto calib = StereoRectifier::Calibration::from_rpc(calib_rpc);
rectifier_ = std::make_unique<StereoRectifier>(calib); rectifier_ = std::make_unique<StereoRectifier>(calib);
matcher_ = StereoMatcherFactory::create(algo_, num_disparities_); matcher_ = StereoMatcherFactory::create(algo_, num_disparities_,
matcher_params_);
builder_ = std::make_unique<PointCloudBuilder>(rectifier_->q(), opts_); builder_ = std::make_unique<PointCloudBuilder>(rectifier_->q(), opts_);
} }
@ -95,21 +103,164 @@ CloudPointClient::compute_cloud() {
// PLY helper // PLY helper
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
void write_ply(const PointCloud &cloud, const std::string &path) { namespace {
const auto valid = cloud.valid_points();
std::ofstream out(path); struct Vertex {
float x, y, z;
};
/// Longest edge of a triangle must not exceed max_ratio * mean depth.
bool triangle_ok(const Vertex &a, const Vertex &b, const Vertex &c,
float max_ratio) {
const auto dist2 = [](const Vertex &p, const Vertex &q) {
const float dx = p.x - q.x, dy = p.y - q.y, dz = p.z - q.z;
return dx * dx + dy * dy + dz * dz;
};
const float longest2 = std::max({dist2(a, b), dist2(b, c), dist2(c, a)});
const float limit = max_ratio * (a.z + b.z + c.z) / 3.0f;
return longest2 <= limit * limit;
}
} // namespace
size_t write_ply(const PointCloud &cloud, const std::string &path,
const PlyOptions &opts) {
if (opts.stride < 1) {
throw std::invalid_argument("write_ply: stride must be >= 1, got " +
std::to_string(opts.stride));
}
// Decimated grid dimensions; pixel (r, c) of the sub-grid maps to
// (r * stride, c * stride) of the source cloud.
const int grid_w = (cloud.width + opts.stride - 1) / opts.stride;
const int grid_h = (cloud.height + opts.stride - 1) / opts.stride;
// Map every valid sub-grid cell to its index in the vertex list. With
// stride > 1 each vertex is the mean of the valid pixels in its
// stride x stride block, which divides the per-pixel disparity noise by
// roughly the stride and yields a far smoother mesh than sub-sampling.
const size_t n_px =
static_cast<size_t>(grid_w) * static_cast<size_t>(grid_h);
std::vector<int> index(n_px, -1);
std::vector<Vertex> vertices;
vertices.reserve(n_px);
for (int gr = 0; gr < grid_h; ++gr) {
for (int gc = 0; gc < grid_w; ++gc) {
double sx = 0.0, sy = 0.0, sz = 0.0;
int count = 0;
const int row_begin = gr * opts.stride;
const int col_begin = gc * opts.stride;
const int row_end = std::min(row_begin + opts.stride, cloud.height);
const int col_end = std::min(col_begin + opts.stride, cloud.width);
for (int r = row_begin; r < row_end; ++r) {
for (int c = col_begin; c < col_end; ++c) {
const size_t src = (static_cast<size_t>(r) *
static_cast<size_t>(cloud.width) +
static_cast<size_t>(c)) *
3u;
const float x = cloud.data[src];
const float y = cloud.data[src + 1];
const float z = cloud.data[src + 2];
if (!std::isnan(x) && !std::isnan(y) && !std::isnan(z)) {
sx += x;
sy += y;
sz += z;
++count;
}
}
}
// Require at least half the block to be valid so that a lone
// pixel cannot fabricate a vertex inside a hole.
const int block = (row_end - row_begin) * (col_end - col_begin);
if (count * 2 >= block && count > 0) {
index[static_cast<size_t>(gr) * static_cast<size_t>(grid_w) +
static_cast<size_t>(gc)] =
static_cast<int>(vertices.size());
vertices.push_back({static_cast<float>(sx / count),
static_cast<float>(sy / count),
static_cast<float>(sz / count)});
}
}
}
// Grid triangulation: each 2x2 cell yields up to two triangles, wound so
// the normal faces the camera (-z in OpenCV coordinates).
std::vector<std::array<int, 3>> faces;
if (opts.triangulate) {
faces.reserve(2 * n_px);
const auto at = [&](int r, int c) {
return index[static_cast<size_t>(r) * static_cast<size_t>(grid_w) +
static_cast<size_t>(c)];
};
const auto emit = [&](int i0, int i1, int i2) {
if (i0 < 0 || i1 < 0 || i2 < 0)
return;
if (triangle_ok(vertices[static_cast<size_t>(i0)],
vertices[static_cast<size_t>(i1)],
vertices[static_cast<size_t>(i2)],
opts.max_edge_depth_ratio)) {
faces.push_back({i0, i1, i2});
}
};
for (int r = 0; r + 1 < grid_h; ++r) {
for (int c = 0; c + 1 < grid_w; ++c) {
const int i00 = at(r, c), i01 = at(r, c + 1);
const int i10 = at(r + 1, c), i11 = at(r + 1, c + 1);
const int valid =
(i00 >= 0) + (i01 >= 0) + (i10 >= 0) + (i11 >= 0);
if (valid == 4) {
emit(i00, i10, i11);
emit(i00, i11, i01);
} else if (valid == 3) {
// One missing corner: keep the single remaining triangle.
if (i00 < 0)
emit(i10, i11, i01);
else if (i01 < 0)
emit(i00, i10, i11);
else if (i10 < 0)
emit(i00, i11, i01);
else
emit(i00, i10, i01);
}
}
}
}
std::ofstream out(path, std::ios::binary);
out << "ply\n" out << "ply\n"
<< "format ascii 1.0\n" << (opts.binary ? "format binary_little_endian 1.0\n"
<< "element vertex " << valid.size() << "\n" : "format ascii 1.0\n")
<< "element vertex " << vertices.size() << "\n"
<< "property float x\n" << "property float x\n"
<< "property float y\n" << "property float y\n"
<< "property float z\n" << "property float z\n";
<< "end_header\n"; if (opts.triangulate) {
out << "element face " << faces.size() << "\n"
for (const auto &pt : valid) { << "property list uchar int vertex_indices\n";
out << pt[0] << " " << pt[1] << " " << pt[2] << "\n";
} }
out << "end_header\n";
if (opts.binary) {
static_assert(std::endian::native == std::endian::little,
"binary PLY writer assumes a little-endian host");
static_assert(sizeof(Vertex) == 3 * sizeof(float));
out.write(
reinterpret_cast<const char *>(vertices.data()),
static_cast<std::streamsize>(vertices.size() * sizeof(Vertex)));
for (const auto &f : faces) {
const unsigned char count = 3;
out.write(reinterpret_cast<const char *>(&count), 1);
out.write(reinterpret_cast<const char *>(f.data()),
static_cast<std::streamsize>(3 * sizeof(int)));
}
} else {
for (const auto &v : vertices) {
out << v.x << " " << v.y << " " << v.z << "\n";
}
for (const auto &f : faces) {
out << "3 " << f[0] << " " << f[1] << " " << f[2] << "\n";
}
}
return faces.size();
} }
} // namespace score } // namespace score

View File

@ -1,15 +1,102 @@
#include "cloud_point/cpu_stereo_matcher.hpp" #include "cloud_point/cpu_stereo_matcher.hpp"
#include <algorithm>
#include <glog/logging.h>
#include <opencv2/imgproc.hpp>
#include <stdexcept>
#include <string>
#ifdef CLOUD_POINT_HAVE_OPENCV_XIMGPROC
#include <opencv2/ximgproc/disparity_filter.hpp>
#endif
namespace score { namespace score {
bool CpuStereoMatcher::wls_available() noexcept {
#ifdef CLOUD_POINT_HAVE_OPENCV_XIMGPROC
return true;
#else
return false;
#endif
}
CpuStereoMatcher::CpuStereoMatcher(int min_disparity, int num_disparities, CpuStereoMatcher::CpuStereoMatcher(int min_disparity, int num_disparities,
int block_size) { Params params)
sgbm_ = cv::StereoSGBM::create(min_disparity, num_disparities, block_size); : median_kernel_(params.median_kernel), wls_lambda_(params.wls_lambda),
wls_sigma_(params.wls_sigma) {
if (params.median_kernel != 0 && params.median_kernel != 3 &&
params.median_kernel != 5) {
throw std::invalid_argument(
"CpuStereoMatcher: median_kernel must be 0, 3 or 5, got " +
std::to_string(params.median_kernel));
}
if (params.block_size < 1 || params.block_size % 2 == 0) {
throw std::invalid_argument(
"CpuStereoMatcher: block_size must be a positive odd number, got " +
std::to_string(params.block_size));
}
const int bs2 = params.block_size * params.block_size;
// OpenCV's recommended penalties for a single-channel input.
const int p1 = 8 * bs2;
const int p2 = 32 * bs2;
sgbm_ = cv::StereoSGBM::create(
min_disparity, num_disparities, params.block_size, p1, p2,
params.disp12_max_diff, params.pre_filter_cap, params.uniqueness_ratio,
params.speckle_window_size, params.speckle_range,
cv::StereoSGBM::MODE_SGBM);
if (params.wls_filter) {
#ifdef CLOUD_POINT_HAVE_OPENCV_XIMGPROC
// NB: createDisparityWLSFilter(sgbm_) would silently disable the
// left matcher's uniqueness/speckle/LRC post-filters to obtain a
// dense map for its own confidence estimate. The generic factory
// leaves our tuned matcher alone; we then keep SGBM's holes invalid.
right_matcher_ = cv::ximgproc::createRightMatcher(sgbm_);
auto wls = cv::ximgproc::createDisparityWLSFilterGeneric(
/*use_confidence=*/true);
wls->setLambda(params.wls_lambda);
wls->setSigmaColor(params.wls_sigma);
wls->setLRCthresh(24); // 1.5 px in SGBM's 16x fixed point
wls->setDepthDiscontinuityRadius(
std::max(1, params.block_size / 2 + 1));
wls_ = wls;
#else
LOG(WARNING) << "CpuStereoMatcher: WLS filter requested but OpenCV "
"was built without ximgproc; disparity is unfiltered";
#endif
}
} }
cv::Mat CpuStereoMatcher::compute(const cv::Mat &left, const cv::Mat &right) { cv::Mat CpuStereoMatcher::compute(const cv::Mat &left, const cv::Mat &right) {
cv::Mat disparity; cv::Mat disparity;
sgbm_->compute(left, right, disparity); sgbm_->compute(left, right, disparity);
#ifdef CLOUD_POINT_HAVE_OPENCV_XIMGPROC
// The WLS filter needs a valid ROI to the right of the disparity search
// range; skip it (raw SGBM output) for images narrower than that.
const int roi_width = sgbm_->getMinDisparity() +
sgbm_->getNumDisparities() + sgbm_->getBlockSize();
if (wls_ && left.cols > roi_width) {
// Snapshot SGBM's holes first: the filter may modify its inputs.
const cv::Mat holes = disparity <= 0;
cv::Mat right_disparity, filtered;
right_matcher_->compute(right, left, right_disparity);
auto wls = wls_.dynamicCast<cv::ximgproc::DisparityWLSFilter>();
wls->filter(disparity, left, filtered, right_disparity);
// WLS extrapolates into pixels SGBM rejected; keep those invalid so
// holes stay holes and coverage is not inflated with guesses.
filtered.setTo(cv::Scalar(sgbm_->getMinDisparity() * 16 - 16), holes);
disparity = filtered;
}
#endif
if (median_kernel_ > 0) {
cv::Mat filtered;
cv::medianBlur(disparity, filtered, median_kernel_);
disparity = filtered;
}
return disparity; return disparity;
} }

View File

@ -1,5 +1,6 @@
#include "cloud_point/gpu_stereo_matcher.hpp" #include "cloud_point/gpu_stereo_matcher.hpp"
#include <stdexcept> #include <stdexcept>
#include <string>
#ifdef HAVE_OPENCV_CUDA #ifdef HAVE_OPENCV_CUDA
#include <opencv2/cudaimgproc.hpp> #include <opencv2/cudaimgproc.hpp>
@ -8,18 +9,35 @@
namespace score { namespace score {
int GpuStereoMatcher::supported_num_disparities(int num_disparities) {
if (num_disparities <= 64)
return 64;
if (num_disparities <= 128)
return 128;
if (num_disparities <= 256)
return 256;
throw std::invalid_argument(
"GpuStereoMatcher: cv::cuda::StereoSGM supports at most 256 "
"disparities, got " +
std::to_string(num_disparities));
}
GpuStereoMatcher::GpuStereoMatcher(int min_disparity, int num_disparities, GpuStereoMatcher::GpuStereoMatcher(int min_disparity, int num_disparities,
int block_size) { int uniqueness_ratio) {
const int levels = supported_num_disparities(num_disparities);
#ifdef HAVE_OPENCV_CUDA #ifdef HAVE_OPENCV_CUDA
if (cv::cuda::getCudaEnabledDeviceCount() == 0) { if (cv::cuda::getCudaEnabledDeviceCount() == 0) {
throw std::runtime_error("No CUDA devices available"); throw std::runtime_error("No CUDA devices available");
} }
sgm_ = // P1/P2 follow the cv::cuda::StereoSGM defaults (10/120), which are
cv::cuda::createStereoSGM(min_disparity, num_disparities, block_size); // expressed on the census-transform cost scale rather than SAD.
sgm_ = cv::cuda::createStereoSGM(min_disparity, levels, /*P1=*/10,
/*P2=*/120, uniqueness_ratio,
cv::cuda::StereoSGM::MODE_HH4);
#else #else
(void)min_disparity; (void)min_disparity;
(void)num_disparities; (void)levels;
(void)block_size; (void)uniqueness_ratio;
throw std::runtime_error("OpenCV CUDA modules not available in this build"); throw std::runtime_error("OpenCV CUDA modules not available in this build");
#endif #endif
} }

View File

@ -31,6 +31,19 @@ if opencv_cuda_available
cpp_args += '-DHAVE_OPENCV_CUDA' cpp_args += '-DHAVE_OPENCV_CUDA'
endif endif
# Optional ximgproc (opencv_contrib) for the WLS disparity post-filter.
opencv_ximgproc_lib = disabler()
if cxx.has_header('opencv2/ximgproc/disparity_filter.hpp', dependencies: opencv_dep)
opencv_ximgproc_lib = cxx.find_library('opencv_ximgproc', required: false)
endif
if opencv_ximgproc_lib.found()
cpc_deps += opencv_ximgproc_lib
cpp_args += '-DCLOUD_POINT_HAVE_OPENCV_XIMGPROC'
message('opencv_ximgproc found: WLS disparity filter enabled')
else
message('opencv_ximgproc not found: WLS disparity filter disabled')
endif
cloud_point_compute_lib = shared_library('cloud_point_compute', cloud_point_compute_lib = shared_library('cloud_point_compute',
sources: cloud_point_sources, sources: cloud_point_sources,
include_directories: inc, include_directories: inc,

View File

@ -1,13 +1,20 @@
/// @file scared_dataset_benchmark.cpp /// @file scared_dataset_benchmark.cpp
/// @brief Evaluate the CPU stereo reconstruction against SCARED XYZ ground /// @brief Reconstruct a SCARED keyframe with the CPU stereo pipeline, report
/// truth. /// depth statistics and, when point_cloud.obj is present, accuracy against
/// the XYZ ground truth.
#include <algorithm>
#include <chrono> #include <chrono>
#include <cmath>
#include <filesystem>
#include <iostream> #include <iostream>
#include <optional>
#include <stdexcept> #include <stdexcept>
#include <string> #include <string>
#include <vector>
#include <glog/logging.h> #include <glog/logging.h>
#include <nlohmann/json.hpp> #include <nlohmann/json.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/imgproc.hpp> #include <opencv2/imgproc.hpp>
#include "cloud_point/imageFactory.h" #include "cloud_point/imageFactory.h"
@ -37,34 +44,114 @@ cv::Mat to_gray(const score::ImageRPC &rpc) {
return gray; return gray;
} }
/// Depth percentiles (metres) over the valid points of a cloud.
struct DepthStats {
std::size_t valid_points{0};
double valid_fraction{0.0};
double z_min_m{0.0}, z_p05_m{0.0}, z_median_m{0.0}, z_p95_m{0.0},
z_max_m{0.0};
};
DepthStats depth_stats(const score::PointCloud &cloud) {
std::vector<float> z;
z.reserve(static_cast<std::size_t>(cloud.width) *
static_cast<std::size_t>(cloud.height));
for (const auto &pt : cloud.valid_points())
z.push_back(pt[2]);
DepthStats stats;
stats.valid_points = z.size();
stats.valid_fraction =
static_cast<double>(z.size()) /
(static_cast<double>(cloud.width) * static_cast<double>(cloud.height));
if (z.empty())
return stats;
std::sort(z.begin(), z.end());
const auto at = [&](double q) {
return static_cast<double>(
z[std::min(z.size() - 1, static_cast<std::size_t>(
q * static_cast<double>(z.size())))]);
};
stats.z_min_m = z.front();
stats.z_p05_m = at(0.05);
stats.z_median_m = at(0.5);
stats.z_p95_m = at(0.95);
stats.z_max_m = z.back();
return stats;
}
/// Colour-mapped depth image (invalid pixels black) for visual inspection.
void write_depth_png(const score::PointCloud &cloud, const std::string &path,
double z_lo, double z_hi) {
cv::Mat depth8(cloud.height, cloud.width, CV_8UC1, cv::Scalar(0));
cv::Mat valid(cloud.height, cloud.width, CV_8UC1, cv::Scalar(0));
const double span = std::max(z_hi - z_lo, 1e-6);
for (int r = 0; r < cloud.height; ++r) {
for (int c = 0; c < cloud.width; ++c) {
const std::size_t idx = (static_cast<std::size_t>(r) *
static_cast<std::size_t>(cloud.width) +
static_cast<std::size_t>(c)) *
3u;
const float z = cloud.data[idx + 2];
if (std::isnan(z))
continue;
const double t = std::clamp((z - z_lo) / span, 0.0, 1.0);
depth8.at<uchar>(r, c) = static_cast<uchar>(255.0 * (1.0 - t));
valid.at<uchar>(r, c) = 255;
}
}
cv::Mat colour;
cv::applyColorMap(depth8, colour, cv::COLORMAP_TURBO);
colour.setTo(cv::Scalar(0, 0, 0), valid == 0);
if (!cv::imwrite(path, colour))
throw std::runtime_error("failed to write " + path);
}
} // namespace } // namespace
int main(int argc, char *argv[]) { int main(int argc, char *argv[]) {
google::InitGoogleLogging(argv[0]); google::InitGoogleLogging(argv[0]);
FLAGS_alsologtostderr = true; FLAGS_alsologtostderr = true;
if (argc < 2 || argc > 3) { if (argc < 2 || argc > 6 || argc == 5) {
std::cerr << "Usage: " << argv[0] std::cerr << "Usage: " << argv[0]
<< " <keyframe_dir> [num_disparities]\n"; << " <keyframe_dir> [num_disparities] [depth_png|-] "
"[min_depth_m max_depth_m]\n"
<< " Accuracy metrics are reported when "
"<keyframe_dir>/point_cloud.obj exists;\n"
<< " depth statistics are always reported.\n";
return 1; return 1;
} }
try { try {
const std::string keyframe_dir = argv[1]; const std::string keyframe_dir = argv[1];
const int num_disparities = argc == 3 ? std::stoi(argv[2]) : 160; const int num_disparities = argc >= 3 ? std::stoi(argv[2]) : 160;
std::string depth_png = argc >= 4 ? argv[3] : "";
if (depth_png == "-")
depth_png.clear();
score::PointCloudBuilder::Options depth_range;
if (argc == 6) {
depth_range = score::PointCloudBuilder::Options{std::stof(argv[4]),
std::stof(argv[5])};
}
score::ScaredDatasetLoader dataset(keyframe_dir); score::ScaredDatasetLoader dataset(keyframe_dir);
const auto &calibration_rpc = dataset.calibration(); const auto &calibration_rpc = dataset.calibration();
score::ScaredGroundTruthLoader ground_truth( const cv::Size image_size(calibration_rpc.width,
keyframe_dir, calibration_rpc.height);
cv::Size(calibration_rpc.width, calibration_rpc.height)); std::optional<score::ScaredGroundTruthLoader> ground_truth;
if (std::filesystem::exists(keyframe_dir + "/point_cloud.obj")) {
ground_truth.emplace(keyframe_dir, image_size);
} else {
LOG(WARNING) << "No point_cloud.obj in " << keyframe_dir
<< "; reporting depth statistics only";
}
const auto calibration = const auto calibration =
score::StereoRectifier::Calibration::from_rpc(calibration_rpc); score::StereoRectifier::Calibration::from_rpc(calibration_rpc);
score::StereoRectifier rectifier(calibration); score::StereoRectifier rectifier(calibration);
auto matcher = score::StereoMatcherFactory::create( auto matcher = score::StereoMatcherFactory::create(
score::StereoAlgorithmType::CPU, num_disparities); score::StereoAlgorithmType::CPU, num_disparities);
score::PointCloudBuilder builder(rectifier.q()); score::PointCloudBuilder builder(rectifier.q(), depth_range);
const auto pair = dataset.image_pair(0); const auto pair = dataset.image_pair(0);
const cv::Mat left_gray = to_gray(pair.left); const cv::Mat left_gray = to_gray(pair.left);
@ -79,10 +166,10 @@ int main(int argc, char *argv[]) {
const score::PointCloud cloud = builder.build(disparity); const score::PointCloud cloud = builder.build(disparity);
const auto reconstruction_end = std::chrono::steady_clock::now(); const auto reconstruction_end = std::chrono::steady_clock::now();
const cv::Mat rectified_ground_truth = const auto stats = depth_stats(cloud);
rectifier.rectify_left_point_map(ground_truth.point_map()); if (!depth_png.empty()) {
const auto metrics = write_depth_png(cloud, depth_png, stats.z_p05_m, stats.z_p95_m);
score::PointCloudEvaluator::evaluate(cloud, rectified_ground_truth); }
const auto matching_ms = const auto matching_ms =
std::chrono::duration<double, std::milli>(matching_end - start) std::chrono::duration<double, std::milli>(matching_end - start)
@ -92,25 +179,46 @@ int main(int argc, char *argv[]) {
matching_end) matching_end)
.count(); .count();
const nlohmann::json output = { nlohmann::json output = {
{"keyframe_dir", keyframe_dir}, {"keyframe_dir", keyframe_dir},
{"algorithm", "StereoSGBM"}, {"algorithm", "StereoSGBM"},
{"num_disparities", num_disparities}, {"num_disparities", num_disparities},
{"ground_truth_points", metrics.ground_truth_points}, {"min_depth_m", depth_range.min_depth_m},
{"matched_points", metrics.matched_points}, {"max_depth_m", depth_range.max_depth_m},
{"coverage", metrics.coverage}, {"image_width", cloud.width},
{"mae_x_m", metrics.mae_x_m}, {"image_height", cloud.height},
{"mae_y_m", metrics.mae_y_m}, {"valid_points", stats.valid_points},
{"mae_z_m", metrics.mae_z_m}, {"valid_fraction", stats.valid_fraction},
{"mae_3d_m", metrics.mae_3d_m}, {"z_min_m", stats.z_min_m},
{"rmse_3d_m", metrics.rmse_3d_m}, {"z_p05_m", stats.z_p05_m},
{"median_3d_m", metrics.median_3d_m}, {"z_median_m", stats.z_median_m},
{"within_1mm", metrics.within_1mm}, {"z_p95_m", stats.z_p95_m},
{"within_2mm", metrics.within_2mm}, {"z_max_m", stats.z_max_m},
{"within_5mm", metrics.within_5mm},
{"matching_ms", matching_ms}, {"matching_ms", matching_ms},
{"reconstruction_ms", reconstruction_ms}, {"reconstruction_ms", reconstruction_ms},
{"has_ground_truth", ground_truth.has_value()},
}; };
if (!depth_png.empty())
output["depth_png"] = depth_png;
if (ground_truth) {
const cv::Mat rectified_ground_truth =
rectifier.rectify_left_point_map(ground_truth->point_map());
const auto metrics = score::PointCloudEvaluator::evaluate(
cloud, rectified_ground_truth);
output["ground_truth_points"] = metrics.ground_truth_points;
output["matched_points"] = metrics.matched_points;
output["coverage"] = metrics.coverage;
output["mae_x_m"] = metrics.mae_x_m;
output["mae_y_m"] = metrics.mae_y_m;
output["mae_z_m"] = metrics.mae_z_m;
output["mae_3d_m"] = metrics.mae_3d_m;
output["rmse_3d_m"] = metrics.rmse_3d_m;
output["median_3d_m"] = metrics.median_3d_m;
output["within_1mm"] = metrics.within_1mm;
output["within_2mm"] = metrics.within_2mm;
output["within_5mm"] = metrics.within_5mm;
}
std::cout << output.dump(2) << '\n'; std::cout << output.dump(2) << '\n';
} catch (const std::exception &error) { } catch (const std::exception &error) {
std::cerr << "Benchmark failed: " << error.what() << '\n'; std::cerr << "Benchmark failed: " << error.what() << '\n';

View File

@ -8,7 +8,8 @@
namespace score { namespace score {
std::unique_ptr<IStereoMatcher> std::unique_ptr<IStereoMatcher>
StereoMatcherFactory::create(StereoAlgorithmType type, int num_disparities) { StereoMatcherFactory::create(StereoAlgorithmType type, int num_disparities,
CpuStereoMatcher::Params cpu_params) {
if (num_disparities <= 0 || num_disparities % 16 != 0) { if (num_disparities <= 0 || num_disparities % 16 != 0) {
throw std::invalid_argument( throw std::invalid_argument(
"StereoMatcherFactory: num_disparities must be a positive " "StereoMatcherFactory: num_disparities must be a positive "
@ -17,14 +18,16 @@ StereoMatcherFactory::create(StereoAlgorithmType type, int num_disparities) {
} }
switch (type) { switch (type) {
case StereoAlgorithmType::CPU: case StereoAlgorithmType::CPU:
return std::make_unique<CpuStereoMatcher>(0, num_disparities); return std::make_unique<CpuStereoMatcher>(0, num_disparities,
cpu_params);
case StereoAlgorithmType::GPU: case StereoAlgorithmType::GPU:
try { try {
return std::make_unique<GpuStereoMatcher>(); return std::make_unique<GpuStereoMatcher>(0, num_disparities);
} catch (const std::exception &e) { } catch (const std::exception &e) {
LOG(WARNING) << "GPU stereo matcher unavailable: " << e.what() LOG(WARNING) << "GPU stereo matcher unavailable: " << e.what()
<< ". Falling back to CPU."; << ". Falling back to CPU.";
return std::make_unique<CpuStereoMatcher>(0, num_disparities); return std::make_unique<CpuStereoMatcher>(0, num_disparities,
cpu_params);
} }
} }
return nullptr; return nullptr;

View File

@ -25,8 +25,15 @@ int main(int argc, char *argv[]) {
try { try {
auto config = score::ConfigLoader::load(config_path); auto config = score::ConfigLoader::load(config_path);
score::CliStereoOptions stereo;
stereo.use_gpu = config.cloud_point.algorithm == "gpu";
stereo.num_disparities = config.cloud_point.num_disparities;
stereo.min_depth_m = static_cast<float>(config.cloud_point.min_depth_m);
stereo.max_depth_m = static_cast<float>(config.cloud_point.max_depth_m);
stereo.ply_stride = config.cloud_point.ply_stride;
stereo.wls_filter = config.cloud_point.wls_filter;
return score::run_cli(std::cin, std::cout, config.server.ip, return score::run_cli(std::cin, std::cout, config.server.ip,
config.server.port); config.server.port, stereo);
} catch (const std::exception &e) { } catch (const std::exception &e) {
std::cerr << "Failed to start CLI: " << e.what() << std::endl; std::cerr << "Failed to start CLI: " << e.what() << std::endl;
return 1; return 1;

View File

@ -3,6 +3,7 @@ test_sources = files(
'test_integration.cpp', 'test_integration.cpp',
'test_tcp.cpp', 'test_tcp.cpp',
'test_cli.cpp', 'test_cli.cpp',
'test_config.cpp',
'test_c_api.cpp', 'test_c_api.cpp',
'test_base64.cpp', 'test_base64.cpp',
'test_serialize_image.cpp' 'test_serialize_image.cpp'
@ -20,6 +21,7 @@ if opencv_dep.found()
'test_point_cloud_builder.cpp', 'test_point_cloud_builder.cpp',
'test_point_cloud_evaluator.cpp', 'test_point_cloud_evaluator.cpp',
'test_cloud_point_client.cpp', 'test_cloud_point_client.cpp',
'test_ply_export.cpp',
'test_scared_dataset.cpp', 'test_scared_dataset.cpp',
'test_scared_ground_truth_loader.cpp' 'test_scared_ground_truth_loader.cpp'
) )

76
tests/test_config.cpp Normal file
View File

@ -0,0 +1,76 @@
#include <cstdio>
#include <fstream>
#include <gtest/gtest.h>
#include <string>
#include <tuple>
#include "cloud_point_rpc/config.hpp"
namespace {
struct TempConfig {
std::string path;
explicit TempConfig(const std::string &yaml)
: path("test_config_" +
std::to_string(reinterpret_cast<uintptr_t>(this)) + ".yaml") {
std::ofstream(path) << yaml;
}
~TempConfig() { std::remove(path.c_str()); }
};
} // namespace
TEST(ConfigLoaderTest, CloudPointSectionDefaultsWhenAbsent) {
TempConfig cfg("server:\n ip: \"127.0.0.1\"\n port: 8080\n");
const auto c = score::ConfigLoader::load(cfg.path);
EXPECT_EQ(c.cloud_point.algorithm, "gpu");
EXPECT_EQ(c.cloud_point.num_disparities, 128);
EXPECT_DOUBLE_EQ(c.cloud_point.min_depth_m, 0.01);
EXPECT_DOUBLE_EQ(c.cloud_point.max_depth_m, 10.0);
}
TEST(ConfigLoaderTest, CloudPointSectionIsParsed) {
TempConfig cfg("server:\n ip: \"127.0.0.1\"\n port: 8080\n"
"cloud_point:\n algorithm: cpu\n num_disparities: 160\n"
" min_depth_m: 0.02\n max_depth_m: 0.3\n");
const auto c = score::ConfigLoader::load(cfg.path);
EXPECT_EQ(c.cloud_point.algorithm, "cpu");
EXPECT_EQ(c.cloud_point.num_disparities, 160);
EXPECT_DOUBLE_EQ(c.cloud_point.min_depth_m, 0.02);
EXPECT_DOUBLE_EQ(c.cloud_point.max_depth_m, 0.3);
}
TEST(ConfigLoaderTest, CloudPointExportAndFilterKeys) {
TempConfig cfg("server:\n ip: \"127.0.0.1\"\n port: 8080\n"
"cloud_point:\n ply_stride: 4\n wls_filter: true\n");
const auto c = score::ConfigLoader::load(cfg.path);
EXPECT_EQ(c.cloud_point.ply_stride, 4);
EXPECT_TRUE(c.cloud_point.wls_filter);
TempConfig bad("server:\n ip: \"127.0.0.1\"\n port: 8080\n"
"cloud_point:\n ply_stride: 0\n");
EXPECT_THROW(std::ignore = score::ConfigLoader::load(bad.path),
std::runtime_error);
}
TEST(ConfigLoaderTest, CloudPointPartialSectionKeepsDefaults) {
TempConfig cfg("server:\n ip: \"127.0.0.1\"\n port: 8080\n"
"cloud_point:\n max_depth_m: 0.5\n");
const auto c = score::ConfigLoader::load(cfg.path);
EXPECT_EQ(c.cloud_point.algorithm, "gpu");
EXPECT_EQ(c.cloud_point.num_disparities, 128);
EXPECT_DOUBLE_EQ(c.cloud_point.max_depth_m, 0.5);
}
TEST(ConfigLoaderTest, CloudPointRejectsBadAlgorithm) {
TempConfig cfg("server:\n ip: \"127.0.0.1\"\n port: 8080\n"
"cloud_point:\n algorithm: fpga\n");
EXPECT_THROW(std::ignore = score::ConfigLoader::load(cfg.path),
std::runtime_error);
}
TEST(ConfigLoaderTest, CloudPointRejectsInvertedDepthRange) {
TempConfig cfg("server:\n ip: \"127.0.0.1\"\n port: 8080\n"
"cloud_point:\n min_depth_m: 1.0\n max_depth_m: 0.5\n");
EXPECT_THROW(std::ignore = score::ConfigLoader::load(cfg.path),
std::runtime_error);
}

157
tests/test_ply_export.cpp Normal file
View File

@ -0,0 +1,157 @@
#include <cmath>
#include <cstdio>
#include <cstring>
#include <fstream>
#include <gtest/gtest.h>
#include <limits>
#include <sstream>
#include <string>
#include <tuple>
#include "cloud_point/cloud_point_client.hpp"
using namespace score;
namespace {
/// 3x3 organised cloud on a plane z = 1 m with 1 cm pixel spacing.
PointCloud make_grid_cloud() {
PointCloud cloud;
cloud.width = 3;
cloud.height = 3;
cloud.data.resize(27);
for (int r = 0; r < 3; ++r) {
for (int c = 0; c < 3; ++c) {
const size_t i = static_cast<size_t>(r * 3 + c) * 3;
cloud.data[i] = 0.01f * static_cast<float>(c);
cloud.data[i + 1] = 0.01f * static_cast<float>(r);
cloud.data[i + 2] = 1.0f;
}
}
return cloud;
}
void set_nan(PointCloud &cloud, int r, int c) {
const size_t i = static_cast<size_t>(r * cloud.width + c) * 3;
const float nan = std::numeric_limits<float>::quiet_NaN();
cloud.data[i] = cloud.data[i + 1] = cloud.data[i + 2] = nan;
}
struct TempFile {
std::string path{"test_ply_export.ply"};
~TempFile() { std::remove(path.c_str()); }
std::string read() const {
std::ifstream in(path);
std::stringstream ss;
ss << in.rdbuf();
return ss.str();
}
};
} // namespace
TEST(PlyExportTest, PointsOnlyHasNoFaceElement) {
TempFile file;
const auto faces =
write_ply(make_grid_cloud(), file.path, PlyOptions{false, 0.05f});
EXPECT_EQ(faces, 0u);
const auto text = file.read();
EXPECT_NE(text.find("element vertex 9\n"), std::string::npos);
EXPECT_EQ(text.find("element face"), std::string::npos);
}
TEST(PlyExportTest, FullGridProducesTwoTrianglesPerCell) {
TempFile file;
const auto faces =
write_ply(make_grid_cloud(), file.path, PlyOptions{true, 0.05f, false});
EXPECT_EQ(faces, 8u); // 2x2 cells * 2 triangles
EXPECT_NE(file.read().find("format ascii 1.0\n"), std::string::npos);
const auto text = file.read();
EXPECT_NE(text.find("element face 8\n"), std::string::npos);
EXPECT_NE(text.find("property list uchar int vertex_indices\n"),
std::string::npos);
// First cell, first triangle: (0,0) -> (1,0) -> (1,1) = indices 0,3,4
EXPECT_NE(text.find("\n3 0 3 4\n"), std::string::npos);
}
TEST(PlyExportTest, MissingCornerKeepsSingleTriangle) {
auto cloud = make_grid_cloud();
set_nan(cloud, 0, 0); // top-left cell has 3 valid corners
TempFile file;
const auto faces =
write_ply(cloud, file.path, PlyOptions{true, 0.05f, false});
EXPECT_EQ(faces, 7u);
const auto text = file.read();
EXPECT_NE(text.find("element vertex 8\n"), std::string::npos);
}
TEST(PlyExportTest, DepthJumpBreaksMesh) {
auto cloud = make_grid_cloud();
// Push the centre column 20 cm away: every triangle touching it now has
// an edge longer than 5% of its mean depth and must be dropped.
for (int r = 0; r < 3; ++r) {
cloud.data[static_cast<size_t>(r * 3 + 1) * 3 + 2] = 1.2f;
}
TempFile file;
EXPECT_EQ(write_ply(cloud, file.path), 0u);
}
TEST(PlyExportTest, StrideAveragesBlocks) {
// 3x3 grid with stride 2 -> 2x2 blocks: one full 2x2 block, two 2x1 /
// 1x2 edge blocks and the single corner pixel -> 4 vertices, 2 faces.
TempFile file;
const auto faces = write_ply(make_grid_cloud(), file.path,
PlyOptions{true, 0.05f, false, 2});
EXPECT_EQ(faces, 2u);
const auto text = file.read();
EXPECT_NE(text.find("element vertex 4\n"), std::string::npos);
// First vertex is the mean of pixels (0,0),(0,1),(1,0),(1,1).
EXPECT_NE(text.find("\n0.005 0.005 1\n"), std::string::npos);
// Last vertex is the lone corner pixel (2,2).
EXPECT_NE(text.find("\n0.02 0.02 1\n"), std::string::npos);
}
TEST(PlyExportTest, StrideDropsMostlyInvalidBlocks) {
auto cloud = make_grid_cloud();
// Invalidate 3 of the 4 pixels of the top-left 2x2 block.
set_nan(cloud, 0, 0);
set_nan(cloud, 0, 1);
set_nan(cloud, 1, 0);
TempFile file;
write_ply(cloud, file.path, PlyOptions{true, 0.05f, false, 2});
EXPECT_NE(file.read().find("element vertex 3\n"), std::string::npos);
}
TEST(PlyExportTest, InvalidStrideThrows) {
TempFile file;
EXPECT_THROW(std::ignore = write_ply(make_grid_cloud(), file.path,
PlyOptions{true, 0.05f, false, 0}),
std::invalid_argument);
}
TEST(PlyExportTest, BinaryLayoutMatchesHeader) {
TempFile file;
const auto faces = write_ply(make_grid_cloud(), file.path); // binary
ASSERT_EQ(faces, 8u);
const auto text = file.read();
EXPECT_NE(text.find("format binary_little_endian 1.0\n"),
std::string::npos);
const auto header_end = text.find("end_header\n") + 11;
ASSERT_NE(header_end, std::string::npos + 11);
// 9 vertices * 12 bytes + 8 faces * (1 + 12) bytes
EXPECT_EQ(text.size() - header_end, 9u * 12u + 8u * 13u);
// First vertex is (0, 0, 1)
float v[3];
std::memcpy(v, text.data() + header_end, sizeof(v));
EXPECT_FLOAT_EQ(v[0], 0.0f);
EXPECT_FLOAT_EQ(v[1], 0.0f);
EXPECT_FLOAT_EQ(v[2], 1.0f);
// First face: count byte 3 then indices 0,3,4
const char *f = text.data() + header_end + 9 * 12;
EXPECT_EQ(static_cast<unsigned char>(f[0]), 3u);
int idx[3];
std::memcpy(idx, f + 1, sizeof(idx));
EXPECT_EQ(idx[0], 0);
EXPECT_EQ(idx[1], 3);
EXPECT_EQ(idx[2], 4);
}

View File

@ -1,5 +1,7 @@
#include <cmath>
#include <gtest/gtest.h> #include <gtest/gtest.h>
#include <opencv2/core.hpp> #include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
#include <tuple> #include <tuple>
#include "cloud_point/cpu_stereo_matcher.hpp" #include "cloud_point/cpu_stereo_matcher.hpp"
@ -47,9 +49,89 @@ TEST(StereoMatcherTest, FactoryCpuCreatesNonNull) {
EXPECT_FALSE(disparity.empty()); EXPECT_FALSE(disparity.empty());
} }
TEST(StereoMatcherTest, CpuMatcherRejectsEvenBlockSize) {
CpuStereoMatcher::Params params;
params.block_size = 4;
EXPECT_THROW(CpuStereoMatcher(0, 64, params), std::invalid_argument);
}
TEST(StereoMatcherTest, CpuMatcherRejectsBadMedianKernel) {
CpuStereoMatcher::Params params;
params.median_kernel = 4;
EXPECT_THROW(CpuStereoMatcher(0, 64, params), std::invalid_argument);
}
TEST(StereoMatcherTest, WlsKeepsSgbmHolesInvalid) {
if (!CpuStereoMatcher::wls_available())
GTEST_SKIP() << "OpenCV built without ximgproc";
// Left 64 px of the right image have no counterpart -> SGBM leaves the
// left border invalid; WLS must not fill it with extrapolated values.
cv::Mat left(120, 200, CV_8UC1);
cv::randu(left, 0, 255);
cv::blur(left, left, cv::Size(3, 3));
cv::Mat right = cv::Mat::zeros(left.size(), CV_8UC1);
left(cv::Rect(8, 0, left.cols - 8, left.rows))
.copyTo(right(cv::Rect(0, 0, left.cols - 8, left.rows)));
CpuStereoMatcher::Params raw;
raw.wls_filter = false;
CpuStereoMatcher::Params wls;
wls.wls_filter = true;
cv::Mat d_raw = CpuStereoMatcher(0, 64, raw).compute(left, right);
cv::Mat d_wls = CpuStereoMatcher(0, 64, wls).compute(left, right);
ASSERT_EQ(d_wls.type(), CV_16S);
ASSERT_EQ(d_wls.size(), d_raw.size());
const int raw_invalid = cv::countNonZero(d_raw <= 0);
EXPECT_GT(raw_invalid, 0);
// No SGBM hole may be filled with an extrapolated disparity.
const int filled = cv::countNonZero((d_raw <= 0) & (d_wls > 0));
EXPECT_EQ(filled, 0);
// The filter must still produce a valid map elsewhere.
EXPECT_GT(cv::countNonZero(d_wls > 0), 0);
}
TEST(StereoMatcherTest, CpuMatcherRecoversKnownShift) {
// Textured synthetic pair: right image is the left shifted by 8 px, so
// the (16x fixed-point) disparity in the interior should be ~8 px.
constexpr int kShift = 8;
cv::Mat left(120, 200, CV_8UC1);
cv::randu(left, 0, 255);
cv::blur(left, left, cv::Size(3, 3));
cv::Mat right = cv::Mat::zeros(left.size(), CV_8UC1);
left(cv::Rect(kShift, 0, left.cols - kShift, left.rows))
.copyTo(right(cv::Rect(0, 0, left.cols - kShift, left.rows)));
CpuStereoMatcher matcher(0, 64);
cv::Mat disparity = matcher.compute(left, right);
ASSERT_EQ(disparity.type(), CV_16S);
int good = 0, total = 0;
for (int y = 10; y < left.rows - 10; ++y) {
for (int x = 70; x < left.cols - 20; ++x) {
const float d = disparity.at<short>(y, x) / 16.0f;
if (d <= 0)
continue;
++total;
if (std::abs(d - kShift) <= 1.0f)
++good;
}
}
ASSERT_GT(total, 0);
EXPECT_GT(static_cast<double>(good) / total, 0.9);
}
TEST(StereoMatcherTest, GpuSupportedNumDisparitiesRoundsUp) {
EXPECT_EQ(GpuStereoMatcher::supported_num_disparities(16), 64);
EXPECT_EQ(GpuStereoMatcher::supported_num_disparities(64), 64);
EXPECT_EQ(GpuStereoMatcher::supported_num_disparities(96), 128);
EXPECT_EQ(GpuStereoMatcher::supported_num_disparities(160), 256);
EXPECT_THROW(std::ignore = GpuStereoMatcher::supported_num_disparities(272),
std::invalid_argument);
}
TEST(StereoMatcherTest, FactoryRejectsInvalidNumDisparities) { TEST(StereoMatcherTest, FactoryRejectsInvalidNumDisparities) {
EXPECT_THROW(std::ignore = StereoMatcherFactory::create( EXPECT_THROW(std::ignore =
StereoAlgorithmType::CPU, 0), StereoMatcherFactory::create(StereoAlgorithmType::CPU, 0),
std::invalid_argument); std::invalid_argument);
EXPECT_THROW(std::ignore = StereoMatcherFactory::create( EXPECT_THROW(std::ignore = StereoMatcherFactory::create(
StereoAlgorithmType::CPU, -16), StereoAlgorithmType::CPU, -16),