Compare commits
4 Commits
cd97e7b3f1
...
14ae0a901f
| Author | SHA1 | Date | |
|---|---|---|---|
| 14ae0a901f | |||
| 508570cb5c | |||
| 96bc82f0fb | |||
| 4f21e2ea38 |
3
.gitignore
vendored
3
.gitignore
vendored
@ -17,3 +17,6 @@ large_tool_results/
|
|||||||
|
|
||||||
# IDE
|
# IDE
|
||||||
.idea/
|
.idea/
|
||||||
|
|
||||||
|
# Point-cloud exports
|
||||||
|
*.ply
|
||||||
|
|||||||
104
README.md
104
README.md
@ -151,38 +151,106 @@ Each keyframe directory contains:
|
|||||||
/path/to/test_dataset_8/keyframe_0 8080
|
/path/to/test_dataset_8/keyframe_0 8080
|
||||||
```
|
```
|
||||||
|
|
||||||
|
If port 8080 is already taken on your machine (Docker's `rootlesskit`
|
||||||
|
commonly holds it) pass another port and update `server.port` in the CLI
|
||||||
|
config accordingly. Connecting the CLI to a foreign service on 8080 shows up
|
||||||
|
as `invalid JSON response from server` / `std::bad_alloc` errors.
|
||||||
|
|
||||||
### Connecting with the CLI
|
### Connecting with the CLI
|
||||||
|
|
||||||
In a second terminal run the interactive CLI against the same host and port:
|
In a second terminal run the interactive CLI with the SCARED-tuned config:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Adjust ip/port in config.yaml if needed, then:
|
./build/src/cloud_point_rpc_cli config.scared.yml
|
||||||
./build/src/cloud_point_rpc_cli config.yaml
|
# Option 4 — compute point cloud and print valid point count + bounding box
|
||||||
# Option 4 — compute point cloud and print valid point count
|
# Option 5 — compute point cloud and save a triangulated PLY mesh
|
||||||
# Option 5 — compute point cloud and save to output.ply (inspect in MeshLab)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`config.scared.yml` sets the optional `cloud_point` section that options 4/5
|
||||||
|
honour:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
cloud_point:
|
||||||
|
algorithm: cpu # "gpu" falls back to CPU when CUDA is unavailable
|
||||||
|
num_disparities: 160 # fx ~1024 px, baseline ~4.35 mm -> up to ~160 px
|
||||||
|
min_depth_m: 0.02 # endoscopic working range: 20 mm .. 300 mm
|
||||||
|
max_depth_m: 0.30
|
||||||
|
ply_stride: 1 # option 5: 4 = 16x smaller, block-averaged mesh
|
||||||
|
wls_filter: false # true = smoother mesh for viewers, less accurate
|
||||||
|
```
|
||||||
|
|
||||||
|
Depth limits are a physical bound on the scene: with this rig depth is
|
||||||
|
roughly `4.45 m / disparity_px`, so any mismatch with a disparity below
|
||||||
|
~15 px reprojects metres away. Without the section the CLI falls back to the
|
||||||
|
generic defaults (GPU, 128 disparities, 0.01–10 m). The SGBM matcher itself
|
||||||
|
is configured with OpenCV's recommended smoothness penalties (P1 = 8·bs²,
|
||||||
|
P2 = 32·bs²), a 5×5 block, uniqueness ratio 10, speckle filtering and a
|
||||||
|
left-right consistency check; see `CpuStereoMatcher::Params`.
|
||||||
|
|
||||||
|
### Checking the result
|
||||||
|
|
||||||
|
Option 4 should report a bounding box with z inside roughly
|
||||||
|
`[0.03, 0.16]` m for `test_dataset_8` keyframes. Option 5 writes a binary
|
||||||
|
little-endian PLY containing every valid point **and** a mesh triangulated
|
||||||
|
from the pixel grid (triangles are dropped where the longest edge exceeds 5 % of the local
|
||||||
|
depth, so the mesh breaks at occlusions). The mesh is what makes web viewers
|
||||||
|
usable: viewers such as Meshy's online PLY viewer fabricate a triangle from
|
||||||
|
every three consecutive vertices of a vertex-only PLY, which draws long
|
||||||
|
slivers across the surface and makes a correct cloud look like a fan of
|
||||||
|
rays. Pass `PlyOptions{false, 0.0f, false}` to `write_ply` for a points-only ASCII
|
||||||
|
file.
|
||||||
|
|
||||||
|
**Surface roughness vs accuracy.** SGBM's sub-pixel disparity noise
|
||||||
|
(~0.2 px, correlated over several pixels) is ~1 mm of depth on this rig,
|
||||||
|
far more than the 0.07 mm lateral pixel pitch, so a mesh built from the raw
|
||||||
|
cloud is "hairy": face normals sit a median 45° off the camera axis on
|
||||||
|
tissue that faces the camera. Colour depth maps hide this; shaded mesh
|
||||||
|
viewers show it as fuzz. `wls_filter: true` applies OpenCV's edge-aware
|
||||||
|
WLS disparity filter (needs `opencv_ximgproc`, doubles matching time) and
|
||||||
|
brings the median normal to ~23° with neighbour depth jumps down from 0.15 mm
|
||||||
|
to 0.06 mm, but it also costs accuracy on SCARED (keyframe 1: MAE 0.84 →
|
||||||
|
0.92 mm, within 2 mm 80 → 78 %; dataset_3: MAE 1.75 → 2.20 mm). It is
|
||||||
|
therefore off by default: use it for pictures, not for measurements. SGBM's
|
||||||
|
own holes are never filled by the filter.
|
||||||
|
|
||||||
When the keyframe contains `point_cloud.obj`, run the benchmark to compare the
|
When the keyframe contains `point_cloud.obj`, run the benchmark to compare the
|
||||||
reconstruction with its pixel-aligned XYZ ground truth:
|
reconstruction with its pixel-aligned XYZ ground truth:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
./build/src/cloud_point/scared_dataset_benchmark \
|
./build/src/cloud_point/scared_dataset_benchmark \
|
||||||
/path/to/test_dataset_8/keyframe_0 160
|
/path/to/dataset_1/keyframe_1 160 [depth.png|-] [min_depth_m max_depth_m]
|
||||||
```
|
```
|
||||||
|
|
||||||
The benchmark reports coverage, component-wise and 3-D errors, threshold
|
The benchmark always reports the valid-point fraction, depth percentiles
|
||||||
accuracy, and matching/reconstruction timings as JSON. OBJ coordinates are
|
and matching/reconstruction timings as JSON, and optionally writes a
|
||||||
converted from millimetres to metres and rectified into the same left-camera
|
colour-mapped depth image (third argument, `-` to skip) for visual
|
||||||
frame as the reconstructed cloud before evaluation.
|
inspection. The optional depth range applies the same filter as the CLI's
|
||||||
|
`cloud_point` section; without it the generic 0.01–10 m defaults are used,
|
||||||
|
which lets a few residual mismatches at metres of depth inflate RMSE. When
|
||||||
|
`point_cloud.obj` is present it also reports coverage, component-wise and
|
||||||
|
3-D errors and threshold accuracy. Note that the `test_dataset_*` archives
|
||||||
|
ship without `point_cloud.obj`; ground truth is only in the full
|
||||||
|
`dataset_N.zip` archives (13–40 GB each). The zips are served with HTTP
|
||||||
|
range support, so single keyframes can be extracted remotely with Python's
|
||||||
|
`zipfile` over a seekable HTTP file object instead of downloading the whole
|
||||||
|
archive.
|
||||||
|
|
||||||
**Disparity range caveat:** the CLI constructs `CloudPointClient` with the
|
`scripts/scared_overview.py` runs the benchmark over many keyframes and
|
||||||
default of 128 disparity levels, while this rig (fx ≈ 1024 px, baseline
|
prints a Markdown table:
|
||||||
≈ 4.35 mm) produces disparities above 128 px for tissue nearer than
|
|
||||||
~35 mm — those pixels silently drop out of the cloud. The E2E test
|
```bash
|
||||||
(`tests/test_scared_dataset.cpp`) passes `num_disparities = 160` for full
|
scripts/scared_overview.py --png-dir out/depth --depth-range 0.02 0.30 \
|
||||||
coverage; programmatic consumers should do the same via the
|
datasets/scared/dataset_1/keyframe_* datasets/scared/test_dataset_8/keyframe_*
|
||||||
`CloudPointClient` constructor. For the CLI's qualitative check the default
|
``` OBJ coordinates are
|
||||||
is fine (observed median depth ≈ 115 mm is well within range).
|
converted from millimetres to metres and rectified into the same left-camera
|
||||||
|
frame as the reconstructed cloud before evaluation. Reference numbers for
|
||||||
|
`dataset_1/keyframe_1` with the tuned SGBM configuration: coverage ≈ 0.85,
|
||||||
|
MAE₃D ≈ 0.8 mm, RMSE₃D ≈ 1.4 mm, 80 % of points within 2 mm (the previous
|
||||||
|
unregularised SGBM gave MAE₃D ≈ 44 mm and RMSE₃D ≈ 289 mm).
|
||||||
|
|
||||||
|
The E2E test (`tests/test_scared_dataset.cpp`) exercises the same pipeline
|
||||||
|
with `num_disparities = 160` and asserts >50 000 valid points and a median
|
||||||
|
depth in `[0.02, 0.20]` m.
|
||||||
|
|
||||||
## Communication model
|
## Communication model
|
||||||
|
|
||||||
|
|||||||
13
config.scared.yml
Normal file
13
config.scared.yml
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
# CLI configuration for validating against a SCARED keyframe served by
|
||||||
|
# scared_dataset_server (default port 8080; pick another if 8080 is busy).
|
||||||
|
server:
|
||||||
|
ip: "127.0.0.1"
|
||||||
|
port: 8080
|
||||||
|
|
||||||
|
cloud_point:
|
||||||
|
algorithm: cpu # "gpu" falls back to CPU when CUDA is unavailable
|
||||||
|
num_disparities: 160 # fx ~1024 px, baseline ~4.35 mm -> up to ~160 px
|
||||||
|
min_depth_m: 0.02 # endoscopic working range: 20 mm .. 300 mm
|
||||||
|
max_depth_m: 0.30
|
||||||
|
ply_stride: 1 # option 5: 4 = 16x smaller mesh, block-averaged
|
||||||
|
wls_filter: false # true = smoother mesh for viewers, ~10 % less accurate
|
||||||
@ -1,3 +1,11 @@
|
|||||||
server:
|
server:
|
||||||
ip: "127.0.0.1"
|
ip: "127.0.0.1"
|
||||||
port: 9095
|
port: 9095
|
||||||
|
|
||||||
|
# Optional stereo reconstruction settings for CLI options 4/5.
|
||||||
|
# See config.scared.yml for values tuned to the SCARED endoscopic rig.
|
||||||
|
# cloud_point:
|
||||||
|
# algorithm: gpu # or cpu
|
||||||
|
# num_disparities: 128 # positive multiple of 16
|
||||||
|
# min_depth_m: 0.01
|
||||||
|
# max_depth_m: 10.0
|
||||||
|
|||||||
@ -29,15 +29,18 @@ class CloudPointClient {
|
|||||||
/// @brief Construct client (does not connect).
|
/// @brief Construct client (does not connect).
|
||||||
/// @param ip Server IP address.
|
/// @param ip Server IP address.
|
||||||
/// @param port Server port.
|
/// @param port Server port.
|
||||||
/// @param algo Stereo matching algorithm (GPU falls back to CPU if
|
/// @param algo Stereo matching algorithm (GPU falls back to CPU
|
||||||
|
/// if
|
||||||
/// unavailable).
|
/// unavailable).
|
||||||
/// @param opts Depth filtering options.
|
/// @param opts Depth filtering options.
|
||||||
/// @param num_disparities SGBM disparity levels (default 128; use 160 for
|
/// @param num_disparities SGBM disparity levels (default 128; use 160 for
|
||||||
/// small-baseline rigs such as SCARED).
|
/// small-baseline rigs such as SCARED).
|
||||||
CloudPointClient(std::string ip, int port,
|
/// @param matcher_params SGBM tuning / post-filters for the CPU matcher.
|
||||||
StereoAlgorithmType algo = StereoAlgorithmType::GPU,
|
CloudPointClient(
|
||||||
PointCloudBuilder::Options opts = {},
|
std::string ip, int port,
|
||||||
int num_disparities = 128);
|
StereoAlgorithmType algo = StereoAlgorithmType::GPU,
|
||||||
|
PointCloudBuilder::Options opts = {}, int num_disparities = 128,
|
||||||
|
CpuStereoMatcher::Params matcher_params = CpuStereoMatcher::Params{});
|
||||||
|
|
||||||
~CloudPointClient();
|
~CloudPointClient();
|
||||||
|
|
||||||
@ -61,6 +64,7 @@ class CloudPointClient {
|
|||||||
StereoAlgorithmType algo_;
|
StereoAlgorithmType algo_;
|
||||||
PointCloudBuilder::Options opts_;
|
PointCloudBuilder::Options opts_;
|
||||||
int num_disparities_;
|
int num_disparities_;
|
||||||
|
CpuStereoMatcher::Params matcher_params_;
|
||||||
std::unique_ptr<TCPConnector> connector_;
|
std::unique_ptr<TCPConnector> connector_;
|
||||||
std::unique_ptr<RpcClient> client_;
|
std::unique_ptr<RpcClient> client_;
|
||||||
std::unique_ptr<StereoRectifier> rectifier_;
|
std::unique_ptr<StereoRectifier> rectifier_;
|
||||||
@ -68,9 +72,41 @@ class CloudPointClient {
|
|||||||
std::unique_ptr<PointCloudBuilder> builder_;
|
std::unique_ptr<PointCloudBuilder> builder_;
|
||||||
};
|
};
|
||||||
|
|
||||||
/// @brief Write valid points as ASCII PLY (for MeshLab inspection).
|
/// @brief PLY export settings.
|
||||||
|
struct PlyOptions {
|
||||||
|
/// Emit a triangle mesh built from the organised pixel grid. Web viewers
|
||||||
|
/// (e.g. Meshy) fabricate triangles from consecutive vertices of a
|
||||||
|
/// vertex-only PLY, which draws long slivers across the surface; a real
|
||||||
|
/// mesh renders correctly everywhere and still contains every point.
|
||||||
|
bool triangulate;
|
||||||
|
/// Reject triangles whose longest edge exceeds this fraction of the
|
||||||
|
/// triangle's mean depth (breaks the mesh at occlusion boundaries).
|
||||||
|
float max_edge_depth_ratio;
|
||||||
|
/// Write binary_little_endian instead of ASCII. A full 1280x1024 mesh
|
||||||
|
/// is ~40 MB in binary versus ~85 MB in ASCII and parses much faster.
|
||||||
|
bool binary;
|
||||||
|
/// Decimate the grid by this factor (1 = full resolution). Each output
|
||||||
|
/// vertex is the mean of the valid pixels in its stride x stride block
|
||||||
|
/// (blocks less than half valid are dropped), so stride 4 both shrinks
|
||||||
|
/// the mesh 16x and cuts per-pixel disparity noise ~4x. Web viewers
|
||||||
|
/// cope far better with such a mesh than with the raw 1.1 M vertices.
|
||||||
|
int stride;
|
||||||
|
PlyOptions() noexcept
|
||||||
|
: triangulate(true), max_edge_depth_ratio(0.05f), binary(true),
|
||||||
|
stride(1) {}
|
||||||
|
PlyOptions(bool tri, float ratio, bool bin = true, int step = 1) noexcept
|
||||||
|
: triangulate(tri), max_edge_depth_ratio(ratio), binary(bin),
|
||||||
|
stride(step) {}
|
||||||
|
};
|
||||||
|
|
||||||
|
/// @brief Write valid points (and optionally a grid-triangulated mesh) as
|
||||||
|
/// PLY for MeshLab / web-viewer inspection.
|
||||||
/// @param cloud Source point cloud.
|
/// @param cloud Source point cloud.
|
||||||
/// @param path Output file path.
|
/// @param path Output file path.
|
||||||
void write_ply(const PointCloud &cloud, const std::string &path);
|
/// @param opts Export settings; defaults to a binary triangulated mesh.
|
||||||
|
/// @return Number of faces written (0 when not triangulating).
|
||||||
|
/// @throws std::invalid_argument if opts.stride < 1.
|
||||||
|
size_t write_ply(const PointCloud &cloud, const std::string &path,
|
||||||
|
const PlyOptions &opts = PlyOptions{});
|
||||||
|
|
||||||
} // namespace score
|
} // namespace score
|
||||||
|
|||||||
@ -6,10 +6,51 @@
|
|||||||
namespace score {
|
namespace score {
|
||||||
|
|
||||||
/// @brief CPU-based stereo matcher using cv::StereoSGBM.
|
/// @brief CPU-based stereo matcher using cv::StereoSGBM.
|
||||||
|
///
|
||||||
|
/// The matcher is configured with the smoothness penalties and post-filters
|
||||||
|
/// recommended by OpenCV (P1 = 8·bs², P2 = 32·bs², uniqueness ratio,
|
||||||
|
/// left-right consistency check, speckle filter). Without them SGBM
|
||||||
|
/// degenerates to unregularised winner-take-all block matching, which
|
||||||
|
/// produces a heavy tail of low-disparity outliers that reproject metres
|
||||||
|
/// away from the true surface.
|
||||||
class CpuStereoMatcher : public IStereoMatcher {
|
class CpuStereoMatcher : public IStereoMatcher {
|
||||||
public:
|
public:
|
||||||
|
/// @brief True when this build can apply the WLS disparity filter.
|
||||||
|
[[nodiscard]] static bool wls_available() noexcept;
|
||||||
|
|
||||||
|
/// @brief SGBM tuning parameters.
|
||||||
|
struct Params {
|
||||||
|
int block_size; ///< Odd matching block size (SADWindowSize).
|
||||||
|
int uniqueness_ratio; ///< Best/second-best cost margin (%).
|
||||||
|
int speckle_window_size; ///< Max blob size flagged as speckle (0 off).
|
||||||
|
int speckle_range; ///< Max disparity variation inside a blob.
|
||||||
|
int disp12_max_diff; ///< Max left-right disparity mismatch (px).
|
||||||
|
int pre_filter_cap; ///< x-derivative clipping value.
|
||||||
|
/// Median filter applied to the disparity map (0 disables, else odd
|
||||||
|
/// 3 or 5). Only trims isolated spikes; SGBM's sub-pixel noise is
|
||||||
|
/// correlated over several pixels, so prefer the WLS filter.
|
||||||
|
int median_kernel;
|
||||||
|
/// Edge-aware weighted-least-squares smoothing of the disparity
|
||||||
|
/// (cv::ximgproc::DisparityWLSFilter). Off by default: on SCARED it
|
||||||
|
/// halves the fine-scale surface roughness (mesh normals 45 -> 23
|
||||||
|
/// deg off-axis) but costs accuracy (MAE 0.84 -> 0.92 mm, within
|
||||||
|
/// 2 mm 80 -> 78 %) and a second SGBM pass (~2x matching time).
|
||||||
|
/// Enable for visualisation, keep off for measurement. Ignored,
|
||||||
|
/// with a warning, when OpenCV was built without ximgproc.
|
||||||
|
bool wls_filter;
|
||||||
|
double wls_lambda; ///< Smoothness weight (2000; 8000 over-smooths).
|
||||||
|
double wls_sigma; ///< Edge sensitivity (typical 0.8-2.0).
|
||||||
|
// Explicit constructor avoids a GCC limitation with nested-struct
|
||||||
|
// default-member-initialisers used as default function arguments.
|
||||||
|
Params() noexcept
|
||||||
|
: block_size(5), uniqueness_ratio(10), speckle_window_size(100),
|
||||||
|
speckle_range(2), disp12_max_diff(1), pre_filter_cap(31),
|
||||||
|
median_kernel(0), wls_filter(false), wls_lambda(2000.0),
|
||||||
|
wls_sigma(1.5) {}
|
||||||
|
};
|
||||||
|
|
||||||
CpuStereoMatcher(int min_disparity = 0, int num_disparities = 128,
|
CpuStereoMatcher(int min_disparity = 0, int num_disparities = 128,
|
||||||
int block_size = 3);
|
Params params = Params{});
|
||||||
~CpuStereoMatcher() override = default;
|
~CpuStereoMatcher() override = default;
|
||||||
|
|
||||||
[[nodiscard]] cv::Mat compute(const cv::Mat &left,
|
[[nodiscard]] cv::Mat compute(const cv::Mat &left,
|
||||||
@ -17,6 +58,11 @@ class CpuStereoMatcher : public IStereoMatcher {
|
|||||||
|
|
||||||
private:
|
private:
|
||||||
cv::Ptr<cv::StereoSGBM> sgbm_;
|
cv::Ptr<cv::StereoSGBM> sgbm_;
|
||||||
|
cv::Ptr<cv::StereoMatcher> right_matcher_; ///< Only with WLS.
|
||||||
|
cv::Ptr<cv::Algorithm> wls_; ///< DisparityWLSFilter.
|
||||||
|
int median_kernel_;
|
||||||
|
double wls_lambda_;
|
||||||
|
double wls_sigma_;
|
||||||
};
|
};
|
||||||
|
|
||||||
} // namespace score
|
} // namespace score
|
||||||
|
|||||||
@ -6,15 +6,27 @@ namespace score {
|
|||||||
|
|
||||||
/// @brief GPU-based stereo matcher using cv::cuda::StereoSGM.
|
/// @brief GPU-based stereo matcher using cv::cuda::StereoSGM.
|
||||||
/// Falls back to runtime error if CUDA is unavailable.
|
/// Falls back to runtime error if CUDA is unavailable.
|
||||||
|
///
|
||||||
|
/// cv::cuda::StereoSGM only supports 64, 128 or 256 disparity levels; the
|
||||||
|
/// requested count is rounded up to the next supported value.
|
||||||
class GpuStereoMatcher : public IStereoMatcher {
|
class GpuStereoMatcher : public IStereoMatcher {
|
||||||
public:
|
public:
|
||||||
GpuStereoMatcher(int min_disparity = 0, int num_disparities = 16,
|
/// @param min_disparity Minimum disparity (px).
|
||||||
int block_size = 3);
|
/// @param num_disparities Requested disparity levels (rounded up to
|
||||||
|
/// 64/128/256).
|
||||||
|
/// @param uniqueness_ratio Best/second-best cost margin (%).
|
||||||
|
GpuStereoMatcher(int min_disparity = 0, int num_disparities = 128,
|
||||||
|
int uniqueness_ratio = 10);
|
||||||
~GpuStereoMatcher() override = default;
|
~GpuStereoMatcher() override = default;
|
||||||
|
|
||||||
[[nodiscard]] cv::Mat compute(const cv::Mat &left,
|
[[nodiscard]] cv::Mat compute(const cv::Mat &left,
|
||||||
const cv::Mat &right) override;
|
const cv::Mat &right) override;
|
||||||
|
|
||||||
|
/// @brief Round a disparity count up to the nearest value supported by
|
||||||
|
/// cv::cuda::StereoSGM (64, 128 or 256).
|
||||||
|
/// @throws std::invalid_argument if num_disparities exceeds 256.
|
||||||
|
[[nodiscard]] static int supported_num_disparities(int num_disparities);
|
||||||
|
|
||||||
private:
|
private:
|
||||||
#ifdef HAVE_OPENCV_CUDA
|
#ifdef HAVE_OPENCV_CUDA
|
||||||
cv::Ptr<cv::cuda::StereoSGM> sgm_;
|
cv::Ptr<cv::cuda::StereoSGM> sgm_;
|
||||||
|
|||||||
@ -1,5 +1,6 @@
|
|||||||
#pragma once
|
#pragma once
|
||||||
|
|
||||||
|
#include "cloud_point/cpu_stereo_matcher.hpp"
|
||||||
#include "cloud_point/stereo_matcher.hpp"
|
#include "cloud_point/stereo_matcher.hpp"
|
||||||
#include <memory>
|
#include <memory>
|
||||||
|
|
||||||
@ -12,12 +13,16 @@ class StereoMatcherFactory {
|
|||||||
public:
|
public:
|
||||||
/// @brief Create a stereo matcher of the requested type.
|
/// @brief Create a stereo matcher of the requested type.
|
||||||
/// If GPU is requested but unavailable, falls back to CPU.
|
/// If GPU is requested but unavailable, falls back to CPU.
|
||||||
/// @param num_disparities Number of disparity levels for SGBM (default 128).
|
/// @param num_disparities Number of disparity levels for SGBM (default
|
||||||
|
/// 128).
|
||||||
/// Must be a positive multiple of 16.
|
/// Must be a positive multiple of 16.
|
||||||
|
/// @param cpu_params SGBM tuning used for the CPU matcher (and the
|
||||||
|
/// CPU fallback of the GPU matcher).
|
||||||
/// @throws std::invalid_argument if @p num_disparities is not a positive
|
/// @throws std::invalid_argument if @p num_disparities is not a positive
|
||||||
/// multiple of 16.
|
/// multiple of 16.
|
||||||
[[nodiscard]] static std::unique_ptr<IStereoMatcher>
|
[[nodiscard]] static std::unique_ptr<IStereoMatcher>
|
||||||
create(StereoAlgorithmType type, int num_disparities = 128);
|
create(StereoAlgorithmType type, int num_disparities = 128,
|
||||||
|
CpuStereoMatcher::Params cpu_params = CpuStereoMatcher::Params{});
|
||||||
};
|
};
|
||||||
|
|
||||||
} // namespace score
|
} // namespace score
|
||||||
|
|||||||
@ -5,6 +5,21 @@
|
|||||||
#include <string>
|
#include <string>
|
||||||
namespace score {
|
namespace score {
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @brief Stereo reconstruction settings for CLI options 4 and 5.
|
||||||
|
*
|
||||||
|
* Mirrors CloudPointConfig from config.hpp without pulling OpenCV into the
|
||||||
|
* CLI interface.
|
||||||
|
*/
|
||||||
|
struct CliStereoOptions {
|
||||||
|
bool use_gpu{true}; ///< GPU matcher (falls back to CPU) or CPU
|
||||||
|
int num_disparities{128}; ///< SGBM levels, positive multiple of 16
|
||||||
|
float min_depth_m{0.01f}; ///< Reject points nearer than this (m)
|
||||||
|
float max_depth_m{10.0f}; ///< Reject points farther than this (m)
|
||||||
|
int ply_stride{1}; ///< Grid decimation for PLY export
|
||||||
|
bool wls_filter{false}; ///< WLS disparity smoothing (visualisation)
|
||||||
|
};
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @brief Runs the CLI client.
|
* @brief Runs the CLI client.
|
||||||
*
|
*
|
||||||
@ -12,9 +27,11 @@ namespace score {
|
|||||||
* @param output Output stream (usually std::cout)
|
* @param output Output stream (usually std::cout)
|
||||||
* @param ip Server IP
|
* @param ip Server IP
|
||||||
* @param port Server Port
|
* @param port Server Port
|
||||||
|
* @param stereo Stereo reconstruction settings (options 4/5)
|
||||||
* @return int exit code
|
* @return int exit code
|
||||||
*/
|
*/
|
||||||
int CRPC_EXPORT run_cli(std::istream &input, std::ostream &output,
|
int CRPC_EXPORT run_cli(std::istream &input, std::ostream &output,
|
||||||
const std::string &ip, int port);
|
const std::string &ip, int port,
|
||||||
|
const CliStereoOptions &stereo = CliStereoOptions{});
|
||||||
|
|
||||||
} // namespace score
|
} // namespace score
|
||||||
|
|||||||
@ -19,9 +19,28 @@ struct TestData {
|
|||||||
std::vector<std::vector<double>> cloud_point;
|
std::vector<std::vector<double>> cloud_point;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
/// @brief Stereo reconstruction settings consumed by the CLI (option 4/5).
|
||||||
|
///
|
||||||
|
/// Depth limits are physical bounds on the scene: anything reprojected
|
||||||
|
/// outside [min_depth_m, max_depth_m] is discarded as a mismatch. For the
|
||||||
|
/// SCARED endoscopic rig use roughly 0.02–0.3 m; the defaults are wide enough
|
||||||
|
/// for a generic Unity scene.
|
||||||
|
struct CloudPointConfig {
|
||||||
|
std::string algorithm{"gpu"}; ///< "gpu" (falls back to CPU) or "cpu"
|
||||||
|
int num_disparities{128}; ///< SGBM levels, positive multiple of 16
|
||||||
|
double min_depth_m{0.01};
|
||||||
|
double max_depth_m{10.0};
|
||||||
|
int ply_stride{1}; ///< Grid decimation for PLY export (1 = full res)
|
||||||
|
/// Edge-aware WLS smoothing of the disparity map (needs opencv
|
||||||
|
/// ximgproc). Smoother meshes, slightly lower accuracy, ~2x matching
|
||||||
|
/// time; intended for visualisation.
|
||||||
|
bool wls_filter{false};
|
||||||
|
};
|
||||||
|
|
||||||
struct Config {
|
struct Config {
|
||||||
ServerConfig server;
|
ServerConfig server;
|
||||||
TestData test_data;
|
TestData test_data;
|
||||||
|
CloudPointConfig cloud_point;
|
||||||
};
|
};
|
||||||
|
|
||||||
class ConfigLoader {
|
class ConfigLoader {
|
||||||
@ -61,6 +80,39 @@ class ConfigLoader {
|
|||||||
LOG(WARNING) << "No 'test_data' section, using empty/defaults.";
|
LOG(WARNING) << "No 'test_data' section, using empty/defaults.";
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Cloud point (optional)
|
||||||
|
if (config["cloud_point"]) {
|
||||||
|
const auto &cp = config["cloud_point"];
|
||||||
|
CloudPointConfig d;
|
||||||
|
c.cloud_point.algorithm =
|
||||||
|
cp["algorithm"].as<std::string>(d.algorithm);
|
||||||
|
c.cloud_point.num_disparities =
|
||||||
|
cp["num_disparities"].as<int>(d.num_disparities);
|
||||||
|
c.cloud_point.min_depth_m =
|
||||||
|
cp["min_depth_m"].as<double>(d.min_depth_m);
|
||||||
|
c.cloud_point.max_depth_m =
|
||||||
|
cp["max_depth_m"].as<double>(d.max_depth_m);
|
||||||
|
c.cloud_point.ply_stride =
|
||||||
|
cp["ply_stride"].as<int>(d.ply_stride);
|
||||||
|
c.cloud_point.wls_filter =
|
||||||
|
cp["wls_filter"].as<bool>(d.wls_filter);
|
||||||
|
if (c.cloud_point.ply_stride < 1) {
|
||||||
|
throw std::runtime_error(
|
||||||
|
"cloud_point.ply_stride must be >= 1");
|
||||||
|
}
|
||||||
|
if (c.cloud_point.algorithm != "gpu" &&
|
||||||
|
c.cloud_point.algorithm != "cpu") {
|
||||||
|
throw std::runtime_error(
|
||||||
|
"cloud_point.algorithm must be \"gpu\" or \"cpu\"");
|
||||||
|
}
|
||||||
|
if (c.cloud_point.min_depth_m <= 0.0 ||
|
||||||
|
c.cloud_point.max_depth_m <= c.cloud_point.min_depth_m) {
|
||||||
|
throw std::runtime_error(
|
||||||
|
"cloud_point depth range must satisfy "
|
||||||
|
"0 < min_depth_m < max_depth_m");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
return c;
|
return c;
|
||||||
} catch (const YAML::Exception &e) {
|
} catch (const YAML::Exception &e) {
|
||||||
LOG(ERROR) << "Failed to load config: " << e.what();
|
LOG(ERROR) << "Failed to load config: " << e.what();
|
||||||
|
|||||||
83
scripts/scared_overview.py
Executable file
83
scripts/scared_overview.py
Executable file
@ -0,0 +1,83 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Run scared_dataset_benchmark over many keyframes and print a Markdown table.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
scripts/scared_overview.py [--bench PATH] [--disparities N] [--png-dir DIR]
|
||||||
|
[--json-dir DIR] [--depth-range MIN_M MAX_M]
|
||||||
|
KEYFRAME_DIR...
|
||||||
|
|
||||||
|
Each KEYFRAME_DIR must hold Left_Image.png, Right_Image.png and
|
||||||
|
endoscope_calibration.yaml. Accuracy columns are filled in only for keyframes
|
||||||
|
that also contain point_cloud.obj.
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
|
||||||
|
def run(bench, kf, disparities, png_dir, json_dir, depth_range):
|
||||||
|
label = "/".join(kf.rstrip("/").split("/")[-2:])
|
||||||
|
cmd = [bench, kf, str(disparities)]
|
||||||
|
if png_dir:
|
||||||
|
os.makedirs(png_dir, exist_ok=True)
|
||||||
|
cmd.append(os.path.join(png_dir, label.replace("/", "_") + ".png"))
|
||||||
|
else:
|
||||||
|
cmd.append("-")
|
||||||
|
if depth_range:
|
||||||
|
cmd += [str(depth_range[0]), str(depth_range[1])]
|
||||||
|
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
print(f"{label}: benchmark failed\n{proc.stderr}", file=sys.stderr)
|
||||||
|
return label, None
|
||||||
|
result = json.loads(proc.stdout)
|
||||||
|
if json_dir:
|
||||||
|
os.makedirs(json_dir, exist_ok=True)
|
||||||
|
with open(os.path.join(json_dir, label.replace("/", "_") + ".json"), "w") as f:
|
||||||
|
json.dump(result, f, indent=2)
|
||||||
|
return label, result
|
||||||
|
|
||||||
|
|
||||||
|
def fmt_mm(v):
|
||||||
|
return "" if v is None else f"{v * 1000:.2f}"
|
||||||
|
|
||||||
|
|
||||||
|
def fmt_pct(v):
|
||||||
|
return "" if v is None else f"{v * 100:.1f}"
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser()
|
||||||
|
ap.add_argument("--bench", default="build-opencv/src/cloud_point/scared_dataset_benchmark")
|
||||||
|
ap.add_argument("--disparities", type=int, default=160)
|
||||||
|
ap.add_argument("--png-dir")
|
||||||
|
ap.add_argument("--json-dir")
|
||||||
|
ap.add_argument("--depth-range", nargs=2, type=float, metavar=("MIN_M", "MAX_M"),
|
||||||
|
help="depth filter in metres (default: builder defaults 0.01..10)")
|
||||||
|
ap.add_argument("keyframes", nargs="+")
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
rows = [run(args.bench, kf, args.disparities, args.png_dir, args.json_dir,
|
||||||
|
args.depth_range)
|
||||||
|
for kf in args.keyframes]
|
||||||
|
|
||||||
|
print("| keyframe | valid % | z p5 / median / p95 (mm) | match ms | GT | coverage % | MAE3D mm | RMSE3D mm | median mm | <1 mm % | <2 mm % | <5 mm % |")
|
||||||
|
print("|---|---|---|---|---|---|---|---|---|---|---|---|")
|
||||||
|
for label, r in rows:
|
||||||
|
if r is None:
|
||||||
|
print(f"| {label} | failed | | | | | | | | | | |")
|
||||||
|
continue
|
||||||
|
z = f"{r['z_p05_m']*1000:.0f} / {r['z_median_m']*1000:.0f} / {r['z_p95_m']*1000:.0f}"
|
||||||
|
gt = r.get("has_ground_truth", False)
|
||||||
|
print("| {} | {} | {} | {:.0f} | {} | {} | {} | {} | {} | {} | {} | {} |".format(
|
||||||
|
label, fmt_pct(r["valid_fraction"]), z, r["matching_ms"],
|
||||||
|
"yes" if gt else "no",
|
||||||
|
fmt_pct(r.get("coverage")), fmt_mm(r.get("mae_3d_m")),
|
||||||
|
fmt_mm(r.get("rmse_3d_m")), fmt_mm(r.get("median_3d_m")),
|
||||||
|
fmt_pct(r.get("within_1mm")), fmt_pct(r.get("within_2mm")),
|
||||||
|
fmt_pct(r.get("within_5mm"))))
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
22
src/cli.cpp
22
src/cli.cpp
@ -41,7 +41,7 @@ std::string vector_to_string(const std::vector<std::vector<T>> &v) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
int run_cli(std::istream &input, std::ostream &output, const std::string &ip,
|
int run_cli(std::istream &input, std::ostream &output, const std::string &ip,
|
||||||
int port) {
|
int port, const CliStereoOptions &stereo) {
|
||||||
try {
|
try {
|
||||||
TCPConnector connector(ip, port);
|
TCPConnector connector(ip, port);
|
||||||
RpcClient client(connector);
|
RpcClient client(connector);
|
||||||
@ -86,7 +86,15 @@ int run_cli(std::istream &input, std::ostream &output, const std::string &ip,
|
|||||||
} else if (choice == "4" || choice == "5") {
|
} else if (choice == "4" || choice == "5") {
|
||||||
#ifdef HAVE_CLOUD_POINT_COMPUTE
|
#ifdef HAVE_CLOUD_POINT_COMPUTE
|
||||||
try {
|
try {
|
||||||
CloudPointClient cpc(ip, port, StereoAlgorithmType::GPU);
|
const auto algo = stereo.use_gpu ? StereoAlgorithmType::GPU
|
||||||
|
: StereoAlgorithmType::CPU;
|
||||||
|
CpuStereoMatcher::Params matcher;
|
||||||
|
matcher.wls_filter = stereo.wls_filter;
|
||||||
|
CloudPointClient cpc(
|
||||||
|
ip, port, algo,
|
||||||
|
PointCloudBuilder::Options{stereo.min_depth_m,
|
||||||
|
stereo.max_depth_m},
|
||||||
|
stereo.num_disparities, matcher);
|
||||||
cpc.connect();
|
cpc.connect();
|
||||||
auto result = cpc.compute_cloud();
|
auto result = cpc.compute_cloud();
|
||||||
|
|
||||||
@ -127,9 +135,13 @@ int run_cli(std::istream &input, std::ostream &output, const std::string &ip,
|
|||||||
output << "PLY output path: ";
|
output << "PLY output path: ";
|
||||||
std::string path;
|
std::string path;
|
||||||
if (input >> path) {
|
if (input >> path) {
|
||||||
write_ply(cloud, path);
|
PlyOptions ply;
|
||||||
output << "Saved " << valid.size()
|
ply.stride = stereo.ply_stride;
|
||||||
<< " points to " << path << "\n";
|
const auto faces = write_ply(cloud, path, ply);
|
||||||
|
output << "Saved mesh (stride " << ply.stride
|
||||||
|
<< ", " << faces << " faces) from "
|
||||||
|
<< valid.size() << " valid points to "
|
||||||
|
<< path << "\n";
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@ -3,18 +3,25 @@
|
|||||||
#include "cloud_point/imageFactory.h"
|
#include "cloud_point/imageFactory.h"
|
||||||
#include "cloud_point_rpc/rpc_client.hpp"
|
#include "cloud_point_rpc/rpc_client.hpp"
|
||||||
#include "cloud_point_rpc/tcp_connector.hpp"
|
#include "cloud_point_rpc/tcp_connector.hpp"
|
||||||
|
#include <algorithm>
|
||||||
|
#include <array>
|
||||||
|
#include <bit>
|
||||||
|
#include <cmath>
|
||||||
#include <fstream>
|
#include <fstream>
|
||||||
#include <jsonrpccxx/common.hpp>
|
#include <jsonrpccxx/common.hpp>
|
||||||
#include <opencv2/imgproc.hpp>
|
#include <opencv2/imgproc.hpp>
|
||||||
|
#include <stdexcept>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
namespace score {
|
namespace score {
|
||||||
|
|
||||||
CloudPointClient::CloudPointClient(std::string ip, int port,
|
CloudPointClient::CloudPointClient(std::string ip, int port,
|
||||||
StereoAlgorithmType algo,
|
StereoAlgorithmType algo,
|
||||||
PointCloudBuilder::Options opts,
|
PointCloudBuilder::Options opts,
|
||||||
int num_disparities)
|
int num_disparities,
|
||||||
|
CpuStereoMatcher::Params matcher_params)
|
||||||
: ip_(std::move(ip)), port_(port), algo_(algo), opts_(opts),
|
: ip_(std::move(ip)), port_(port), algo_(algo), opts_(opts),
|
||||||
num_disparities_(num_disparities) {}
|
num_disparities_(num_disparities), matcher_params_(matcher_params) {}
|
||||||
|
|
||||||
CloudPointClient::~CloudPointClient() = default;
|
CloudPointClient::~CloudPointClient() = default;
|
||||||
|
|
||||||
@ -25,7 +32,8 @@ void CloudPointClient::connect() {
|
|||||||
const auto calib_rpc = client_->get_stereo_calibration();
|
const auto calib_rpc = client_->get_stereo_calibration();
|
||||||
const auto calib = StereoRectifier::Calibration::from_rpc(calib_rpc);
|
const auto calib = StereoRectifier::Calibration::from_rpc(calib_rpc);
|
||||||
rectifier_ = std::make_unique<StereoRectifier>(calib);
|
rectifier_ = std::make_unique<StereoRectifier>(calib);
|
||||||
matcher_ = StereoMatcherFactory::create(algo_, num_disparities_);
|
matcher_ = StereoMatcherFactory::create(algo_, num_disparities_,
|
||||||
|
matcher_params_);
|
||||||
builder_ = std::make_unique<PointCloudBuilder>(rectifier_->q(), opts_);
|
builder_ = std::make_unique<PointCloudBuilder>(rectifier_->q(), opts_);
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -95,21 +103,164 @@ CloudPointClient::compute_cloud() {
|
|||||||
// PLY helper
|
// PLY helper
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
void write_ply(const PointCloud &cloud, const std::string &path) {
|
namespace {
|
||||||
const auto valid = cloud.valid_points();
|
|
||||||
|
|
||||||
std::ofstream out(path);
|
struct Vertex {
|
||||||
|
float x, y, z;
|
||||||
|
};
|
||||||
|
|
||||||
|
/// Longest edge of a triangle must not exceed max_ratio * mean depth.
|
||||||
|
bool triangle_ok(const Vertex &a, const Vertex &b, const Vertex &c,
|
||||||
|
float max_ratio) {
|
||||||
|
const auto dist2 = [](const Vertex &p, const Vertex &q) {
|
||||||
|
const float dx = p.x - q.x, dy = p.y - q.y, dz = p.z - q.z;
|
||||||
|
return dx * dx + dy * dy + dz * dz;
|
||||||
|
};
|
||||||
|
const float longest2 = std::max({dist2(a, b), dist2(b, c), dist2(c, a)});
|
||||||
|
const float limit = max_ratio * (a.z + b.z + c.z) / 3.0f;
|
||||||
|
return longest2 <= limit * limit;
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace
|
||||||
|
|
||||||
|
size_t write_ply(const PointCloud &cloud, const std::string &path,
|
||||||
|
const PlyOptions &opts) {
|
||||||
|
if (opts.stride < 1) {
|
||||||
|
throw std::invalid_argument("write_ply: stride must be >= 1, got " +
|
||||||
|
std::to_string(opts.stride));
|
||||||
|
}
|
||||||
|
// Decimated grid dimensions; pixel (r, c) of the sub-grid maps to
|
||||||
|
// (r * stride, c * stride) of the source cloud.
|
||||||
|
const int grid_w = (cloud.width + opts.stride - 1) / opts.stride;
|
||||||
|
const int grid_h = (cloud.height + opts.stride - 1) / opts.stride;
|
||||||
|
|
||||||
|
// Map every valid sub-grid cell to its index in the vertex list. With
|
||||||
|
// stride > 1 each vertex is the mean of the valid pixels in its
|
||||||
|
// stride x stride block, which divides the per-pixel disparity noise by
|
||||||
|
// roughly the stride and yields a far smoother mesh than sub-sampling.
|
||||||
|
const size_t n_px =
|
||||||
|
static_cast<size_t>(grid_w) * static_cast<size_t>(grid_h);
|
||||||
|
std::vector<int> index(n_px, -1);
|
||||||
|
std::vector<Vertex> vertices;
|
||||||
|
vertices.reserve(n_px);
|
||||||
|
for (int gr = 0; gr < grid_h; ++gr) {
|
||||||
|
for (int gc = 0; gc < grid_w; ++gc) {
|
||||||
|
double sx = 0.0, sy = 0.0, sz = 0.0;
|
||||||
|
int count = 0;
|
||||||
|
const int row_begin = gr * opts.stride;
|
||||||
|
const int col_begin = gc * opts.stride;
|
||||||
|
const int row_end = std::min(row_begin + opts.stride, cloud.height);
|
||||||
|
const int col_end = std::min(col_begin + opts.stride, cloud.width);
|
||||||
|
for (int r = row_begin; r < row_end; ++r) {
|
||||||
|
for (int c = col_begin; c < col_end; ++c) {
|
||||||
|
const size_t src = (static_cast<size_t>(r) *
|
||||||
|
static_cast<size_t>(cloud.width) +
|
||||||
|
static_cast<size_t>(c)) *
|
||||||
|
3u;
|
||||||
|
const float x = cloud.data[src];
|
||||||
|
const float y = cloud.data[src + 1];
|
||||||
|
const float z = cloud.data[src + 2];
|
||||||
|
if (!std::isnan(x) && !std::isnan(y) && !std::isnan(z)) {
|
||||||
|
sx += x;
|
||||||
|
sy += y;
|
||||||
|
sz += z;
|
||||||
|
++count;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Require at least half the block to be valid so that a lone
|
||||||
|
// pixel cannot fabricate a vertex inside a hole.
|
||||||
|
const int block = (row_end - row_begin) * (col_end - col_begin);
|
||||||
|
if (count * 2 >= block && count > 0) {
|
||||||
|
index[static_cast<size_t>(gr) * static_cast<size_t>(grid_w) +
|
||||||
|
static_cast<size_t>(gc)] =
|
||||||
|
static_cast<int>(vertices.size());
|
||||||
|
vertices.push_back({static_cast<float>(sx / count),
|
||||||
|
static_cast<float>(sy / count),
|
||||||
|
static_cast<float>(sz / count)});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Grid triangulation: each 2x2 cell yields up to two triangles, wound so
|
||||||
|
// the normal faces the camera (-z in OpenCV coordinates).
|
||||||
|
std::vector<std::array<int, 3>> faces;
|
||||||
|
if (opts.triangulate) {
|
||||||
|
faces.reserve(2 * n_px);
|
||||||
|
const auto at = [&](int r, int c) {
|
||||||
|
return index[static_cast<size_t>(r) * static_cast<size_t>(grid_w) +
|
||||||
|
static_cast<size_t>(c)];
|
||||||
|
};
|
||||||
|
const auto emit = [&](int i0, int i1, int i2) {
|
||||||
|
if (i0 < 0 || i1 < 0 || i2 < 0)
|
||||||
|
return;
|
||||||
|
if (triangle_ok(vertices[static_cast<size_t>(i0)],
|
||||||
|
vertices[static_cast<size_t>(i1)],
|
||||||
|
vertices[static_cast<size_t>(i2)],
|
||||||
|
opts.max_edge_depth_ratio)) {
|
||||||
|
faces.push_back({i0, i1, i2});
|
||||||
|
}
|
||||||
|
};
|
||||||
|
for (int r = 0; r + 1 < grid_h; ++r) {
|
||||||
|
for (int c = 0; c + 1 < grid_w; ++c) {
|
||||||
|
const int i00 = at(r, c), i01 = at(r, c + 1);
|
||||||
|
const int i10 = at(r + 1, c), i11 = at(r + 1, c + 1);
|
||||||
|
const int valid =
|
||||||
|
(i00 >= 0) + (i01 >= 0) + (i10 >= 0) + (i11 >= 0);
|
||||||
|
if (valid == 4) {
|
||||||
|
emit(i00, i10, i11);
|
||||||
|
emit(i00, i11, i01);
|
||||||
|
} else if (valid == 3) {
|
||||||
|
// One missing corner: keep the single remaining triangle.
|
||||||
|
if (i00 < 0)
|
||||||
|
emit(i10, i11, i01);
|
||||||
|
else if (i01 < 0)
|
||||||
|
emit(i00, i10, i11);
|
||||||
|
else if (i10 < 0)
|
||||||
|
emit(i00, i11, i01);
|
||||||
|
else
|
||||||
|
emit(i00, i10, i01);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
std::ofstream out(path, std::ios::binary);
|
||||||
out << "ply\n"
|
out << "ply\n"
|
||||||
<< "format ascii 1.0\n"
|
<< (opts.binary ? "format binary_little_endian 1.0\n"
|
||||||
<< "element vertex " << valid.size() << "\n"
|
: "format ascii 1.0\n")
|
||||||
|
<< "element vertex " << vertices.size() << "\n"
|
||||||
<< "property float x\n"
|
<< "property float x\n"
|
||||||
<< "property float y\n"
|
<< "property float y\n"
|
||||||
<< "property float z\n"
|
<< "property float z\n";
|
||||||
<< "end_header\n";
|
if (opts.triangulate) {
|
||||||
|
out << "element face " << faces.size() << "\n"
|
||||||
for (const auto &pt : valid) {
|
<< "property list uchar int vertex_indices\n";
|
||||||
out << pt[0] << " " << pt[1] << " " << pt[2] << "\n";
|
|
||||||
}
|
}
|
||||||
|
out << "end_header\n";
|
||||||
|
|
||||||
|
if (opts.binary) {
|
||||||
|
static_assert(std::endian::native == std::endian::little,
|
||||||
|
"binary PLY writer assumes a little-endian host");
|
||||||
|
static_assert(sizeof(Vertex) == 3 * sizeof(float));
|
||||||
|
out.write(
|
||||||
|
reinterpret_cast<const char *>(vertices.data()),
|
||||||
|
static_cast<std::streamsize>(vertices.size() * sizeof(Vertex)));
|
||||||
|
for (const auto &f : faces) {
|
||||||
|
const unsigned char count = 3;
|
||||||
|
out.write(reinterpret_cast<const char *>(&count), 1);
|
||||||
|
out.write(reinterpret_cast<const char *>(f.data()),
|
||||||
|
static_cast<std::streamsize>(3 * sizeof(int)));
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
for (const auto &v : vertices) {
|
||||||
|
out << v.x << " " << v.y << " " << v.z << "\n";
|
||||||
|
}
|
||||||
|
for (const auto &f : faces) {
|
||||||
|
out << "3 " << f[0] << " " << f[1] << " " << f[2] << "\n";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return faces.size();
|
||||||
}
|
}
|
||||||
|
|
||||||
} // namespace score
|
} // namespace score
|
||||||
|
|||||||
@ -1,15 +1,102 @@
|
|||||||
#include "cloud_point/cpu_stereo_matcher.hpp"
|
#include "cloud_point/cpu_stereo_matcher.hpp"
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
|
#include <glog/logging.h>
|
||||||
|
#include <opencv2/imgproc.hpp>
|
||||||
|
#include <stdexcept>
|
||||||
|
#include <string>
|
||||||
|
|
||||||
|
#ifdef CLOUD_POINT_HAVE_OPENCV_XIMGPROC
|
||||||
|
#include <opencv2/ximgproc/disparity_filter.hpp>
|
||||||
|
#endif
|
||||||
|
|
||||||
namespace score {
|
namespace score {
|
||||||
|
|
||||||
|
bool CpuStereoMatcher::wls_available() noexcept {
|
||||||
|
#ifdef CLOUD_POINT_HAVE_OPENCV_XIMGPROC
|
||||||
|
return true;
|
||||||
|
#else
|
||||||
|
return false;
|
||||||
|
#endif
|
||||||
|
}
|
||||||
|
|
||||||
CpuStereoMatcher::CpuStereoMatcher(int min_disparity, int num_disparities,
|
CpuStereoMatcher::CpuStereoMatcher(int min_disparity, int num_disparities,
|
||||||
int block_size) {
|
Params params)
|
||||||
sgbm_ = cv::StereoSGBM::create(min_disparity, num_disparities, block_size);
|
: median_kernel_(params.median_kernel), wls_lambda_(params.wls_lambda),
|
||||||
|
wls_sigma_(params.wls_sigma) {
|
||||||
|
if (params.median_kernel != 0 && params.median_kernel != 3 &&
|
||||||
|
params.median_kernel != 5) {
|
||||||
|
throw std::invalid_argument(
|
||||||
|
"CpuStereoMatcher: median_kernel must be 0, 3 or 5, got " +
|
||||||
|
std::to_string(params.median_kernel));
|
||||||
|
}
|
||||||
|
if (params.block_size < 1 || params.block_size % 2 == 0) {
|
||||||
|
throw std::invalid_argument(
|
||||||
|
"CpuStereoMatcher: block_size must be a positive odd number, got " +
|
||||||
|
std::to_string(params.block_size));
|
||||||
|
}
|
||||||
|
|
||||||
|
const int bs2 = params.block_size * params.block_size;
|
||||||
|
// OpenCV's recommended penalties for a single-channel input.
|
||||||
|
const int p1 = 8 * bs2;
|
||||||
|
const int p2 = 32 * bs2;
|
||||||
|
|
||||||
|
sgbm_ = cv::StereoSGBM::create(
|
||||||
|
min_disparity, num_disparities, params.block_size, p1, p2,
|
||||||
|
params.disp12_max_diff, params.pre_filter_cap, params.uniqueness_ratio,
|
||||||
|
params.speckle_window_size, params.speckle_range,
|
||||||
|
cv::StereoSGBM::MODE_SGBM);
|
||||||
|
|
||||||
|
if (params.wls_filter) {
|
||||||
|
#ifdef CLOUD_POINT_HAVE_OPENCV_XIMGPROC
|
||||||
|
// NB: createDisparityWLSFilter(sgbm_) would silently disable the
|
||||||
|
// left matcher's uniqueness/speckle/LRC post-filters to obtain a
|
||||||
|
// dense map for its own confidence estimate. The generic factory
|
||||||
|
// leaves our tuned matcher alone; we then keep SGBM's holes invalid.
|
||||||
|
right_matcher_ = cv::ximgproc::createRightMatcher(sgbm_);
|
||||||
|
auto wls = cv::ximgproc::createDisparityWLSFilterGeneric(
|
||||||
|
/*use_confidence=*/true);
|
||||||
|
wls->setLambda(params.wls_lambda);
|
||||||
|
wls->setSigmaColor(params.wls_sigma);
|
||||||
|
wls->setLRCthresh(24); // 1.5 px in SGBM's 16x fixed point
|
||||||
|
wls->setDepthDiscontinuityRadius(
|
||||||
|
std::max(1, params.block_size / 2 + 1));
|
||||||
|
wls_ = wls;
|
||||||
|
#else
|
||||||
|
LOG(WARNING) << "CpuStereoMatcher: WLS filter requested but OpenCV "
|
||||||
|
"was built without ximgproc; disparity is unfiltered";
|
||||||
|
#endif
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
cv::Mat CpuStereoMatcher::compute(const cv::Mat &left, const cv::Mat &right) {
|
cv::Mat CpuStereoMatcher::compute(const cv::Mat &left, const cv::Mat &right) {
|
||||||
cv::Mat disparity;
|
cv::Mat disparity;
|
||||||
sgbm_->compute(left, right, disparity);
|
sgbm_->compute(left, right, disparity);
|
||||||
|
|
||||||
|
#ifdef CLOUD_POINT_HAVE_OPENCV_XIMGPROC
|
||||||
|
// The WLS filter needs a valid ROI to the right of the disparity search
|
||||||
|
// range; skip it (raw SGBM output) for images narrower than that.
|
||||||
|
const int roi_width = sgbm_->getMinDisparity() +
|
||||||
|
sgbm_->getNumDisparities() + sgbm_->getBlockSize();
|
||||||
|
if (wls_ && left.cols > roi_width) {
|
||||||
|
// Snapshot SGBM's holes first: the filter may modify its inputs.
|
||||||
|
const cv::Mat holes = disparity <= 0;
|
||||||
|
cv::Mat right_disparity, filtered;
|
||||||
|
right_matcher_->compute(right, left, right_disparity);
|
||||||
|
auto wls = wls_.dynamicCast<cv::ximgproc::DisparityWLSFilter>();
|
||||||
|
wls->filter(disparity, left, filtered, right_disparity);
|
||||||
|
// WLS extrapolates into pixels SGBM rejected; keep those invalid so
|
||||||
|
// holes stay holes and coverage is not inflated with guesses.
|
||||||
|
filtered.setTo(cv::Scalar(sgbm_->getMinDisparity() * 16 - 16), holes);
|
||||||
|
disparity = filtered;
|
||||||
|
}
|
||||||
|
#endif
|
||||||
|
|
||||||
|
if (median_kernel_ > 0) {
|
||||||
|
cv::Mat filtered;
|
||||||
|
cv::medianBlur(disparity, filtered, median_kernel_);
|
||||||
|
disparity = filtered;
|
||||||
|
}
|
||||||
return disparity;
|
return disparity;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -1,5 +1,6 @@
|
|||||||
#include "cloud_point/gpu_stereo_matcher.hpp"
|
#include "cloud_point/gpu_stereo_matcher.hpp"
|
||||||
#include <stdexcept>
|
#include <stdexcept>
|
||||||
|
#include <string>
|
||||||
|
|
||||||
#ifdef HAVE_OPENCV_CUDA
|
#ifdef HAVE_OPENCV_CUDA
|
||||||
#include <opencv2/cudaimgproc.hpp>
|
#include <opencv2/cudaimgproc.hpp>
|
||||||
@ -8,18 +9,35 @@
|
|||||||
|
|
||||||
namespace score {
|
namespace score {
|
||||||
|
|
||||||
|
int GpuStereoMatcher::supported_num_disparities(int num_disparities) {
|
||||||
|
if (num_disparities <= 64)
|
||||||
|
return 64;
|
||||||
|
if (num_disparities <= 128)
|
||||||
|
return 128;
|
||||||
|
if (num_disparities <= 256)
|
||||||
|
return 256;
|
||||||
|
throw std::invalid_argument(
|
||||||
|
"GpuStereoMatcher: cv::cuda::StereoSGM supports at most 256 "
|
||||||
|
"disparities, got " +
|
||||||
|
std::to_string(num_disparities));
|
||||||
|
}
|
||||||
|
|
||||||
GpuStereoMatcher::GpuStereoMatcher(int min_disparity, int num_disparities,
|
GpuStereoMatcher::GpuStereoMatcher(int min_disparity, int num_disparities,
|
||||||
int block_size) {
|
int uniqueness_ratio) {
|
||||||
|
const int levels = supported_num_disparities(num_disparities);
|
||||||
#ifdef HAVE_OPENCV_CUDA
|
#ifdef HAVE_OPENCV_CUDA
|
||||||
if (cv::cuda::getCudaEnabledDeviceCount() == 0) {
|
if (cv::cuda::getCudaEnabledDeviceCount() == 0) {
|
||||||
throw std::runtime_error("No CUDA devices available");
|
throw std::runtime_error("No CUDA devices available");
|
||||||
}
|
}
|
||||||
sgm_ =
|
// P1/P2 follow the cv::cuda::StereoSGM defaults (10/120), which are
|
||||||
cv::cuda::createStereoSGM(min_disparity, num_disparities, block_size);
|
// expressed on the census-transform cost scale rather than SAD.
|
||||||
|
sgm_ = cv::cuda::createStereoSGM(min_disparity, levels, /*P1=*/10,
|
||||||
|
/*P2=*/120, uniqueness_ratio,
|
||||||
|
cv::cuda::StereoSGM::MODE_HH4);
|
||||||
#else
|
#else
|
||||||
(void)min_disparity;
|
(void)min_disparity;
|
||||||
(void)num_disparities;
|
(void)levels;
|
||||||
(void)block_size;
|
(void)uniqueness_ratio;
|
||||||
throw std::runtime_error("OpenCV CUDA modules not available in this build");
|
throw std::runtime_error("OpenCV CUDA modules not available in this build");
|
||||||
#endif
|
#endif
|
||||||
}
|
}
|
||||||
|
|||||||
@ -31,6 +31,19 @@ if opencv_cuda_available
|
|||||||
cpp_args += '-DHAVE_OPENCV_CUDA'
|
cpp_args += '-DHAVE_OPENCV_CUDA'
|
||||||
endif
|
endif
|
||||||
|
|
||||||
|
# Optional ximgproc (opencv_contrib) for the WLS disparity post-filter.
|
||||||
|
opencv_ximgproc_lib = disabler()
|
||||||
|
if cxx.has_header('opencv2/ximgproc/disparity_filter.hpp', dependencies: opencv_dep)
|
||||||
|
opencv_ximgproc_lib = cxx.find_library('opencv_ximgproc', required: false)
|
||||||
|
endif
|
||||||
|
if opencv_ximgproc_lib.found()
|
||||||
|
cpc_deps += opencv_ximgproc_lib
|
||||||
|
cpp_args += '-DCLOUD_POINT_HAVE_OPENCV_XIMGPROC'
|
||||||
|
message('opencv_ximgproc found: WLS disparity filter enabled')
|
||||||
|
else
|
||||||
|
message('opencv_ximgproc not found: WLS disparity filter disabled')
|
||||||
|
endif
|
||||||
|
|
||||||
cloud_point_compute_lib = shared_library('cloud_point_compute',
|
cloud_point_compute_lib = shared_library('cloud_point_compute',
|
||||||
sources: cloud_point_sources,
|
sources: cloud_point_sources,
|
||||||
include_directories: inc,
|
include_directories: inc,
|
||||||
|
|||||||
@ -1,13 +1,20 @@
|
|||||||
/// @file scared_dataset_benchmark.cpp
|
/// @file scared_dataset_benchmark.cpp
|
||||||
/// @brief Evaluate the CPU stereo reconstruction against SCARED XYZ ground
|
/// @brief Reconstruct a SCARED keyframe with the CPU stereo pipeline, report
|
||||||
/// truth.
|
/// depth statistics and, when point_cloud.obj is present, accuracy against
|
||||||
|
/// the XYZ ground truth.
|
||||||
|
#include <algorithm>
|
||||||
#include <chrono>
|
#include <chrono>
|
||||||
|
#include <cmath>
|
||||||
|
#include <filesystem>
|
||||||
#include <iostream>
|
#include <iostream>
|
||||||
|
#include <optional>
|
||||||
#include <stdexcept>
|
#include <stdexcept>
|
||||||
#include <string>
|
#include <string>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
#include <glog/logging.h>
|
#include <glog/logging.h>
|
||||||
#include <nlohmann/json.hpp>
|
#include <nlohmann/json.hpp>
|
||||||
|
#include <opencv2/imgcodecs.hpp>
|
||||||
#include <opencv2/imgproc.hpp>
|
#include <opencv2/imgproc.hpp>
|
||||||
|
|
||||||
#include "cloud_point/imageFactory.h"
|
#include "cloud_point/imageFactory.h"
|
||||||
@ -37,34 +44,114 @@ cv::Mat to_gray(const score::ImageRPC &rpc) {
|
|||||||
return gray;
|
return gray;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Depth percentiles (metres) over the valid points of a cloud.
|
||||||
|
struct DepthStats {
|
||||||
|
std::size_t valid_points{0};
|
||||||
|
double valid_fraction{0.0};
|
||||||
|
double z_min_m{0.0}, z_p05_m{0.0}, z_median_m{0.0}, z_p95_m{0.0},
|
||||||
|
z_max_m{0.0};
|
||||||
|
};
|
||||||
|
|
||||||
|
DepthStats depth_stats(const score::PointCloud &cloud) {
|
||||||
|
std::vector<float> z;
|
||||||
|
z.reserve(static_cast<std::size_t>(cloud.width) *
|
||||||
|
static_cast<std::size_t>(cloud.height));
|
||||||
|
for (const auto &pt : cloud.valid_points())
|
||||||
|
z.push_back(pt[2]);
|
||||||
|
DepthStats stats;
|
||||||
|
stats.valid_points = z.size();
|
||||||
|
stats.valid_fraction =
|
||||||
|
static_cast<double>(z.size()) /
|
||||||
|
(static_cast<double>(cloud.width) * static_cast<double>(cloud.height));
|
||||||
|
if (z.empty())
|
||||||
|
return stats;
|
||||||
|
std::sort(z.begin(), z.end());
|
||||||
|
const auto at = [&](double q) {
|
||||||
|
return static_cast<double>(
|
||||||
|
z[std::min(z.size() - 1, static_cast<std::size_t>(
|
||||||
|
q * static_cast<double>(z.size())))]);
|
||||||
|
};
|
||||||
|
stats.z_min_m = z.front();
|
||||||
|
stats.z_p05_m = at(0.05);
|
||||||
|
stats.z_median_m = at(0.5);
|
||||||
|
stats.z_p95_m = at(0.95);
|
||||||
|
stats.z_max_m = z.back();
|
||||||
|
return stats;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Colour-mapped depth image (invalid pixels black) for visual inspection.
|
||||||
|
void write_depth_png(const score::PointCloud &cloud, const std::string &path,
|
||||||
|
double z_lo, double z_hi) {
|
||||||
|
cv::Mat depth8(cloud.height, cloud.width, CV_8UC1, cv::Scalar(0));
|
||||||
|
cv::Mat valid(cloud.height, cloud.width, CV_8UC1, cv::Scalar(0));
|
||||||
|
const double span = std::max(z_hi - z_lo, 1e-6);
|
||||||
|
for (int r = 0; r < cloud.height; ++r) {
|
||||||
|
for (int c = 0; c < cloud.width; ++c) {
|
||||||
|
const std::size_t idx = (static_cast<std::size_t>(r) *
|
||||||
|
static_cast<std::size_t>(cloud.width) +
|
||||||
|
static_cast<std::size_t>(c)) *
|
||||||
|
3u;
|
||||||
|
const float z = cloud.data[idx + 2];
|
||||||
|
if (std::isnan(z))
|
||||||
|
continue;
|
||||||
|
const double t = std::clamp((z - z_lo) / span, 0.0, 1.0);
|
||||||
|
depth8.at<uchar>(r, c) = static_cast<uchar>(255.0 * (1.0 - t));
|
||||||
|
valid.at<uchar>(r, c) = 255;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
cv::Mat colour;
|
||||||
|
cv::applyColorMap(depth8, colour, cv::COLORMAP_TURBO);
|
||||||
|
colour.setTo(cv::Scalar(0, 0, 0), valid == 0);
|
||||||
|
if (!cv::imwrite(path, colour))
|
||||||
|
throw std::runtime_error("failed to write " + path);
|
||||||
|
}
|
||||||
|
|
||||||
} // namespace
|
} // namespace
|
||||||
|
|
||||||
int main(int argc, char *argv[]) {
|
int main(int argc, char *argv[]) {
|
||||||
google::InitGoogleLogging(argv[0]);
|
google::InitGoogleLogging(argv[0]);
|
||||||
FLAGS_alsologtostderr = true;
|
FLAGS_alsologtostderr = true;
|
||||||
|
|
||||||
if (argc < 2 || argc > 3) {
|
if (argc < 2 || argc > 6 || argc == 5) {
|
||||||
std::cerr << "Usage: " << argv[0]
|
std::cerr << "Usage: " << argv[0]
|
||||||
<< " <keyframe_dir> [num_disparities]\n";
|
<< " <keyframe_dir> [num_disparities] [depth_png|-] "
|
||||||
|
"[min_depth_m max_depth_m]\n"
|
||||||
|
<< " Accuracy metrics are reported when "
|
||||||
|
"<keyframe_dir>/point_cloud.obj exists;\n"
|
||||||
|
<< " depth statistics are always reported.\n";
|
||||||
return 1;
|
return 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const std::string keyframe_dir = argv[1];
|
const std::string keyframe_dir = argv[1];
|
||||||
const int num_disparities = argc == 3 ? std::stoi(argv[2]) : 160;
|
const int num_disparities = argc >= 3 ? std::stoi(argv[2]) : 160;
|
||||||
|
std::string depth_png = argc >= 4 ? argv[3] : "";
|
||||||
|
if (depth_png == "-")
|
||||||
|
depth_png.clear();
|
||||||
|
score::PointCloudBuilder::Options depth_range;
|
||||||
|
if (argc == 6) {
|
||||||
|
depth_range = score::PointCloudBuilder::Options{std::stof(argv[4]),
|
||||||
|
std::stof(argv[5])};
|
||||||
|
}
|
||||||
|
|
||||||
score::ScaredDatasetLoader dataset(keyframe_dir);
|
score::ScaredDatasetLoader dataset(keyframe_dir);
|
||||||
const auto &calibration_rpc = dataset.calibration();
|
const auto &calibration_rpc = dataset.calibration();
|
||||||
score::ScaredGroundTruthLoader ground_truth(
|
const cv::Size image_size(calibration_rpc.width,
|
||||||
keyframe_dir,
|
calibration_rpc.height);
|
||||||
cv::Size(calibration_rpc.width, calibration_rpc.height));
|
std::optional<score::ScaredGroundTruthLoader> ground_truth;
|
||||||
|
if (std::filesystem::exists(keyframe_dir + "/point_cloud.obj")) {
|
||||||
|
ground_truth.emplace(keyframe_dir, image_size);
|
||||||
|
} else {
|
||||||
|
LOG(WARNING) << "No point_cloud.obj in " << keyframe_dir
|
||||||
|
<< "; reporting depth statistics only";
|
||||||
|
}
|
||||||
|
|
||||||
const auto calibration =
|
const auto calibration =
|
||||||
score::StereoRectifier::Calibration::from_rpc(calibration_rpc);
|
score::StereoRectifier::Calibration::from_rpc(calibration_rpc);
|
||||||
score::StereoRectifier rectifier(calibration);
|
score::StereoRectifier rectifier(calibration);
|
||||||
auto matcher = score::StereoMatcherFactory::create(
|
auto matcher = score::StereoMatcherFactory::create(
|
||||||
score::StereoAlgorithmType::CPU, num_disparities);
|
score::StereoAlgorithmType::CPU, num_disparities);
|
||||||
score::PointCloudBuilder builder(rectifier.q());
|
score::PointCloudBuilder builder(rectifier.q(), depth_range);
|
||||||
|
|
||||||
const auto pair = dataset.image_pair(0);
|
const auto pair = dataset.image_pair(0);
|
||||||
const cv::Mat left_gray = to_gray(pair.left);
|
const cv::Mat left_gray = to_gray(pair.left);
|
||||||
@ -79,10 +166,10 @@ int main(int argc, char *argv[]) {
|
|||||||
const score::PointCloud cloud = builder.build(disparity);
|
const score::PointCloud cloud = builder.build(disparity);
|
||||||
const auto reconstruction_end = std::chrono::steady_clock::now();
|
const auto reconstruction_end = std::chrono::steady_clock::now();
|
||||||
|
|
||||||
const cv::Mat rectified_ground_truth =
|
const auto stats = depth_stats(cloud);
|
||||||
rectifier.rectify_left_point_map(ground_truth.point_map());
|
if (!depth_png.empty()) {
|
||||||
const auto metrics =
|
write_depth_png(cloud, depth_png, stats.z_p05_m, stats.z_p95_m);
|
||||||
score::PointCloudEvaluator::evaluate(cloud, rectified_ground_truth);
|
}
|
||||||
|
|
||||||
const auto matching_ms =
|
const auto matching_ms =
|
||||||
std::chrono::duration<double, std::milli>(matching_end - start)
|
std::chrono::duration<double, std::milli>(matching_end - start)
|
||||||
@ -92,25 +179,46 @@ int main(int argc, char *argv[]) {
|
|||||||
matching_end)
|
matching_end)
|
||||||
.count();
|
.count();
|
||||||
|
|
||||||
const nlohmann::json output = {
|
nlohmann::json output = {
|
||||||
{"keyframe_dir", keyframe_dir},
|
{"keyframe_dir", keyframe_dir},
|
||||||
{"algorithm", "StereoSGBM"},
|
{"algorithm", "StereoSGBM"},
|
||||||
{"num_disparities", num_disparities},
|
{"num_disparities", num_disparities},
|
||||||
{"ground_truth_points", metrics.ground_truth_points},
|
{"min_depth_m", depth_range.min_depth_m},
|
||||||
{"matched_points", metrics.matched_points},
|
{"max_depth_m", depth_range.max_depth_m},
|
||||||
{"coverage", metrics.coverage},
|
{"image_width", cloud.width},
|
||||||
{"mae_x_m", metrics.mae_x_m},
|
{"image_height", cloud.height},
|
||||||
{"mae_y_m", metrics.mae_y_m},
|
{"valid_points", stats.valid_points},
|
||||||
{"mae_z_m", metrics.mae_z_m},
|
{"valid_fraction", stats.valid_fraction},
|
||||||
{"mae_3d_m", metrics.mae_3d_m},
|
{"z_min_m", stats.z_min_m},
|
||||||
{"rmse_3d_m", metrics.rmse_3d_m},
|
{"z_p05_m", stats.z_p05_m},
|
||||||
{"median_3d_m", metrics.median_3d_m},
|
{"z_median_m", stats.z_median_m},
|
||||||
{"within_1mm", metrics.within_1mm},
|
{"z_p95_m", stats.z_p95_m},
|
||||||
{"within_2mm", metrics.within_2mm},
|
{"z_max_m", stats.z_max_m},
|
||||||
{"within_5mm", metrics.within_5mm},
|
|
||||||
{"matching_ms", matching_ms},
|
{"matching_ms", matching_ms},
|
||||||
{"reconstruction_ms", reconstruction_ms},
|
{"reconstruction_ms", reconstruction_ms},
|
||||||
|
{"has_ground_truth", ground_truth.has_value()},
|
||||||
};
|
};
|
||||||
|
if (!depth_png.empty())
|
||||||
|
output["depth_png"] = depth_png;
|
||||||
|
|
||||||
|
if (ground_truth) {
|
||||||
|
const cv::Mat rectified_ground_truth =
|
||||||
|
rectifier.rectify_left_point_map(ground_truth->point_map());
|
||||||
|
const auto metrics = score::PointCloudEvaluator::evaluate(
|
||||||
|
cloud, rectified_ground_truth);
|
||||||
|
output["ground_truth_points"] = metrics.ground_truth_points;
|
||||||
|
output["matched_points"] = metrics.matched_points;
|
||||||
|
output["coverage"] = metrics.coverage;
|
||||||
|
output["mae_x_m"] = metrics.mae_x_m;
|
||||||
|
output["mae_y_m"] = metrics.mae_y_m;
|
||||||
|
output["mae_z_m"] = metrics.mae_z_m;
|
||||||
|
output["mae_3d_m"] = metrics.mae_3d_m;
|
||||||
|
output["rmse_3d_m"] = metrics.rmse_3d_m;
|
||||||
|
output["median_3d_m"] = metrics.median_3d_m;
|
||||||
|
output["within_1mm"] = metrics.within_1mm;
|
||||||
|
output["within_2mm"] = metrics.within_2mm;
|
||||||
|
output["within_5mm"] = metrics.within_5mm;
|
||||||
|
}
|
||||||
std::cout << output.dump(2) << '\n';
|
std::cout << output.dump(2) << '\n';
|
||||||
} catch (const std::exception &error) {
|
} catch (const std::exception &error) {
|
||||||
std::cerr << "Benchmark failed: " << error.what() << '\n';
|
std::cerr << "Benchmark failed: " << error.what() << '\n';
|
||||||
|
|||||||
@ -8,7 +8,8 @@
|
|||||||
namespace score {
|
namespace score {
|
||||||
|
|
||||||
std::unique_ptr<IStereoMatcher>
|
std::unique_ptr<IStereoMatcher>
|
||||||
StereoMatcherFactory::create(StereoAlgorithmType type, int num_disparities) {
|
StereoMatcherFactory::create(StereoAlgorithmType type, int num_disparities,
|
||||||
|
CpuStereoMatcher::Params cpu_params) {
|
||||||
if (num_disparities <= 0 || num_disparities % 16 != 0) {
|
if (num_disparities <= 0 || num_disparities % 16 != 0) {
|
||||||
throw std::invalid_argument(
|
throw std::invalid_argument(
|
||||||
"StereoMatcherFactory: num_disparities must be a positive "
|
"StereoMatcherFactory: num_disparities must be a positive "
|
||||||
@ -17,14 +18,16 @@ StereoMatcherFactory::create(StereoAlgorithmType type, int num_disparities) {
|
|||||||
}
|
}
|
||||||
switch (type) {
|
switch (type) {
|
||||||
case StereoAlgorithmType::CPU:
|
case StereoAlgorithmType::CPU:
|
||||||
return std::make_unique<CpuStereoMatcher>(0, num_disparities);
|
return std::make_unique<CpuStereoMatcher>(0, num_disparities,
|
||||||
|
cpu_params);
|
||||||
case StereoAlgorithmType::GPU:
|
case StereoAlgorithmType::GPU:
|
||||||
try {
|
try {
|
||||||
return std::make_unique<GpuStereoMatcher>();
|
return std::make_unique<GpuStereoMatcher>(0, num_disparities);
|
||||||
} catch (const std::exception &e) {
|
} catch (const std::exception &e) {
|
||||||
LOG(WARNING) << "GPU stereo matcher unavailable: " << e.what()
|
LOG(WARNING) << "GPU stereo matcher unavailable: " << e.what()
|
||||||
<< ". Falling back to CPU.";
|
<< ". Falling back to CPU.";
|
||||||
return std::make_unique<CpuStereoMatcher>(0, num_disparities);
|
return std::make_unique<CpuStereoMatcher>(0, num_disparities,
|
||||||
|
cpu_params);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return nullptr;
|
return nullptr;
|
||||||
|
|||||||
@ -25,8 +25,15 @@ int main(int argc, char *argv[]) {
|
|||||||
|
|
||||||
try {
|
try {
|
||||||
auto config = score::ConfigLoader::load(config_path);
|
auto config = score::ConfigLoader::load(config_path);
|
||||||
|
score::CliStereoOptions stereo;
|
||||||
|
stereo.use_gpu = config.cloud_point.algorithm == "gpu";
|
||||||
|
stereo.num_disparities = config.cloud_point.num_disparities;
|
||||||
|
stereo.min_depth_m = static_cast<float>(config.cloud_point.min_depth_m);
|
||||||
|
stereo.max_depth_m = static_cast<float>(config.cloud_point.max_depth_m);
|
||||||
|
stereo.ply_stride = config.cloud_point.ply_stride;
|
||||||
|
stereo.wls_filter = config.cloud_point.wls_filter;
|
||||||
return score::run_cli(std::cin, std::cout, config.server.ip,
|
return score::run_cli(std::cin, std::cout, config.server.ip,
|
||||||
config.server.port);
|
config.server.port, stereo);
|
||||||
} catch (const std::exception &e) {
|
} catch (const std::exception &e) {
|
||||||
std::cerr << "Failed to start CLI: " << e.what() << std::endl;
|
std::cerr << "Failed to start CLI: " << e.what() << std::endl;
|
||||||
return 1;
|
return 1;
|
||||||
|
|||||||
@ -3,6 +3,7 @@ test_sources = files(
|
|||||||
'test_integration.cpp',
|
'test_integration.cpp',
|
||||||
'test_tcp.cpp',
|
'test_tcp.cpp',
|
||||||
'test_cli.cpp',
|
'test_cli.cpp',
|
||||||
|
'test_config.cpp',
|
||||||
'test_c_api.cpp',
|
'test_c_api.cpp',
|
||||||
'test_base64.cpp',
|
'test_base64.cpp',
|
||||||
'test_serialize_image.cpp'
|
'test_serialize_image.cpp'
|
||||||
@ -20,6 +21,7 @@ if opencv_dep.found()
|
|||||||
'test_point_cloud_builder.cpp',
|
'test_point_cloud_builder.cpp',
|
||||||
'test_point_cloud_evaluator.cpp',
|
'test_point_cloud_evaluator.cpp',
|
||||||
'test_cloud_point_client.cpp',
|
'test_cloud_point_client.cpp',
|
||||||
|
'test_ply_export.cpp',
|
||||||
'test_scared_dataset.cpp',
|
'test_scared_dataset.cpp',
|
||||||
'test_scared_ground_truth_loader.cpp'
|
'test_scared_ground_truth_loader.cpp'
|
||||||
)
|
)
|
||||||
|
|||||||
76
tests/test_config.cpp
Normal file
76
tests/test_config.cpp
Normal file
@ -0,0 +1,76 @@
|
|||||||
|
#include <cstdio>
|
||||||
|
#include <fstream>
|
||||||
|
#include <gtest/gtest.h>
|
||||||
|
#include <string>
|
||||||
|
#include <tuple>
|
||||||
|
|
||||||
|
#include "cloud_point_rpc/config.hpp"
|
||||||
|
|
||||||
|
namespace {
|
||||||
|
|
||||||
|
struct TempConfig {
|
||||||
|
std::string path;
|
||||||
|
explicit TempConfig(const std::string &yaml)
|
||||||
|
: path("test_config_" +
|
||||||
|
std::to_string(reinterpret_cast<uintptr_t>(this)) + ".yaml") {
|
||||||
|
std::ofstream(path) << yaml;
|
||||||
|
}
|
||||||
|
~TempConfig() { std::remove(path.c_str()); }
|
||||||
|
};
|
||||||
|
|
||||||
|
} // namespace
|
||||||
|
|
||||||
|
TEST(ConfigLoaderTest, CloudPointSectionDefaultsWhenAbsent) {
|
||||||
|
TempConfig cfg("server:\n ip: \"127.0.0.1\"\n port: 8080\n");
|
||||||
|
const auto c = score::ConfigLoader::load(cfg.path);
|
||||||
|
EXPECT_EQ(c.cloud_point.algorithm, "gpu");
|
||||||
|
EXPECT_EQ(c.cloud_point.num_disparities, 128);
|
||||||
|
EXPECT_DOUBLE_EQ(c.cloud_point.min_depth_m, 0.01);
|
||||||
|
EXPECT_DOUBLE_EQ(c.cloud_point.max_depth_m, 10.0);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(ConfigLoaderTest, CloudPointSectionIsParsed) {
|
||||||
|
TempConfig cfg("server:\n ip: \"127.0.0.1\"\n port: 8080\n"
|
||||||
|
"cloud_point:\n algorithm: cpu\n num_disparities: 160\n"
|
||||||
|
" min_depth_m: 0.02\n max_depth_m: 0.3\n");
|
||||||
|
const auto c = score::ConfigLoader::load(cfg.path);
|
||||||
|
EXPECT_EQ(c.cloud_point.algorithm, "cpu");
|
||||||
|
EXPECT_EQ(c.cloud_point.num_disparities, 160);
|
||||||
|
EXPECT_DOUBLE_EQ(c.cloud_point.min_depth_m, 0.02);
|
||||||
|
EXPECT_DOUBLE_EQ(c.cloud_point.max_depth_m, 0.3);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(ConfigLoaderTest, CloudPointExportAndFilterKeys) {
|
||||||
|
TempConfig cfg("server:\n ip: \"127.0.0.1\"\n port: 8080\n"
|
||||||
|
"cloud_point:\n ply_stride: 4\n wls_filter: true\n");
|
||||||
|
const auto c = score::ConfigLoader::load(cfg.path);
|
||||||
|
EXPECT_EQ(c.cloud_point.ply_stride, 4);
|
||||||
|
EXPECT_TRUE(c.cloud_point.wls_filter);
|
||||||
|
TempConfig bad("server:\n ip: \"127.0.0.1\"\n port: 8080\n"
|
||||||
|
"cloud_point:\n ply_stride: 0\n");
|
||||||
|
EXPECT_THROW(std::ignore = score::ConfigLoader::load(bad.path),
|
||||||
|
std::runtime_error);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(ConfigLoaderTest, CloudPointPartialSectionKeepsDefaults) {
|
||||||
|
TempConfig cfg("server:\n ip: \"127.0.0.1\"\n port: 8080\n"
|
||||||
|
"cloud_point:\n max_depth_m: 0.5\n");
|
||||||
|
const auto c = score::ConfigLoader::load(cfg.path);
|
||||||
|
EXPECT_EQ(c.cloud_point.algorithm, "gpu");
|
||||||
|
EXPECT_EQ(c.cloud_point.num_disparities, 128);
|
||||||
|
EXPECT_DOUBLE_EQ(c.cloud_point.max_depth_m, 0.5);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(ConfigLoaderTest, CloudPointRejectsBadAlgorithm) {
|
||||||
|
TempConfig cfg("server:\n ip: \"127.0.0.1\"\n port: 8080\n"
|
||||||
|
"cloud_point:\n algorithm: fpga\n");
|
||||||
|
EXPECT_THROW(std::ignore = score::ConfigLoader::load(cfg.path),
|
||||||
|
std::runtime_error);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(ConfigLoaderTest, CloudPointRejectsInvertedDepthRange) {
|
||||||
|
TempConfig cfg("server:\n ip: \"127.0.0.1\"\n port: 8080\n"
|
||||||
|
"cloud_point:\n min_depth_m: 1.0\n max_depth_m: 0.5\n");
|
||||||
|
EXPECT_THROW(std::ignore = score::ConfigLoader::load(cfg.path),
|
||||||
|
std::runtime_error);
|
||||||
|
}
|
||||||
157
tests/test_ply_export.cpp
Normal file
157
tests/test_ply_export.cpp
Normal file
@ -0,0 +1,157 @@
|
|||||||
|
#include <cmath>
|
||||||
|
#include <cstdio>
|
||||||
|
#include <cstring>
|
||||||
|
#include <fstream>
|
||||||
|
#include <gtest/gtest.h>
|
||||||
|
#include <limits>
|
||||||
|
#include <sstream>
|
||||||
|
#include <string>
|
||||||
|
#include <tuple>
|
||||||
|
|
||||||
|
#include "cloud_point/cloud_point_client.hpp"
|
||||||
|
|
||||||
|
using namespace score;
|
||||||
|
|
||||||
|
namespace {
|
||||||
|
|
||||||
|
/// 3x3 organised cloud on a plane z = 1 m with 1 cm pixel spacing.
|
||||||
|
PointCloud make_grid_cloud() {
|
||||||
|
PointCloud cloud;
|
||||||
|
cloud.width = 3;
|
||||||
|
cloud.height = 3;
|
||||||
|
cloud.data.resize(27);
|
||||||
|
for (int r = 0; r < 3; ++r) {
|
||||||
|
for (int c = 0; c < 3; ++c) {
|
||||||
|
const size_t i = static_cast<size_t>(r * 3 + c) * 3;
|
||||||
|
cloud.data[i] = 0.01f * static_cast<float>(c);
|
||||||
|
cloud.data[i + 1] = 0.01f * static_cast<float>(r);
|
||||||
|
cloud.data[i + 2] = 1.0f;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return cloud;
|
||||||
|
}
|
||||||
|
|
||||||
|
void set_nan(PointCloud &cloud, int r, int c) {
|
||||||
|
const size_t i = static_cast<size_t>(r * cloud.width + c) * 3;
|
||||||
|
const float nan = std::numeric_limits<float>::quiet_NaN();
|
||||||
|
cloud.data[i] = cloud.data[i + 1] = cloud.data[i + 2] = nan;
|
||||||
|
}
|
||||||
|
|
||||||
|
struct TempFile {
|
||||||
|
std::string path{"test_ply_export.ply"};
|
||||||
|
~TempFile() { std::remove(path.c_str()); }
|
||||||
|
std::string read() const {
|
||||||
|
std::ifstream in(path);
|
||||||
|
std::stringstream ss;
|
||||||
|
ss << in.rdbuf();
|
||||||
|
return ss.str();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
} // namespace
|
||||||
|
|
||||||
|
TEST(PlyExportTest, PointsOnlyHasNoFaceElement) {
|
||||||
|
TempFile file;
|
||||||
|
const auto faces =
|
||||||
|
write_ply(make_grid_cloud(), file.path, PlyOptions{false, 0.05f});
|
||||||
|
EXPECT_EQ(faces, 0u);
|
||||||
|
const auto text = file.read();
|
||||||
|
EXPECT_NE(text.find("element vertex 9\n"), std::string::npos);
|
||||||
|
EXPECT_EQ(text.find("element face"), std::string::npos);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(PlyExportTest, FullGridProducesTwoTrianglesPerCell) {
|
||||||
|
TempFile file;
|
||||||
|
const auto faces =
|
||||||
|
write_ply(make_grid_cloud(), file.path, PlyOptions{true, 0.05f, false});
|
||||||
|
EXPECT_EQ(faces, 8u); // 2x2 cells * 2 triangles
|
||||||
|
EXPECT_NE(file.read().find("format ascii 1.0\n"), std::string::npos);
|
||||||
|
const auto text = file.read();
|
||||||
|
EXPECT_NE(text.find("element face 8\n"), std::string::npos);
|
||||||
|
EXPECT_NE(text.find("property list uchar int vertex_indices\n"),
|
||||||
|
std::string::npos);
|
||||||
|
// First cell, first triangle: (0,0) -> (1,0) -> (1,1) = indices 0,3,4
|
||||||
|
EXPECT_NE(text.find("\n3 0 3 4\n"), std::string::npos);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(PlyExportTest, MissingCornerKeepsSingleTriangle) {
|
||||||
|
auto cloud = make_grid_cloud();
|
||||||
|
set_nan(cloud, 0, 0); // top-left cell has 3 valid corners
|
||||||
|
TempFile file;
|
||||||
|
const auto faces =
|
||||||
|
write_ply(cloud, file.path, PlyOptions{true, 0.05f, false});
|
||||||
|
EXPECT_EQ(faces, 7u);
|
||||||
|
const auto text = file.read();
|
||||||
|
EXPECT_NE(text.find("element vertex 8\n"), std::string::npos);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(PlyExportTest, DepthJumpBreaksMesh) {
|
||||||
|
auto cloud = make_grid_cloud();
|
||||||
|
// Push the centre column 20 cm away: every triangle touching it now has
|
||||||
|
// an edge longer than 5% of its mean depth and must be dropped.
|
||||||
|
for (int r = 0; r < 3; ++r) {
|
||||||
|
cloud.data[static_cast<size_t>(r * 3 + 1) * 3 + 2] = 1.2f;
|
||||||
|
}
|
||||||
|
TempFile file;
|
||||||
|
EXPECT_EQ(write_ply(cloud, file.path), 0u);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(PlyExportTest, StrideAveragesBlocks) {
|
||||||
|
// 3x3 grid with stride 2 -> 2x2 blocks: one full 2x2 block, two 2x1 /
|
||||||
|
// 1x2 edge blocks and the single corner pixel -> 4 vertices, 2 faces.
|
||||||
|
TempFile file;
|
||||||
|
const auto faces = write_ply(make_grid_cloud(), file.path,
|
||||||
|
PlyOptions{true, 0.05f, false, 2});
|
||||||
|
EXPECT_EQ(faces, 2u);
|
||||||
|
const auto text = file.read();
|
||||||
|
EXPECT_NE(text.find("element vertex 4\n"), std::string::npos);
|
||||||
|
// First vertex is the mean of pixels (0,0),(0,1),(1,0),(1,1).
|
||||||
|
EXPECT_NE(text.find("\n0.005 0.005 1\n"), std::string::npos);
|
||||||
|
// Last vertex is the lone corner pixel (2,2).
|
||||||
|
EXPECT_NE(text.find("\n0.02 0.02 1\n"), std::string::npos);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(PlyExportTest, StrideDropsMostlyInvalidBlocks) {
|
||||||
|
auto cloud = make_grid_cloud();
|
||||||
|
// Invalidate 3 of the 4 pixels of the top-left 2x2 block.
|
||||||
|
set_nan(cloud, 0, 0);
|
||||||
|
set_nan(cloud, 0, 1);
|
||||||
|
set_nan(cloud, 1, 0);
|
||||||
|
TempFile file;
|
||||||
|
write_ply(cloud, file.path, PlyOptions{true, 0.05f, false, 2});
|
||||||
|
EXPECT_NE(file.read().find("element vertex 3\n"), std::string::npos);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(PlyExportTest, InvalidStrideThrows) {
|
||||||
|
TempFile file;
|
||||||
|
EXPECT_THROW(std::ignore = write_ply(make_grid_cloud(), file.path,
|
||||||
|
PlyOptions{true, 0.05f, false, 0}),
|
||||||
|
std::invalid_argument);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(PlyExportTest, BinaryLayoutMatchesHeader) {
|
||||||
|
TempFile file;
|
||||||
|
const auto faces = write_ply(make_grid_cloud(), file.path); // binary
|
||||||
|
ASSERT_EQ(faces, 8u);
|
||||||
|
const auto text = file.read();
|
||||||
|
EXPECT_NE(text.find("format binary_little_endian 1.0\n"),
|
||||||
|
std::string::npos);
|
||||||
|
const auto header_end = text.find("end_header\n") + 11;
|
||||||
|
ASSERT_NE(header_end, std::string::npos + 11);
|
||||||
|
// 9 vertices * 12 bytes + 8 faces * (1 + 12) bytes
|
||||||
|
EXPECT_EQ(text.size() - header_end, 9u * 12u + 8u * 13u);
|
||||||
|
// First vertex is (0, 0, 1)
|
||||||
|
float v[3];
|
||||||
|
std::memcpy(v, text.data() + header_end, sizeof(v));
|
||||||
|
EXPECT_FLOAT_EQ(v[0], 0.0f);
|
||||||
|
EXPECT_FLOAT_EQ(v[1], 0.0f);
|
||||||
|
EXPECT_FLOAT_EQ(v[2], 1.0f);
|
||||||
|
// First face: count byte 3 then indices 0,3,4
|
||||||
|
const char *f = text.data() + header_end + 9 * 12;
|
||||||
|
EXPECT_EQ(static_cast<unsigned char>(f[0]), 3u);
|
||||||
|
int idx[3];
|
||||||
|
std::memcpy(idx, f + 1, sizeof(idx));
|
||||||
|
EXPECT_EQ(idx[0], 0);
|
||||||
|
EXPECT_EQ(idx[1], 3);
|
||||||
|
EXPECT_EQ(idx[2], 4);
|
||||||
|
}
|
||||||
@ -1,5 +1,7 @@
|
|||||||
|
#include <cmath>
|
||||||
#include <gtest/gtest.h>
|
#include <gtest/gtest.h>
|
||||||
#include <opencv2/core.hpp>
|
#include <opencv2/core.hpp>
|
||||||
|
#include <opencv2/imgproc.hpp>
|
||||||
#include <tuple>
|
#include <tuple>
|
||||||
|
|
||||||
#include "cloud_point/cpu_stereo_matcher.hpp"
|
#include "cloud_point/cpu_stereo_matcher.hpp"
|
||||||
@ -47,9 +49,89 @@ TEST(StereoMatcherTest, FactoryCpuCreatesNonNull) {
|
|||||||
EXPECT_FALSE(disparity.empty());
|
EXPECT_FALSE(disparity.empty());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
TEST(StereoMatcherTest, CpuMatcherRejectsEvenBlockSize) {
|
||||||
|
CpuStereoMatcher::Params params;
|
||||||
|
params.block_size = 4;
|
||||||
|
EXPECT_THROW(CpuStereoMatcher(0, 64, params), std::invalid_argument);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(StereoMatcherTest, CpuMatcherRejectsBadMedianKernel) {
|
||||||
|
CpuStereoMatcher::Params params;
|
||||||
|
params.median_kernel = 4;
|
||||||
|
EXPECT_THROW(CpuStereoMatcher(0, 64, params), std::invalid_argument);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(StereoMatcherTest, WlsKeepsSgbmHolesInvalid) {
|
||||||
|
if (!CpuStereoMatcher::wls_available())
|
||||||
|
GTEST_SKIP() << "OpenCV built without ximgproc";
|
||||||
|
// Left 64 px of the right image have no counterpart -> SGBM leaves the
|
||||||
|
// left border invalid; WLS must not fill it with extrapolated values.
|
||||||
|
cv::Mat left(120, 200, CV_8UC1);
|
||||||
|
cv::randu(left, 0, 255);
|
||||||
|
cv::blur(left, left, cv::Size(3, 3));
|
||||||
|
cv::Mat right = cv::Mat::zeros(left.size(), CV_8UC1);
|
||||||
|
left(cv::Rect(8, 0, left.cols - 8, left.rows))
|
||||||
|
.copyTo(right(cv::Rect(0, 0, left.cols - 8, left.rows)));
|
||||||
|
|
||||||
|
CpuStereoMatcher::Params raw;
|
||||||
|
raw.wls_filter = false;
|
||||||
|
CpuStereoMatcher::Params wls;
|
||||||
|
wls.wls_filter = true;
|
||||||
|
cv::Mat d_raw = CpuStereoMatcher(0, 64, raw).compute(left, right);
|
||||||
|
cv::Mat d_wls = CpuStereoMatcher(0, 64, wls).compute(left, right);
|
||||||
|
ASSERT_EQ(d_wls.type(), CV_16S);
|
||||||
|
ASSERT_EQ(d_wls.size(), d_raw.size());
|
||||||
|
const int raw_invalid = cv::countNonZero(d_raw <= 0);
|
||||||
|
EXPECT_GT(raw_invalid, 0);
|
||||||
|
// No SGBM hole may be filled with an extrapolated disparity.
|
||||||
|
const int filled = cv::countNonZero((d_raw <= 0) & (d_wls > 0));
|
||||||
|
EXPECT_EQ(filled, 0);
|
||||||
|
// The filter must still produce a valid map elsewhere.
|
||||||
|
EXPECT_GT(cv::countNonZero(d_wls > 0), 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(StereoMatcherTest, CpuMatcherRecoversKnownShift) {
|
||||||
|
// Textured synthetic pair: right image is the left shifted by 8 px, so
|
||||||
|
// the (16x fixed-point) disparity in the interior should be ~8 px.
|
||||||
|
constexpr int kShift = 8;
|
||||||
|
cv::Mat left(120, 200, CV_8UC1);
|
||||||
|
cv::randu(left, 0, 255);
|
||||||
|
cv::blur(left, left, cv::Size(3, 3));
|
||||||
|
cv::Mat right = cv::Mat::zeros(left.size(), CV_8UC1);
|
||||||
|
left(cv::Rect(kShift, 0, left.cols - kShift, left.rows))
|
||||||
|
.copyTo(right(cv::Rect(0, 0, left.cols - kShift, left.rows)));
|
||||||
|
|
||||||
|
CpuStereoMatcher matcher(0, 64);
|
||||||
|
cv::Mat disparity = matcher.compute(left, right);
|
||||||
|
ASSERT_EQ(disparity.type(), CV_16S);
|
||||||
|
|
||||||
|
int good = 0, total = 0;
|
||||||
|
for (int y = 10; y < left.rows - 10; ++y) {
|
||||||
|
for (int x = 70; x < left.cols - 20; ++x) {
|
||||||
|
const float d = disparity.at<short>(y, x) / 16.0f;
|
||||||
|
if (d <= 0)
|
||||||
|
continue;
|
||||||
|
++total;
|
||||||
|
if (std::abs(d - kShift) <= 1.0f)
|
||||||
|
++good;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ASSERT_GT(total, 0);
|
||||||
|
EXPECT_GT(static_cast<double>(good) / total, 0.9);
|
||||||
|
}
|
||||||
|
|
||||||
|
TEST(StereoMatcherTest, GpuSupportedNumDisparitiesRoundsUp) {
|
||||||
|
EXPECT_EQ(GpuStereoMatcher::supported_num_disparities(16), 64);
|
||||||
|
EXPECT_EQ(GpuStereoMatcher::supported_num_disparities(64), 64);
|
||||||
|
EXPECT_EQ(GpuStereoMatcher::supported_num_disparities(96), 128);
|
||||||
|
EXPECT_EQ(GpuStereoMatcher::supported_num_disparities(160), 256);
|
||||||
|
EXPECT_THROW(std::ignore = GpuStereoMatcher::supported_num_disparities(272),
|
||||||
|
std::invalid_argument);
|
||||||
|
}
|
||||||
|
|
||||||
TEST(StereoMatcherTest, FactoryRejectsInvalidNumDisparities) {
|
TEST(StereoMatcherTest, FactoryRejectsInvalidNumDisparities) {
|
||||||
EXPECT_THROW(std::ignore = StereoMatcherFactory::create(
|
EXPECT_THROW(std::ignore =
|
||||||
StereoAlgorithmType::CPU, 0),
|
StereoMatcherFactory::create(StereoAlgorithmType::CPU, 0),
|
||||||
std::invalid_argument);
|
std::invalid_argument);
|
||||||
EXPECT_THROW(std::ignore = StereoMatcherFactory::create(
|
EXPECT_THROW(std::ignore = StereoMatcherFactory::create(
|
||||||
StereoAlgorithmType::CPU, -16),
|
StereoAlgorithmType::CPU, -16),
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user