brushline.integrations.lidar_benchmark

The offline FEMIC/WS4/FHOPS LiDAR benchmark: request and value contracts, tolerance comparison, case statuses, and the report validator.

Offline benchmark against the FEMIC, WS4, and FHOPS LiDAR consumers.

The benchmark compares the same bundle pin, segment, CRS, resolution, datum, statistic, buffer, and valid-pixel rule. It records absolute and relative differences for every metric and consumer, fails on an unexplained missing value or a difference outside the tolerance, and never calls a sibling runtime.

class brushline.integrations.lidar_benchmark.BenchmarkCase(*, metric: Annotated[str, MinLen(min_length=1)], consumer: Annotated[str, MinLen(min_length=1)], brushline_value: float, comparison_value: float | None = None, absolute_difference: float | None = None, relative_difference: float | None = None, absolute_tolerance: float, relative_tolerance: float, status: str, reason: str = '')[source]

Bases: BaseModel

One comparison result.

Parameters:
  • metric (str) – Metric name.

  • consumer (str) – Consumer name.

  • brushline_value (float) – Brushline value.

  • comparison_value (float or None) – Sibling value.

  • absolute_difference (float or None) – Absolute difference.

  • relative_difference (float or None) – Relative difference.

  • absolute_tolerance (float) – Absolute tolerance.

  • relative_tolerance (float) – Relative tolerance.

  • status (str) – pass, fail, missing, or contract.

  • reason (str) – Result reason.

model_config = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class brushline.integrations.lidar_benchmark.BenchmarkContract(*, crs: Annotated[str, MinLen(min_length=1)], resolution_m: Annotated[float, Gt(gt=0)], vertical_datum: Annotated[str, MinLen(min_length=1)], statistic: Annotated[str, MinLen(min_length=1)], buffer_m: Annotated[float, Ge(ge=0)], valid_pixel_rule: Annotated[str, MinLen(min_length=1)])[source]

Bases: BaseModel

The shared measurement contract for one benchmark.

Parameters:
  • crs (str) – Coordinate reference system.

  • resolution_m (float) – Native resolution in metres.

  • vertical_datum (str) – Vertical datum.

  • statistic (str) – Aggregation statistic.

  • buffer_m (float) – Segment buffer.

  • valid_pixel_rule (str) – Valid-pixel rule.

model_config = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class brushline.integrations.lidar_benchmark.BenchmarkMetric(*, name: Annotated[str, MinLen(min_length=1)], unit: Annotated[str, MinLen(min_length=1)], brushline_value: float, comparisons: Annotated[list[BenchmarkValue], MinLen(min_length=1)], absolute_tolerance: Annotated[float, Ge(ge=0)], relative_tolerance: Annotated[float, Ge(ge=0), Le(le=1)])[source]

Bases: BaseModel

One metric and its Brushline and sibling-consumer values.

Parameters:
  • name (str) – Metric name.

  • unit (str) – Metric unit.

  • brushline_value (float) – Brushline value.

  • comparisons (list of BenchmarkValue) – Sibling-consumer values.

  • absolute_tolerance (float) – Absolute tolerance in the metric unit.

  • relative_tolerance (float) – Relative tolerance as a fraction.

model_config = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class brushline.integrations.lidar_benchmark.BenchmarkRequest(*, schema_version: Literal['brushline.lidar-benchmark-request.v1'] = 'brushline.lidar-benchmark-request.v1', benchmark_id: Annotated[str, MinLen(min_length=1)], bundle_id: Annotated[str, MinLen(min_length=1)], bundle_version: Annotated[str, MinLen(min_length=1)], segment_id: Annotated[str, MinLen(min_length=1)], product_sha256: Annotated[dict[str, str], MinLen(min_length=1)], contract: BenchmarkContract, metrics: Annotated[list[BenchmarkMetric], MinLen(min_length=1)])[source]

Bases: BaseModel

A deterministic benchmark request.

Parameters:
  • schema_version ({"brushline.lidar-benchmark-request.v1"}) – Request schema version.

  • benchmark_id (str) – Benchmark id.

  • bundle_id (str) – Pinned LiDAR bundle id.

  • bundle_version (str) – Pinned bundle version.

  • segment_id (str) – Segment id.

  • product_sha256 (dict of str to str) – Product ids and their SHA-256 checksums.

  • contract (BenchmarkContract) – Shared measurement contract.

  • metrics (list of BenchmarkMetric) – Metric comparisons.

model_config = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class brushline.integrations.lidar_benchmark.BenchmarkValue(*, consumer: Annotated[str, MinLen(min_length=1)], value: float | None = None, provenance: Annotated[str, MinLen(min_length=1)], contract: BenchmarkContract, missing_reason: str = '', required: bool = True)[source]

Bases: BaseModel

One consumer’s value for one metric.

Parameters:
  • consumer (str) – Consumer name, such as femic, ws4, or fhops.

  • value (float or None) – Metric value, or None when unavailable.

  • provenance (str) – Evidence reference.

  • contract (BenchmarkContract) – Measurement contract declared by the consumer.

  • missing_reason (str, optional) – Why the value is missing.

  • required (bool, optional) – Whether a missing value fails the benchmark. Defaults to True.

model_config = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class brushline.integrations.lidar_benchmark.LidarBenchmarkReport(*, schema_version: ~typing.Literal['brushline.lidar-benchmark.v1'] = 'brushline.lidar-benchmark.v1', created_utc: ~datetime.datetime, request: ~brushline.integrations.lidar_benchmark.BenchmarkRequest, cases: list[~brushline.integrations.lidar_benchmark.BenchmarkCase] = <factory>, errors: list[str] = <factory>, warnings: list[str] = <factory>, counts: dict[str, int | float] = <factory>, bundle_sha256: ~typing.Annotated[str, _PydanticGeneralMetadata(pattern='^[0-9a-f]{64}$')])[source]

Bases: BaseModel

The deterministic benchmark report.

Parameters:
  • schema_version ({"brushline.lidar-benchmark.v1"}) – Report schema version.

  • created_utc (datetime) – Timezone-aware creation time.

  • request (BenchmarkRequest) – Benchmark request.

  • cases (list of BenchmarkCase) – Comparison cases.

  • errors (list of str) – Named failures.

  • warnings (list of str) – Non-fatal findings.

  • counts (dict of str to int or float) – Case counts.

  • bundle_sha256 (str) – Hash of the pinned product checksums.

model_config = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

property is_valid: bool

Return whether the benchmark passed.

Returns:

bool – True when there are no errors.

to_dict() → dict[str, object][source]

Return a JSON-ready report.

Returns:

dict of str to object – Validity, cases, errors, warnings, and counts.

brushline.integrations.lidar_benchmark.run_lidar_benchmark(request: BenchmarkRequest, *, created_utc: datetime | None = None) → LidarBenchmarkReport[source]

Run the offline sibling-consumer benchmark.

Parameters:
  • request (BenchmarkRequest) – Benchmark request.

  • created_utc (datetime, optional) – Report time. Defaults to the current UTC time.

Returns:

LidarBenchmarkReport – Cases, errors, warnings, and counts.

brushline.integrations.lidar_benchmark.validate_lidar_benchmark(report: LidarBenchmarkReport) → ContractValidation[source]

Validate the benchmark report and its case/status consistency.

Parameters:

report (LidarBenchmarkReport) – Report to validate.

Returns:

ContractValidation – Errors, warnings, and counts.