brushline.vegetation.validation

The held-out validation dataset, spatial holdout and lag rules, group metrics, interval coverage, candidate benchmark, and validation report.

Held-out validation and candidate benchmark for the low-vegetation model.

The validator compares observed and predicted height, cover, and density on a spatial holdout. It reports error by functional group, interval coverage, time-lag sensitivity, out-of-domain and coverage exclusions, and a candidate benchmark with recorded disagreement. It does not run a live sibling model.

class brushline.vegetation.validation.CandidatePrediction(*, name: Annotated[str, MinLen(min_length=1)], height_m: Annotated[float, Ge(ge=0)], cover_pct: Annotated[float, Ge(ge=0), Le(le=100)], density_per_ha: Annotated[float | None, Ge(ge=0)] = None)[source]

Bases: BaseModel

One candidate model’s prediction for an observation.

Parameters:
  • name (str) – Candidate name.

  • height_m (float) – Predicted height.

  • cover_pct (float) – Predicted cover.

  • density_per_ha (float or None) – Predicted density.

model_config = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class brushline.vegetation.validation.CandidateSummary(*, name: str, n: Annotated[int, Ge(ge=0)], height_rmse: Annotated[float, Ge(ge=0)], cover_rmse: Annotated[float, Ge(ge=0)], height_bias: float, cover_bias: float, height_rmse_difference: float, cover_rmse_difference: float)[source]

Bases: BaseModel

Error summary for one candidate model.

Parameters:
  • name (str) – Candidate name.

  • n (int) – Record count.

  • height_rmse (float) – Height RMSE.

  • cover_rmse (float) – Cover RMSE.

  • height_bias (float) – Height bias.

  • cover_bias (float) – Cover bias.

  • height_rmse_difference (float) – Candidate height RMSE minus Brushline height RMSE.

  • cover_rmse_difference (float) – Candidate cover RMSE minus Brushline cover RMSE.

model_config = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class brushline.vegetation.validation.LagSummary(*, lag_years: Annotated[int, Ge(ge=0)], n: Annotated[int, Ge(ge=0)], height_rmse: Annotated[float, Ge(ge=0)], cover_rmse: Annotated[float, Ge(ge=0)], height_bias: float, cover_bias: float)[source]

Bases: BaseModel

Error summary at one prediction lag.

Parameters:
  • lag_years (int) – Absolute prediction lag.

  • n (int) – Record count.

  • height_rmse (float) – Height RMSE.

  • cover_rmse (float) – Cover RMSE.

  • height_bias (float) – Height bias.

  • cover_bias (float) – Cover bias.

model_config = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class brushline.vegetation.validation.MetricSummary(*, group: str, variable: str, n: Annotated[int, Ge(ge=0)], observed_mean: float, predicted_mean: float, rmse: Annotated[float, Ge(ge=0)], mae: Annotated[float, Ge(ge=0)], bias: float, interval_coverage: Annotated[float | None, Ge(ge=0), Le(le=1)] = None)[source]

Bases: BaseModel

Error summary for one group and variable.

Parameters:
  • group (str) – Functional group.

  • variable (str) – Variable name.

  • n (int) – Record count.

  • observed_mean (float) – Observed mean.

  • predicted_mean (float) – Predicted mean.

  • rmse (float) – Root mean squared error.

  • mae (float) – Mean absolute error.

  • bias (float) – Mean predicted minus observed.

  • interval_coverage (float or None) – Share inside the predicted interval.

model_config = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class brushline.vegetation.validation.ValidationConfig(*, holdout_blocks: Annotated[list[str], MinLen(min_length=1)], min_coverage_pct: Annotated[float, Ge(ge=0), Le(le=100)] = 50.0, max_lag_years: Annotated[int, Ge(ge=0)] = 5, interval_z: Annotated[float, Gt(gt=0)] = 1.96, include_out_of_domain: bool = False, candidate_disagreement_tolerance: Annotated[float, Ge(ge=0)] = 0.25)[source]

Bases: BaseModel

Holdout, lag, coverage, and comparison rules.

Parameters:
  • holdout_blocks (list of str) – Spatial blocks held out.

  • min_coverage_pct (float, optional) – Minimum observation coverage. Defaults to 50.

  • max_lag_years (int, optional) – Maximum absolute prediction lag. Defaults to 5.

  • interval_z (float, optional) – Interval multiplier. Defaults to 1.96.

  • include_out_of_domain (bool, optional) – Include out-of-domain records in metrics. Defaults to False.

  • candidate_disagreement_tolerance (float, optional) – RMSE difference that raises a question. Defaults to 0.25.

model_config = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class brushline.vegetation.validation.ValidationDataset(*, schema_version: Literal['brushline.vegetation-validation.v1'] = 'brushline.vegetation-validation.v1', records: Annotated[list[ValidationRecord], MinLen(min_length=1)])[source]

Bases: BaseModel

A deterministic validation dataset.

Parameters:
  • schema_version ({"brushline.vegetation-validation.v1"}) – Dataset schema version.

  • records (list of ValidationRecord) – Validation records.

model_config = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class brushline.vegetation.validation.ValidationRecord(*, observation_id: ~typing.Annotated[str, ~annotated_types.MinLen(min_length=1)], segment_id: ~typing.Annotated[str, ~annotated_types.MinLen(min_length=1)], spatial_block: ~typing.Annotated[str, ~annotated_types.MinLen(min_length=1)], observed_date: ~datetime.date, prediction_year: ~typing.Annotated[int, ~annotated_types.Ge(ge=1900), ~annotated_types.Le(le=2200)], functional_group: ~typing.Literal['unknown', 'shrub', 'deciduous_sprout', 'conifer', 'grass_herb'], observed_height_m: ~typing.Annotated[float, ~annotated_types.Ge(ge=0)], predicted_height_m: ~typing.Annotated[float, ~annotated_types.Ge(ge=0)], predicted_height_sd: ~typing.Annotated[float, ~annotated_types.Ge(ge=0)], observed_cover_pct: ~typing.Annotated[float, ~annotated_types.Ge(ge=0), ~annotated_types.Le(le=100)], predicted_cover_pct: ~typing.Annotated[float, ~annotated_types.Ge(ge=0), ~annotated_types.Le(le=100)], predicted_cover_sd: ~typing.Annotated[float, ~annotated_types.Ge(ge=0)], observed_density_per_ha: ~typing.Annotated[float | None, ~annotated_types.Ge(ge=0)] = None, predicted_density_per_ha: ~typing.Annotated[float | None, ~annotated_types.Ge(ge=0)] = None, predicted_density_sd: ~typing.Annotated[float, ~annotated_types.Ge(ge=0)] = 0.0, coverage_pct: ~typing.Annotated[float, ~annotated_types.Ge(ge=0), ~annotated_types.Le(le=100)], in_domain: bool = True, domain_reason: str = '', treatment_history: list[str] = <factory>, source: ~typing.Annotated[str, ~annotated_types.MinLen(min_length=1)], source_version: ~typing.Annotated[str, ~annotated_types.MinLen(min_length=1)], candidates: list[~brushline.vegetation.validation.CandidatePrediction] = <factory>)[source]

Bases: BaseModel

One observed/predicted validation record.

Parameters:
  • observation_id (str) – Stable record id.

  • segment_id (str) – Segment id.

  • spatial_block (str) – Spatial holdout block.

  • observed_date (datetime.date) – Observation date.

  • prediction_year (int) – Year of the prediction.

  • functional_group (str) – Functional group.

  • observed_height_m (float) – Observed height.

  • predicted_height_m (float) – Predicted height.

  • predicted_height_sd (float) – Predicted height standard deviation.

  • observed_cover_pct (float) – Observed cover.

  • predicted_cover_pct (float) – Predicted cover.

  • predicted_cover_sd (float) – Predicted cover standard deviation.

  • observed_density_per_ha (float or None) – Observed density.

  • predicted_density_per_ha (float or None) – Predicted density.

  • predicted_density_sd (float, optional) – Predicted density standard deviation.

  • coverage_pct (float) – Observation coverage.

  • in_domain (bool, optional) – Whether the observation is in domain. Defaults to True.

  • domain_reason (str, optional) – Reason when out of domain.

  • treatment_history (list of str, optional) – Treatment history.

  • source (str) – Source.

  • source_version (str) – Source version.

  • candidates (list of CandidatePrediction, optional) – Candidate predictions.

model_config = {'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

property lag_years: int

Return the prediction-to-observation lag.

Returns:

int – Observation year minus prediction year.

exception brushline.vegetation.validation.VegetationValidationError[source]

Bases: ValueError

Raised when a validation dataset is internally inconsistent.

class brushline.vegetation.validation.VegetationValidationReport(*, schema_version: Literal['brushline.vegetation-validation.v1'] = 'brushline.vegetation-validation.v1', created_utc: datetime, config: ValidationConfig, metrics: list[MetricSummary] = <factory>, lags: list[LagSummary] = <factory>, candidates: list[CandidateSummary] = <factory>, excluded: dict[str, int]=<factory>, errors: list[str] = <factory>, warnings: list[str] = <factory>, open_questions: list[str] = <factory>, counts: dict[str, int | float]=<factory>)[source]

Bases: BaseModel

The held-out validation and candidate benchmark report.

Parameters:
  • schema_version ({"brushline.vegetation-validation.v1"}) – Report schema version.

  • created_utc (datetime) – Timezone-aware creation time.

  • config (ValidationConfig) – Validation rules.

  • metrics (list of MetricSummary) – Brushline error by group and variable.

  • lags (list of LagSummary) – Time-lag sensitivity.

  • candidates (list of CandidateSummary) – Candidate benchmark.

  • excluded (dict of str to int) – Excluded record counts.

  • errors (list of str) – Named validation failures.

  • warnings (list of str) – Non-fatal findings.

  • open_questions (list of str) – Reviewer questions.

  • counts (dict of str to int or float) – Summary counts.

model_config = {'extra': 'forbid'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

property is_valid: bool

Return whether the validation passed.

Returns:

bool – True when there are no errors.

to_dict() → dict[str, object][source]

Return a JSON-ready report summary.

Returns:

dict of str to object – Validity, metrics, lags, candidates, errors, warnings, and counts.

brushline.vegetation.validation.validate_vegetation(dataset: ValidationDataset, config: ValidationConfig, *, created_utc: datetime | None = None) → VegetationValidationReport[source]

Validate held-out observations and benchmark candidate predictions.

Parameters:
  • dataset (ValidationDataset) – Observed and predicted records.

  • config (ValidationConfig) – Holdout, lag, coverage, and candidate rules.

  • created_utc (datetime, optional) – Report time.

Returns:

VegetationValidationReport – Errors, lags, candidates, and reviewer questions.

Raises:

VegetationValidationError – When observation ids or spatial blocks are duplicated.

brushline.vegetation.validation.validate_vegetation_validation(report: VegetationValidationReport) → ContractValidation[source]

Validate a vegetation validation report.

Parameters:

report (VegetationValidationReport) – Report to validate.

Returns:

ContractValidation – Errors, warnings, and counts.