brushline.vegetation.validation
The held-out validation dataset, spatial holdout and lag rules, group metrics, interval coverage, candidate benchmark, and validation report.
Held-out validation and candidate benchmark for the low-vegetation model.
The validator compares observed and predicted height, cover, and density on a spatial holdout. It reports error by functional group, interval coverage, time-lag sensitivity, out-of-domain and coverage exclusions, and a candidate benchmark with recorded disagreement. It does not run a live sibling model.
- class brushline.vegetation.validation.CandidatePrediction(*, name: Annotated[str, MinLen(min_length=1)], height_m: Annotated[float, Ge(ge=0)], cover_pct: Annotated[float, Ge(ge=0), Le(le=100)], density_per_ha: Annotated[float | None, Ge(ge=0)] = None)[source]
Bases:
BaseModelOne candidate model’s prediction for an observation.
- Parameters:
name (str) – Candidate name.
height_m (float) – Predicted height.
cover_pct (float) – Predicted cover.
density_per_ha (float or None) – Predicted density.
- model_config = {'extra': 'forbid', 'frozen': True}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class brushline.vegetation.validation.CandidateSummary(*, name: str, n: Annotated[int, Ge(ge=0)], height_rmse: Annotated[float, Ge(ge=0)], cover_rmse: Annotated[float, Ge(ge=0)], height_bias: float, cover_bias: float, height_rmse_difference: float, cover_rmse_difference: float)[source]
Bases:
BaseModelError summary for one candidate model.
- Parameters:
name (str) – Candidate name.
n (int) – Record count.
height_rmse (float) – Height RMSE.
cover_rmse (float) – Cover RMSE.
height_bias (float) – Height bias.
cover_bias (float) – Cover bias.
height_rmse_difference (float) – Candidate height RMSE minus Brushline height RMSE.
cover_rmse_difference (float) – Candidate cover RMSE minus Brushline cover RMSE.
- model_config = {'extra': 'forbid', 'frozen': True}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class brushline.vegetation.validation.LagSummary(*, lag_years: Annotated[int, Ge(ge=0)], n: Annotated[int, Ge(ge=0)], height_rmse: Annotated[float, Ge(ge=0)], cover_rmse: Annotated[float, Ge(ge=0)], height_bias: float, cover_bias: float)[source]
Bases:
BaseModelError summary at one prediction lag.
- Parameters:
lag_years (int) – Absolute prediction lag.
n (int) – Record count.
height_rmse (float) – Height RMSE.
cover_rmse (float) – Cover RMSE.
height_bias (float) – Height bias.
cover_bias (float) – Cover bias.
- model_config = {'extra': 'forbid', 'frozen': True}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class brushline.vegetation.validation.MetricSummary(*, group: str, variable: str, n: Annotated[int, Ge(ge=0)], observed_mean: float, predicted_mean: float, rmse: Annotated[float, Ge(ge=0)], mae: Annotated[float, Ge(ge=0)], bias: float, interval_coverage: Annotated[float | None, Ge(ge=0), Le(le=1)] = None)[source]
Bases:
BaseModelError summary for one group and variable.
- Parameters:
group (str) – Functional group.
variable (str) – Variable name.
n (int) – Record count.
observed_mean (float) – Observed mean.
predicted_mean (float) – Predicted mean.
rmse (float) – Root mean squared error.
mae (float) – Mean absolute error.
bias (float) – Mean predicted minus observed.
interval_coverage (float or None) – Share inside the predicted interval.
- model_config = {'extra': 'forbid', 'frozen': True}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class brushline.vegetation.validation.ValidationConfig(*, holdout_blocks: Annotated[list[str], MinLen(min_length=1)], min_coverage_pct: Annotated[float, Ge(ge=0), Le(le=100)] = 50.0, max_lag_years: Annotated[int, Ge(ge=0)] = 5, interval_z: Annotated[float, Gt(gt=0)] = 1.96, include_out_of_domain: bool = False, candidate_disagreement_tolerance: Annotated[float, Ge(ge=0)] = 0.25)[source]
Bases:
BaseModelHoldout, lag, coverage, and comparison rules.
- Parameters:
holdout_blocks (list of str) – Spatial blocks held out.
min_coverage_pct (float, optional) – Minimum observation coverage. Defaults to
50.max_lag_years (int, optional) – Maximum absolute prediction lag. Defaults to
5.interval_z (float, optional) – Interval multiplier. Defaults to
1.96.include_out_of_domain (bool, optional) – Include out-of-domain records in metrics. Defaults to
False.candidate_disagreement_tolerance (float, optional) – RMSE difference that raises a question. Defaults to
0.25.
- model_config = {'extra': 'forbid', 'frozen': True}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class brushline.vegetation.validation.ValidationDataset(*, schema_version: Literal['brushline.vegetation-validation.v1'] = 'brushline.vegetation-validation.v1', records: Annotated[list[ValidationRecord], MinLen(min_length=1)])[source]
Bases:
BaseModelA deterministic validation dataset.
- Parameters:
schema_version ({"brushline.vegetation-validation.v1"}) – Dataset schema version.
records (list of ValidationRecord) – Validation records.
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- class brushline.vegetation.validation.ValidationRecord(*, observation_id: ~typing.Annotated[str, ~annotated_types.MinLen(min_length=1)], segment_id: ~typing.Annotated[str, ~annotated_types.MinLen(min_length=1)], spatial_block: ~typing.Annotated[str, ~annotated_types.MinLen(min_length=1)], observed_date: ~datetime.date, prediction_year: ~typing.Annotated[int, ~annotated_types.Ge(ge=1900), ~annotated_types.Le(le=2200)], functional_group: ~typing.Literal['unknown', 'shrub', 'deciduous_sprout', 'conifer', 'grass_herb'], observed_height_m: ~typing.Annotated[float, ~annotated_types.Ge(ge=0)], predicted_height_m: ~typing.Annotated[float, ~annotated_types.Ge(ge=0)], predicted_height_sd: ~typing.Annotated[float, ~annotated_types.Ge(ge=0)], observed_cover_pct: ~typing.Annotated[float, ~annotated_types.Ge(ge=0), ~annotated_types.Le(le=100)], predicted_cover_pct: ~typing.Annotated[float, ~annotated_types.Ge(ge=0), ~annotated_types.Le(le=100)], predicted_cover_sd: ~typing.Annotated[float, ~annotated_types.Ge(ge=0)], observed_density_per_ha: ~typing.Annotated[float | None, ~annotated_types.Ge(ge=0)] = None, predicted_density_per_ha: ~typing.Annotated[float | None, ~annotated_types.Ge(ge=0)] = None, predicted_density_sd: ~typing.Annotated[float, ~annotated_types.Ge(ge=0)] = 0.0, coverage_pct: ~typing.Annotated[float, ~annotated_types.Ge(ge=0), ~annotated_types.Le(le=100)], in_domain: bool = True, domain_reason: str = '', treatment_history: list[str] = <factory>, source: ~typing.Annotated[str, ~annotated_types.MinLen(min_length=1)], source_version: ~typing.Annotated[str, ~annotated_types.MinLen(min_length=1)], candidates: list[~brushline.vegetation.validation.CandidatePrediction] = <factory>)[source]
Bases:
BaseModelOne observed/predicted validation record.
- Parameters:
observation_id (str) – Stable record id.
segment_id (str) – Segment id.
spatial_block (str) – Spatial holdout block.
observed_date (datetime.date) – Observation date.
prediction_year (int) – Year of the prediction.
functional_group (str) – Functional group.
observed_height_m (float) – Observed height.
predicted_height_m (float) – Predicted height.
predicted_height_sd (float) – Predicted height standard deviation.
observed_cover_pct (float) – Observed cover.
predicted_cover_pct (float) – Predicted cover.
predicted_cover_sd (float) – Predicted cover standard deviation.
observed_density_per_ha (float or None) – Observed density.
predicted_density_per_ha (float or None) – Predicted density.
predicted_density_sd (float, optional) – Predicted density standard deviation.
coverage_pct (float) – Observation coverage.
in_domain (bool, optional) – Whether the observation is in domain. Defaults to
True.domain_reason (str, optional) – Reason when out of domain.
treatment_history (list of str, optional) – Treatment history.
source (str) – Source.
source_version (str) – Source version.
candidates (list of CandidatePrediction, optional) – Candidate predictions.
- model_config = {'extra': 'forbid', 'frozen': True}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- property lag_years: int
Return the prediction-to-observation lag.
- Returns:
int – Observation year minus prediction year.
- exception brushline.vegetation.validation.VegetationValidationError[source]
Bases:
ValueErrorRaised when a validation dataset is internally inconsistent.
- class brushline.vegetation.validation.VegetationValidationReport(*, schema_version: Literal['brushline.vegetation-validation.v1'] = 'brushline.vegetation-validation.v1', created_utc: datetime, config: ValidationConfig, metrics: list[MetricSummary] = <factory>, lags: list[LagSummary] = <factory>, candidates: list[CandidateSummary] = <factory>, excluded: dict[str, int]=<factory>, errors: list[str] = <factory>, warnings: list[str] = <factory>, open_questions: list[str] = <factory>, counts: dict[str, int | float]=<factory>)[source]
Bases:
BaseModelThe held-out validation and candidate benchmark report.
- Parameters:
schema_version ({"brushline.vegetation-validation.v1"}) – Report schema version.
created_utc (datetime) – Timezone-aware creation time.
config (ValidationConfig) – Validation rules.
metrics (list of MetricSummary) – Brushline error by group and variable.
lags (list of LagSummary) – Time-lag sensitivity.
candidates (list of CandidateSummary) – Candidate benchmark.
excluded (dict of str to int) – Excluded record counts.
errors (list of str) – Named validation failures.
warnings (list of str) – Non-fatal findings.
open_questions (list of str) – Reviewer questions.
counts (dict of str to int or float) – Summary counts.
- model_config = {'extra': 'forbid'}
Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].
- property is_valid: bool
Return whether the validation passed.
- Returns:
bool –
Truewhen there are no errors.
- brushline.vegetation.validation.validate_vegetation(dataset: ValidationDataset, config: ValidationConfig, *, created_utc: datetime | None = None) VegetationValidationReport[source]
Validate held-out observations and benchmark candidate predictions.
- Parameters:
dataset (ValidationDataset) – Observed and predicted records.
config (ValidationConfig) – Holdout, lag, coverage, and candidate rules.
created_utc (datetime, optional) – Report time.
- Returns:
VegetationValidationReport – Errors, lags, candidates, and reviewer questions.
- Raises:
VegetationValidationError – When observation ids or spatial blocks are duplicated.
- brushline.vegetation.validation.validate_vegetation_validation(report: VegetationValidationReport) ContractValidation[source]
Validate a vegetation validation report.
- Parameters:
report (VegetationValidationReport) – Report to validate.
- Returns:
ContractValidation – Errors, warnings, and counts.