femic.hydrology.validation Module

The femic.hydrology.validation module owns P114.9 validation and closeout. It records published directional benchmarks, runs bounded public-gauge frequency backcasts, audits full-pilot readiness, and emits the validation report and model card for the WS4/FHOPS handoff.

Use this page when you are validating the response layer, running the bounded public-gauge pilot, or checking the model-card conditions.

Start Here If…

Use this page first if you are trying to:

  • run GEV/LP3 backcasts on public gauge annual maxima;

  • check tail plausibility and split-sample stability;

  • review published paired-watershed expectations;

  • run the bounded P114.9 pilot;

  • write or inspect the model card; or

  • understand the WS4/FHOPS handoff conditions.

Typical Usage

from pathlib import Path
from femic.hydrology.validation import build_p1149_closeout

result = build_p1149_closeout(
    gauges_path=Path("tests/fixtures/hydrology/pilot/gauges.json"),
    annual_maxima_path=Path(
        "tests/fixtures/hydrology/pilot/annual_maxima.csv"
    ),
    output_dir=Path("runtime/hydrology/p114_9_closeout"),
)
print(result.validation_report.validation_id)
print(result.model_card.review_status)

Key Entry Surfaces

  • load_pilot_gauges()

  • load_pilot_annual_maxima() Load the bounded public pilot fixture.

  • build_pilot_sample() Build a validated annual-maximum sample.

  • run_gauge_backcast() Fit GEV/LP3, run the split-sample check, and apply the tail screen.

  • run_public_gauge_pilot() Run all pilot gauges and create a deterministic pilot run.

  • published_benchmarks() Return the reviewed directional benchmark catalogue.

  • assess_return_period_ratios() Compare observed ratios with a benchmark expectation.

  • build_benchmark_reviews() Mark benchmark cases passed, failed, or not evaluated.

  • audit_pilot_readiness() Record full-pilot inputs and data gaps.

  • build_validation_report()

  • build_model_card() Build the linked closeout records.

  • build_p1149_closeout() Run the bounded pilot and write all closeout files.

  • PilotGaugeBackcast

  • TailCheck

  • ValidationReport

  • ModelCard Evidence, audit, and handoff records.

Cross-References

P114.9 validation, public-gauge pilot, model card, and close-out.

This module does not add a new hydrologic model. It records published benchmark expectations, runs bounded public-gauge frequency backcasts, audits full-pilot readiness, and emits the validation report and model card that accompany a FEMIC response bundle.

class femic.hydrology.validation.BenchmarkAssessment(case_id, is_consistent, rows, warnings=())[source]

Bases: object

Directional/order-of-magnitude assessment of one paired response.

Parameters:
  • case_id (str)

  • is_consistent (bool)

  • rows (tuple[BenchmarkRow, ...])

  • warnings (tuple[str, ...])

case_id: str
is_consistent: bool
rows: tuple[BenchmarkRow, ...]
to_dict()[source]

Return the assessment as a JSON-serializable mapping.

Return type:

dict[str, object]

warnings: tuple[str, ...] = ()
class femic.hydrology.validation.BenchmarkCaseReview(case, assessment, status, reason)[source]

Bases: object

Review status for one published benchmark case.

Parameters:
assessment: BenchmarkAssessment | None
case: PublishedBenchmark
reason: str
status: str
to_dict()[source]

Return the benchmark review as a JSON-serializable mapping.

Return type:

dict[str, object]

class femic.hydrology.validation.BenchmarkRow(return_period, observed_ratio, direction_ok, scale_ok)[source]

Bases: object

One return-period comparison against a published expectation.

Parameters:
  • return_period (float)

  • observed_ratio (float)

  • direction_ok (bool)

  • scale_ok (bool)

direction_ok: bool
observed_ratio: float
return_period: float
scale_ok: bool
to_dict()[source]

Return the benchmark row as a JSON-serializable mapping.

Return type:

dict[str, object]

class femic.hydrology.validation.CloseoutResult(output_dir, pilot_run, validation_report, model_card, pilot_path, validation_path, model_card_path)[source]

Bases: object

Paths and records produced by the P114.9 closeout run.

Parameters:
model_card: ModelCard
model_card_path: Path
output_dir: Path
pilot_path: Path
pilot_run: PilotRun
to_dict()[source]

Return the closeout result as a JSON-serializable mapping.

Return type:

dict[str, object]

validation_path: Path
validation_report: ValidationReport
exception femic.hydrology.validation.HydrologyValidationError[source]

Bases: RuntimeError

Raised when pilot or validation evidence is invalid.

class femic.hydrology.validation.ModelCard(model_card_id, model_id, model_version, title, intended_use, out_of_scope, regimes, validation_report_id, pilot_run_id, validation_summary, uncertainty, limitations, review_status, review_checklist, consumer_handoff, sources, generated_utc, caveats=(), schema_version='femic.hydrology.model_card.v1')[source]

Bases: object

Versioned model card for the P114 response layer.

Parameters:
  • model_card_id (str)

  • model_id (str)

  • model_version (str)

  • title (str)

  • intended_use (tuple[str, ...])

  • out_of_scope (tuple[str, ...])

  • regimes (tuple[str, ...])

  • validation_report_id (str)

  • pilot_run_id (str)

  • validation_summary (dict[str, object])

  • uncertainty (tuple[str, ...])

  • limitations (tuple[str, ...])

  • review_status (str)

  • review_checklist (tuple[str, ...])

  • consumer_handoff (tuple[str, ...])

  • sources (tuple[str, ...])

  • generated_utc (str)

  • caveats (tuple[str, ...])

  • schema_version (str)

caveats: tuple[str, ...] = ()
consumer_handoff: tuple[str, ...]
generated_utc: str
intended_use: tuple[str, ...]
limitations: tuple[str, ...]
model_card_id: str
model_id: str
model_version: str
out_of_scope: tuple[str, ...]
pilot_run_id: str
regimes: tuple[str, ...]
review_checklist: tuple[str, ...]
review_status: str
schema_version: str = 'femic.hydrology.model_card.v1'
sources: tuple[str, ...]
title: str
to_dict()[source]

Return the model card as a JSON-serializable mapping.

Return type:

dict[str, object]

uncertainty: tuple[str, ...]
validation_report_id: str
validation_summary: dict[str, object]
class femic.hydrology.validation.PilotAnnualMaximum(station_number, year, max_date, max_value_m3s, symbol='')[source]

Bases: object

One public annual maximum daily discharge value.

Parameters:
  • station_number (str)

  • year (int)

  • max_date (str)

  • max_value_m3s (float)

  • symbol (str)

max_date: str
max_value_m3s: float
station_number: str
symbol: str = ''
to_dict()[source]

Return the annual maximum as a JSON-serializable mapping.

Return type:

dict[str, object]

year: int
class femic.hydrology.validation.PilotGauge(station_number, station_name, province, regime, drainage_area_km2, status, contributor, vertical_datum, source_url, station_url, licence, retrieved_utc, notes='')[source]

Bases: object

Public gauge metadata for the bounded P114.9 pilot.

Parameters:
  • station_number (str)

  • station_name (str)

  • province (str)

  • regime (str)

  • drainage_area_km2 (float)

  • status (str)

  • contributor (str)

  • vertical_datum (str)

  • source_url (str)

  • station_url (str)

  • licence (str)

  • retrieved_utc (str)

  • notes (str)

contributor: str
drainage_area_km2: float
licence: str
notes: str = ''
province: str
regime: str
retrieved_utc: str
source_url: str
station_name: str
station_number: str
station_url: str
status: str
to_dict()[source]

Return the gauge metadata as a JSON-serializable mapping.

Return type:

dict[str, object]

vertical_datum: str
class femic.hydrology.validation.PilotGaugeBackcast(gauge, generated_utc, record_start_year, record_end_year, year_count, missing_years, estimated_years, flagged_years, fits, tail_checks, usable_fits, split_sample, warnings=(), caveats=())[source]

Bases: object

Public-gauge frequency backcast and data-quality record.

Parameters:
  • gauge (PilotGauge)

  • generated_utc (str)

  • record_start_year (int)

  • record_end_year (int)

  • year_count (int)

  • missing_years (tuple[int, ...])

  • estimated_years (tuple[int, ...])

  • flagged_years (tuple[int, ...])

  • fits (dict[str, FloodFrequencyFit])

  • tail_checks (tuple[TailCheck, ...])

  • usable_fits (tuple[str, ...])

  • split_sample (SplitSampleBackcast | None)

  • warnings (tuple[str, ...])

  • caveats (tuple[str, ...])

caveats: tuple[str, ...] = ()
estimated_years: tuple[int, ...]
fits: dict[str, FloodFrequencyFit]
flagged_years: tuple[int, ...]
gauge: PilotGauge
generated_utc: str
missing_years: tuple[int, ...]
record_end_year: int
record_start_year: int
split_sample: SplitSampleBackcast | None
tail_checks: tuple[TailCheck, ...]
to_dict()[source]

Return the gauge backcast as a JSON-serializable mapping.

Return type:

dict[str, object]

usable_fits: tuple[str, ...]
warnings: tuple[str, ...] = ()
year_count: int
class femic.hydrology.validation.PilotReadiness(domain_available, lidar_available, stream_available, road_available, disturbance_available, receptor_available, process_model_available, hydrologist_review_recorded, gaps=())[source]

Bases: object

Full-pilot readiness and data-gap audit.

Parameters:
  • domain_available (bool)

  • lidar_available (bool)

  • stream_available (bool)

  • road_available (bool)

  • disturbance_available (bool)

  • receptor_available (bool)

  • process_model_available (bool)

  • hydrologist_review_recorded (bool)

  • gaps (tuple[str, ...])

disturbance_available: bool
domain_available: bool
gaps: tuple[str, ...] = ()
hydrologist_review_recorded: bool
lidar_available: bool
process_model_available: bool
property ready_for_full_process_pilot: bool

Return whether all full-pilot prerequisites are present.

receptor_available: bool
road_available: bool
stream_available: bool
to_dict()[source]

Return the readiness audit as a JSON-serializable mapping.

Return type:

dict[str, object]

class femic.hydrology.validation.PilotRun(run_id, generated_utc, gauges, backcasts, return_periods, bootstrap_samples, seed, caveats=(), provenance=<factory>, schema_version='femic.hydrology.pilot.v1')[source]

Bases: object

Serialized bounded public-gauge pilot.

Parameters:
  • run_id (str)

  • generated_utc (str)

  • gauges (tuple[PilotGauge, ...])

  • backcasts (tuple[PilotGaugeBackcast, ...])

  • return_periods (tuple[float, ...])

  • bootstrap_samples (int)

  • seed (int)

  • caveats (tuple[str, ...])

  • provenance (dict[str, object])

  • schema_version (str)

backcasts: tuple[PilotGaugeBackcast, ...]
bootstrap_samples: int
caveats: tuple[str, ...] = ()
gauges: tuple[PilotGauge, ...]
generated_utc: str
provenance: dict[str, object]
return_periods: tuple[float, ...]
run_id: str
schema_version: str = 'femic.hydrology.pilot.v1'
seed: int
to_dict()[source]

Return the pilot run as a JSON-serializable mapping.

Return type:

dict[str, object]

class femic.hydrology.validation.PublishedBenchmark(case_id, regime, hypothesis, expected_direction, sources, log10_tolerance=0.5, notes='')[source]

Bases: object

One directional/order-of-magnitude expectation from published work.

Parameters:
  • case_id (str)

  • regime (str)

  • hypothesis (str)

  • expected_direction (str)

  • sources (tuple[str, ...])

  • log10_tolerance (float)

  • notes (str)

case_id: str
expected_direction: str
hypothesis: str
log10_tolerance: float = 0.5
notes: str = ''
regime: str
sources: tuple[str, ...]
to_dict()[source]

Return the benchmark case as a JSON-serializable mapping.

Return type:

dict[str, object]

class femic.hydrology.validation.SplitSampleBackcast(split_year, early_sample_size, late_sample_size, early_fit, late_fit, ratios, usable=True, reasons=())[source]

Bases: object

Early/late split-sample frequency stability check.

Parameters:
  • split_year (int)

  • early_sample_size (int)

  • late_sample_size (int)

  • early_fit (FloodFrequencyFit)

  • late_fit (FloodFrequencyFit)

  • ratios (tuple[float, ...])

  • usable (bool)

  • reasons (tuple[str, ...])

early_fit: FloodFrequencyFit
early_sample_size: int
late_fit: FloodFrequencyFit
late_sample_size: int
ratios: tuple[float, ...]
reasons: tuple[str, ...] = ()
split_year: int
to_dict()[source]

Return the split-sample result as a JSON-serializable mapping.

Return type:

dict[str, object]

usable: bool = True
class femic.hydrology.validation.TailCheck(method, shape, q100, observed_max, q100_to_observed_max, is_plausible, reasons=())[source]

Bases: object

Screening plausibility check for one fitted tail.

Parameters:
  • method (str)

  • shape (float | None)

  • q100 (float | None)

  • observed_max (float)

  • q100_to_observed_max (float | None)

  • is_plausible (bool)

  • reasons (tuple[str, ...])

is_plausible: bool
method: str
observed_max: float
q100: float | None
q100_to_observed_max: float | None
reasons: tuple[str, ...] = ()
shape: float | None
to_dict()[source]

Return the tail check as a JSON-serializable mapping.

Return type:

dict[str, object]

class femic.hydrology.validation.ValidationReport(validation_id, generated_utc, pilot_run_id, benchmark_evidence_version, benchmark_reviews, readiness, model_card_id, summary, deviations, uncertainty, caveats, consumer_handoff, provenance=<factory>, schema_version='femic.hydrology.validation.v1')[source]

Bases: object

P114.9 validation report and consumer handoff.

Parameters:
  • validation_id (str)

  • generated_utc (str)

  • pilot_run_id (str)

  • benchmark_evidence_version (str)

  • benchmark_reviews (tuple[BenchmarkCaseReview, ...])

  • readiness (PilotReadiness)

  • model_card_id (str)

  • summary (dict[str, object])

  • deviations (tuple[str, ...])

  • uncertainty (tuple[str, ...])

  • caveats (tuple[str, ...])

  • consumer_handoff (tuple[str, ...])

  • provenance (dict[str, object])

  • schema_version (str)

benchmark_evidence_version: str
benchmark_reviews: tuple[BenchmarkCaseReview, ...]
caveats: tuple[str, ...]
consumer_handoff: tuple[str, ...]
deviations: tuple[str, ...]
generated_utc: str
model_card_id: str
pilot_run_id: str
provenance: dict[str, object]
readiness: PilotReadiness
schema_version: str = 'femic.hydrology.validation.v1'
summary: dict[str, object]
to_dict()[source]

Return the validation report as a JSON-serializable mapping.

Return type:

dict[str, object]

uncertainty: tuple[str, ...]
validation_id: str
femic.hydrology.validation.assess_return_period_ratios(benchmark, return_periods, ratios)[source]

Compare return-period ratios with one published expectation.

Parameters:
  • benchmark (PublishedBenchmark)

  • return_periods (Sequence[float])

  • ratios (Sequence[float])

Return type:

BenchmarkAssessment

femic.hydrology.validation.audit_pilot_readiness(*, domain_manifest_path=None, disturbance_ledger_path=None, process_model_available=False, hydrologist_review_recorded=False)[source]

Audit whether a full two-watershed process pilot can be run.

Parameters:
  • domain_manifest_path (Path | None)

  • disturbance_ledger_path (Path | None)

  • process_model_available (bool)

  • hydrologist_review_recorded (bool)

Return type:

PilotReadiness

femic.hydrology.validation.build_benchmark_reviews(assessments=())[source]

Attach available assessments to the published benchmark catalogue.

Parameters:

assessments (Sequence[BenchmarkAssessment])

Return type:

tuple[BenchmarkCaseReview, …]

femic.hydrology.validation.build_model_card(*, pilot_run, validation_report_id, readiness, generated_utc=None)[source]

Build the response-layer model card.

Parameters:
Return type:

ModelCard

femic.hydrology.validation.build_p1149_closeout(*, gauges_path, annual_maxima_path, output_dir, domain_manifest_path=None, disturbance_ledger_path=None, process_model_available=False, hydrologist_review_recorded=False, bootstrap_samples=200, seed=42, generated_utc=None)[source]

Run the bounded pilot and write the closeout records.

Parameters:
  • gauges_path (Path)

  • annual_maxima_path (Path)

  • output_dir (Path)

  • domain_manifest_path (Path | None)

  • disturbance_ledger_path (Path | None)

  • process_model_available (bool)

  • hydrologist_review_recorded (bool)

  • bootstrap_samples (int)

  • seed (int)

  • generated_utc (str | None)

Return type:

CloseoutResult

femic.hydrology.validation.build_pilot_sample(gauge, rows)[source]

Build a validated flood sample for one pilot gauge.

Parameters:
Return type:

FloodSample

femic.hydrology.validation.build_validation_report(pilot_run, *, readiness, assessments=(), generated_utc=None)[source]

Build the deterministic validation report and linked model card.

Parameters:
Return type:

tuple[ValidationReport, ModelCard]

femic.hydrology.validation.load_pilot_annual_maxima(path)[source]

Load annual maxima from the bounded pilot CSV.

Parameters:

path (Path)

Return type:

tuple[PilotAnnualMaximum, …]

femic.hydrology.validation.load_pilot_gauges(path)[source]

Load the bounded pilot gauge metadata fixture.

Parameters:

path (Path)

Return type:

tuple[PilotGauge, …]

femic.hydrology.validation.published_benchmarks()[source]

Return the reviewed directional benchmark catalogue.

These are deliberately directional and order-of-magnitude checks, not point estimates copied from any one watershed.

Return type:

tuple[PublishedBenchmark, …]

femic.hydrology.validation.run_gauge_backcast(gauge, rows, *, return_periods=(2.0, 10.0, 100.0), bootstrap_samples=200, seed=42, generated_utc=None)[source]

Fit GEV and LP3 to one public gauge and run a split-sample check.

Parameters:
  • gauge (PilotGauge)

  • rows (Sequence[PilotAnnualMaximum])

  • return_periods (Sequence[float])

  • bootstrap_samples (int)

  • seed (int)

  • generated_utc (str | None)

Return type:

PilotGaugeBackcast

femic.hydrology.validation.run_public_gauge_pilot(gauges_path, annual_maxima_path, *, return_periods=(2.0, 10.0, 100.0), bootstrap_samples=200, seed=42, generated_utc=None)[source]

Run the bounded P114.9 public-gauge pilot.

Parameters:
  • gauges_path (Path)

  • annual_maxima_path (Path)

  • return_periods (Sequence[float])

  • bootstrap_samples (int)

  • seed (int)

  • generated_utc (str | None)

Return type:

PilotRun

femic.hydrology.validation.write_model_card(model_card, path)[source]

Write a model card as JSON.

Parameters:
Return type:

Path

femic.hydrology.validation.write_pilot_run(pilot_run, path)[source]

Write a pilot run as JSON.

Parameters:
Return type:

Path

femic.hydrology.validation.write_validation_report(report, path)[source]

Write a validation report as JSON.

Parameters:
Return type:

Path