femic.hydrology.validation Module
The femic.hydrology.validation module owns P114.9 validation and
closeout. It records published directional benchmarks, runs bounded public-gauge
frequency backcasts, audits full-pilot readiness, and emits the validation
report and model card for the WS4/FHOPS handoff.
Use this page when you are validating the response layer, running the bounded public-gauge pilot, or checking the model-card conditions.
Start Here If…
Use this page first if you are trying to:
run GEV/LP3 backcasts on public gauge annual maxima;
check tail plausibility and split-sample stability;
review published paired-watershed expectations;
run the bounded P114.9 pilot;
write or inspect the model card; or
understand the WS4/FHOPS handoff conditions.
Typical Usage
from pathlib import Path
from femic.hydrology.validation import build_p1149_closeout
result = build_p1149_closeout(
gauges_path=Path("tests/fixtures/hydrology/pilot/gauges.json"),
annual_maxima_path=Path(
"tests/fixtures/hydrology/pilot/annual_maxima.csv"
),
output_dir=Path("runtime/hydrology/p114_9_closeout"),
)
print(result.validation_report.validation_id)
print(result.model_card.review_status)
Key Entry Surfaces
load_pilot_gauges()load_pilot_annual_maxima()Load the bounded public pilot fixture.build_pilot_sample()Build a validated annual-maximum sample.run_gauge_backcast()Fit GEV/LP3, run the split-sample check, and apply the tail screen.run_public_gauge_pilot()Run all pilot gauges and create a deterministic pilot run.published_benchmarks()Return the reviewed directional benchmark catalogue.assess_return_period_ratios()Compare observed ratios with a benchmark expectation.build_benchmark_reviews()Mark benchmark cases passed, failed, or not evaluated.audit_pilot_readiness()Record full-pilot inputs and data gaps.build_validation_report()build_model_card()Build the linked closeout records.build_p1149_closeout()Run the bounded pilot and write all closeout files.PilotGaugeBackcastTailCheckValidationReportModelCardEvidence, audit, and handoff records.
Cross-References
P114.9 validation, public-gauge pilot, model card, and close-out.
This module does not add a new hydrologic model. It records published benchmark expectations, runs bounded public-gauge frequency backcasts, audits full-pilot readiness, and emits the validation report and model card that accompany a FEMIC response bundle.
- class femic.hydrology.validation.BenchmarkAssessment(case_id, is_consistent, rows, warnings=())[source]
Bases:
objectDirectional/order-of-magnitude assessment of one paired response.
- Parameters:
case_id (str)
is_consistent (bool)
rows (tuple[BenchmarkRow, ...])
warnings (tuple[str, ...])
- case_id: str
- is_consistent: bool
- rows: tuple[BenchmarkRow, ...]
- to_dict()[source]
Return the assessment as a JSON-serializable mapping.
- Return type:
dict[str, object]
- warnings: tuple[str, ...] = ()
- class femic.hydrology.validation.BenchmarkCaseReview(case, assessment, status, reason)[source]
Bases:
objectReview status for one published benchmark case.
- Parameters:
case (PublishedBenchmark)
assessment (BenchmarkAssessment | None)
status (str)
reason (str)
- assessment: BenchmarkAssessment | None
- case: PublishedBenchmark
- reason: str
- status: str
- class femic.hydrology.validation.BenchmarkRow(return_period, observed_ratio, direction_ok, scale_ok)[source]
Bases:
objectOne return-period comparison against a published expectation.
- Parameters:
return_period (float)
observed_ratio (float)
direction_ok (bool)
scale_ok (bool)
- direction_ok: bool
- observed_ratio: float
- return_period: float
- scale_ok: bool
- class femic.hydrology.validation.CloseoutResult(output_dir, pilot_run, validation_report, model_card, pilot_path, validation_path, model_card_path)[source]
Bases:
objectPaths and records produced by the P114.9 closeout run.
- Parameters:
output_dir (Path)
pilot_run (PilotRun)
validation_report (ValidationReport)
model_card (ModelCard)
pilot_path (Path)
validation_path (Path)
model_card_path (Path)
- model_card_path: Path
- output_dir: Path
- pilot_path: Path
- to_dict()[source]
Return the closeout result as a JSON-serializable mapping.
- Return type:
dict[str, object]
- validation_path: Path
- validation_report: ValidationReport
- exception femic.hydrology.validation.HydrologyValidationError[source]
Bases:
RuntimeErrorRaised when pilot or validation evidence is invalid.
- class femic.hydrology.validation.ModelCard(model_card_id, model_id, model_version, title, intended_use, out_of_scope, regimes, validation_report_id, pilot_run_id, validation_summary, uncertainty, limitations, review_status, review_checklist, consumer_handoff, sources, generated_utc, caveats=(), schema_version='femic.hydrology.model_card.v1')[source]
Bases:
objectVersioned model card for the P114 response layer.
- Parameters:
model_card_id (str)
model_id (str)
model_version (str)
title (str)
intended_use (tuple[str, ...])
out_of_scope (tuple[str, ...])
regimes (tuple[str, ...])
validation_report_id (str)
pilot_run_id (str)
validation_summary (dict[str, object])
uncertainty (tuple[str, ...])
limitations (tuple[str, ...])
review_status (str)
review_checklist (tuple[str, ...])
consumer_handoff (tuple[str, ...])
sources (tuple[str, ...])
generated_utc (str)
caveats (tuple[str, ...])
schema_version (str)
- caveats: tuple[str, ...] = ()
- consumer_handoff: tuple[str, ...]
- generated_utc: str
- intended_use: tuple[str, ...]
- limitations: tuple[str, ...]
- model_card_id: str
- model_id: str
- model_version: str
- out_of_scope: tuple[str, ...]
- pilot_run_id: str
- regimes: tuple[str, ...]
- review_checklist: tuple[str, ...]
- review_status: str
- schema_version: str = 'femic.hydrology.model_card.v1'
- sources: tuple[str, ...]
- title: str
- to_dict()[source]
Return the model card as a JSON-serializable mapping.
- Return type:
dict[str, object]
- uncertainty: tuple[str, ...]
- validation_report_id: str
- validation_summary: dict[str, object]
- class femic.hydrology.validation.PilotAnnualMaximum(station_number, year, max_date, max_value_m3s, symbol='')[source]
Bases:
objectOne public annual maximum daily discharge value.
- Parameters:
station_number (str)
year (int)
max_date (str)
max_value_m3s (float)
symbol (str)
- max_date: str
- max_value_m3s: float
- station_number: str
- symbol: str = ''
- to_dict()[source]
Return the annual maximum as a JSON-serializable mapping.
- Return type:
dict[str, object]
- year: int
- class femic.hydrology.validation.PilotGauge(station_number, station_name, province, regime, drainage_area_km2, status, contributor, vertical_datum, source_url, station_url, licence, retrieved_utc, notes='')[source]
Bases:
objectPublic gauge metadata for the bounded P114.9 pilot.
- Parameters:
station_number (str)
station_name (str)
province (str)
regime (str)
drainage_area_km2 (float)
status (str)
contributor (str)
vertical_datum (str)
source_url (str)
station_url (str)
licence (str)
retrieved_utc (str)
notes (str)
- contributor: str
- drainage_area_km2: float
- licence: str
- notes: str = ''
- province: str
- regime: str
- retrieved_utc: str
- source_url: str
- station_name: str
- station_number: str
- station_url: str
- status: str
- to_dict()[source]
Return the gauge metadata as a JSON-serializable mapping.
- Return type:
dict[str, object]
- vertical_datum: str
- class femic.hydrology.validation.PilotGaugeBackcast(gauge, generated_utc, record_start_year, record_end_year, year_count, missing_years, estimated_years, flagged_years, fits, tail_checks, usable_fits, split_sample, warnings=(), caveats=())[source]
Bases:
objectPublic-gauge frequency backcast and data-quality record.
- Parameters:
gauge (PilotGauge)
generated_utc (str)
record_start_year (int)
record_end_year (int)
year_count (int)
missing_years (tuple[int, ...])
estimated_years (tuple[int, ...])
flagged_years (tuple[int, ...])
fits (dict[str, FloodFrequencyFit])
tail_checks (tuple[TailCheck, ...])
usable_fits (tuple[str, ...])
split_sample (SplitSampleBackcast | None)
warnings (tuple[str, ...])
caveats (tuple[str, ...])
- caveats: tuple[str, ...] = ()
- estimated_years: tuple[int, ...]
- fits: dict[str, FloodFrequencyFit]
- flagged_years: tuple[int, ...]
- gauge: PilotGauge
- generated_utc: str
- missing_years: tuple[int, ...]
- record_end_year: int
- record_start_year: int
- split_sample: SplitSampleBackcast | None
- to_dict()[source]
Return the gauge backcast as a JSON-serializable mapping.
- Return type:
dict[str, object]
- usable_fits: tuple[str, ...]
- warnings: tuple[str, ...] = ()
- year_count: int
- class femic.hydrology.validation.PilotReadiness(domain_available, lidar_available, stream_available, road_available, disturbance_available, receptor_available, process_model_available, hydrologist_review_recorded, gaps=())[source]
Bases:
objectFull-pilot readiness and data-gap audit.
- Parameters:
domain_available (bool)
lidar_available (bool)
stream_available (bool)
road_available (bool)
disturbance_available (bool)
receptor_available (bool)
process_model_available (bool)
hydrologist_review_recorded (bool)
gaps (tuple[str, ...])
- disturbance_available: bool
- domain_available: bool
- gaps: tuple[str, ...] = ()
- hydrologist_review_recorded: bool
- lidar_available: bool
- process_model_available: bool
- property ready_for_full_process_pilot: bool
Return whether all full-pilot prerequisites are present.
- receptor_available: bool
- road_available: bool
- stream_available: bool
- class femic.hydrology.validation.PilotRun(run_id, generated_utc, gauges, backcasts, return_periods, bootstrap_samples, seed, caveats=(), provenance=<factory>, schema_version='femic.hydrology.pilot.v1')[source]
Bases:
objectSerialized bounded public-gauge pilot.
- Parameters:
run_id (str)
generated_utc (str)
gauges (tuple[PilotGauge, ...])
backcasts (tuple[PilotGaugeBackcast, ...])
return_periods (tuple[float, ...])
bootstrap_samples (int)
seed (int)
caveats (tuple[str, ...])
provenance (dict[str, object])
schema_version (str)
- backcasts: tuple[PilotGaugeBackcast, ...]
- bootstrap_samples: int
- caveats: tuple[str, ...] = ()
- gauges: tuple[PilotGauge, ...]
- generated_utc: str
- provenance: dict[str, object]
- return_periods: tuple[float, ...]
- run_id: str
- schema_version: str = 'femic.hydrology.pilot.v1'
- seed: int
- class femic.hydrology.validation.PublishedBenchmark(case_id, regime, hypothesis, expected_direction, sources, log10_tolerance=0.5, notes='')[source]
Bases:
objectOne directional/order-of-magnitude expectation from published work.
- Parameters:
case_id (str)
regime (str)
hypothesis (str)
expected_direction (str)
sources (tuple[str, ...])
log10_tolerance (float)
notes (str)
- case_id: str
- expected_direction: str
- hypothesis: str
- log10_tolerance: float = 0.5
- notes: str = ''
- regime: str
- sources: tuple[str, ...]
- class femic.hydrology.validation.SplitSampleBackcast(split_year, early_sample_size, late_sample_size, early_fit, late_fit, ratios, usable=True, reasons=())[source]
Bases:
objectEarly/late split-sample frequency stability check.
- Parameters:
split_year (int)
early_sample_size (int)
late_sample_size (int)
early_fit (FloodFrequencyFit)
late_fit (FloodFrequencyFit)
ratios (tuple[float, ...])
usable (bool)
reasons (tuple[str, ...])
- early_fit: FloodFrequencyFit
- early_sample_size: int
- late_fit: FloodFrequencyFit
- late_sample_size: int
- ratios: tuple[float, ...]
- reasons: tuple[str, ...] = ()
- split_year: int
- to_dict()[source]
Return the split-sample result as a JSON-serializable mapping.
- Return type:
dict[str, object]
- usable: bool = True
- class femic.hydrology.validation.TailCheck(method, shape, q100, observed_max, q100_to_observed_max, is_plausible, reasons=())[source]
Bases:
objectScreening plausibility check for one fitted tail.
- Parameters:
method (str)
shape (float | None)
q100 (float | None)
observed_max (float)
q100_to_observed_max (float | None)
is_plausible (bool)
reasons (tuple[str, ...])
- is_plausible: bool
- method: str
- observed_max: float
- q100: float | None
- q100_to_observed_max: float | None
- reasons: tuple[str, ...] = ()
- shape: float | None
- class femic.hydrology.validation.ValidationReport(validation_id, generated_utc, pilot_run_id, benchmark_evidence_version, benchmark_reviews, readiness, model_card_id, summary, deviations, uncertainty, caveats, consumer_handoff, provenance=<factory>, schema_version='femic.hydrology.validation.v1')[source]
Bases:
objectP114.9 validation report and consumer handoff.
- Parameters:
validation_id (str)
generated_utc (str)
pilot_run_id (str)
benchmark_evidence_version (str)
benchmark_reviews (tuple[BenchmarkCaseReview, ...])
readiness (PilotReadiness)
model_card_id (str)
summary (dict[str, object])
deviations (tuple[str, ...])
uncertainty (tuple[str, ...])
caveats (tuple[str, ...])
consumer_handoff (tuple[str, ...])
provenance (dict[str, object])
schema_version (str)
- benchmark_evidence_version: str
- benchmark_reviews: tuple[BenchmarkCaseReview, ...]
- caveats: tuple[str, ...]
- consumer_handoff: tuple[str, ...]
- deviations: tuple[str, ...]
- generated_utc: str
- model_card_id: str
- pilot_run_id: str
- provenance: dict[str, object]
- readiness: PilotReadiness
- schema_version: str = 'femic.hydrology.validation.v1'
- summary: dict[str, object]
- to_dict()[source]
Return the validation report as a JSON-serializable mapping.
- Return type:
dict[str, object]
- uncertainty: tuple[str, ...]
- validation_id: str
- femic.hydrology.validation.assess_return_period_ratios(benchmark, return_periods, ratios)[source]
Compare return-period ratios with one published expectation.
- Parameters:
benchmark (PublishedBenchmark)
return_periods (Sequence[float])
ratios (Sequence[float])
- Return type:
- femic.hydrology.validation.audit_pilot_readiness(*, domain_manifest_path=None, disturbance_ledger_path=None, process_model_available=False, hydrologist_review_recorded=False)[source]
Audit whether a full two-watershed process pilot can be run.
- Parameters:
domain_manifest_path (Path | None)
disturbance_ledger_path (Path | None)
process_model_available (bool)
hydrologist_review_recorded (bool)
- Return type:
- femic.hydrology.validation.build_benchmark_reviews(assessments=())[source]
Attach available assessments to the published benchmark catalogue.
- Parameters:
assessments (Sequence[BenchmarkAssessment])
- Return type:
tuple[BenchmarkCaseReview, …]
- femic.hydrology.validation.build_model_card(*, pilot_run, validation_report_id, readiness, generated_utc=None)[source]
Build the response-layer model card.
- Parameters:
pilot_run (PilotRun)
validation_report_id (str)
readiness (PilotReadiness)
generated_utc (str | None)
- Return type:
- femic.hydrology.validation.build_p1149_closeout(*, gauges_path, annual_maxima_path, output_dir, domain_manifest_path=None, disturbance_ledger_path=None, process_model_available=False, hydrologist_review_recorded=False, bootstrap_samples=200, seed=42, generated_utc=None)[source]
Run the bounded pilot and write the closeout records.
- Parameters:
gauges_path (Path)
annual_maxima_path (Path)
output_dir (Path)
domain_manifest_path (Path | None)
disturbance_ledger_path (Path | None)
process_model_available (bool)
hydrologist_review_recorded (bool)
bootstrap_samples (int)
seed (int)
generated_utc (str | None)
- Return type:
- femic.hydrology.validation.build_pilot_sample(gauge, rows)[source]
Build a validated flood sample for one pilot gauge.
- Parameters:
gauge (PilotGauge)
rows (Sequence[PilotAnnualMaximum])
- Return type:
- femic.hydrology.validation.build_validation_report(pilot_run, *, readiness, assessments=(), generated_utc=None)[source]
Build the deterministic validation report and linked model card.
- Parameters:
pilot_run (PilotRun)
readiness (PilotReadiness)
assessments (Sequence[BenchmarkAssessment])
generated_utc (str | None)
- Return type:
tuple[ValidationReport, ModelCard]
- femic.hydrology.validation.load_pilot_annual_maxima(path)[source]
Load annual maxima from the bounded pilot CSV.
- Parameters:
path (Path)
- Return type:
tuple[PilotAnnualMaximum, …]
- femic.hydrology.validation.load_pilot_gauges(path)[source]
Load the bounded pilot gauge metadata fixture.
- Parameters:
path (Path)
- Return type:
tuple[PilotGauge, …]
- femic.hydrology.validation.published_benchmarks()[source]
Return the reviewed directional benchmark catalogue.
These are deliberately directional and order-of-magnitude checks, not point estimates copied from any one watershed.
- Return type:
tuple[PublishedBenchmark, …]
- femic.hydrology.validation.run_gauge_backcast(gauge, rows, *, return_periods=(2.0, 10.0, 100.0), bootstrap_samples=200, seed=42, generated_utc=None)[source]
Fit GEV and LP3 to one public gauge and run a split-sample check.
- Parameters:
gauge (PilotGauge)
rows (Sequence[PilotAnnualMaximum])
return_periods (Sequence[float])
bootstrap_samples (int)
seed (int)
generated_utc (str | None)
- Return type:
- femic.hydrology.validation.run_public_gauge_pilot(gauges_path, annual_maxima_path, *, return_periods=(2.0, 10.0, 100.0), bootstrap_samples=200, seed=42, generated_utc=None)[source]
Run the bounded P114.9 public-gauge pilot.
- Parameters:
gauges_path (Path)
annual_maxima_path (Path)
return_periods (Sequence[float])
bootstrap_samples (int)
seed (int)
generated_utc (str | None)
- Return type:
- femic.hydrology.validation.write_model_card(model_card, path)[source]
Write a model card as JSON.
- Parameters:
model_card (ModelCard)
path (Path)
- Return type:
Path
- femic.hydrology.validation.write_pilot_run(pilot_run, path)[source]
Write a pilot run as JSON.
- Parameters:
pilot_run (PilotRun)
path (Path)
- Return type:
Path
- femic.hydrology.validation.write_validation_report(report, path)[source]
Write a validation report as JSON.
- Parameters:
report (ValidationReport)
path (Path)
- Return type:
Path