Scenario ensembles and risk targets
Purpose
A deterministic estate run evaluates one schedule against one world. A scenario ensemble evaluates the same decision against several worlds, each with its own random-tape seed and optional process overrides. Common random numbers hold because every decision is evaluated on the same scenario seeds, so the difference between two decisions is a decision effect rather than a different roll of the dice.
This page specifies the scenario file, the risk aggregations, the risk pseudo-accounts, the sample-average-approximation loop, and the out-of-sample report. The ensemble can wrap the fire process, a compiled estate model, or a future full scenario simulator.
Scenario file
A scenario file is YAML or JSON:
schema_version: ws4.scenario.v1
name: base
scenarios:
- scenario_id: s1
seed: 11
weight: 1.0
overrides:
ignition_rate_per_fragment_year: 0.02
description: warm dry
- scenario_id: s2
seed: 12
weight: 2.0
overrides:
ignition_rate_per_fragment_year: 0.01
scenario_id (or the alias id) and seed are required. weight is positive and
defaults to 1.0. overrides is a mapping the evaluator interprets; this layer only carries
it. description is optional.
Load with ws4.ecosystem.load_scenarios(). Split with ScenarioSet.split(train_fraction)
to obtain a training ensemble and a validation ensemble in the original order.
Common random numbers
The evaluator receives the scenario and must use scenario.seed for its random tape. The
same scenario set is passed to every candidate decision, so the draws are paired. A paired
comparison removes the scenario noise from the difference between decisions.
This is the mechanism behind the variance-reduction test: with the same seed for both decisions the paired difference is exactly the decision effect; with independent seeds it is not.
Risk aggregations
A target opts into a scenario statistic with aggregation:
Aggregation |
Statistic |
|---|---|
|
No scenario dimension. This is the default and the deterministic path. It is rejected when the account frame carries a scenario column. |
|
Weighted mean across scenarios. |
|
Smallest scenario value. |
|
Largest scenario value. |
|
Lower-tail CVaR at |
|
The explicit name for the lower-tail CVaR. |
|
Upper-tail CVaR at |
|
Weighted quantile at |
|
Weighted probability that a scenario meets the target’s original minimum and maximum. |
|
Range of the scenario values, |
Target fields:
aggregationOne of the names above. Default
single.risk_alphaThe tail or quantile level in (0, 1). Default 0.9.
risk_minimumAn optional bound for the aggregated statistic, overriding the target’s minimum.
risk_maximumAn optional bound for the aggregated statistic, overriding the target’s maximum.
For mean, min, max, cvar, cvar_lower, cvar_upper, and quantile,
the effective target uses the original minimum and maximum unless a risk bound overrides it.
The aggregated value is in the account’s units.
For probability, the statistic is a compliance probability in [0, 1]. The original
minimum and maximum define compliance; the effective target uses risk_minimum (default
0.9) and risk_maximum. For example, a minimum target of 100 with aggregation:
probability and risk_minimum: 0.9 requires at least 90% of scenarios to reach 100.
For regret, the statistic is the range of the scenario values. The effective target uses
risk_maximum (default 0.0) and risk_minimum. This is a robustness target: a smaller
range means the decision is less sensitive to the scenario.
Risk pseudo-accounts
ws4.ecosystem.risk_account_values() turns per-scenario account values into one row per
target at the target’s period end. The pseudo-account name is:
risk.<aggregation>.<target index>.<account>
The target index keeps two targets on the same account and aggregation from colliding. The
function also returns the effective targets: copies with the pseudo-account name, the risk
bounds, and aggregation="single". The existing weighted-goal objective then prices them
exactly as it prices any other target.
The per-scenario input frame has offset (or year), account, value, and
scenario columns. A target with aggregation="single" is rejected when the scenario
column is present; set aggregation="mean" or another statistic.
Flow targets are not risk-aggregated in this version. A flow penalty compares a whole profile,
not a period value, so it needs a different scenario treatment. Use aggregation: single for
a flow target and run it against one scenario, or express the profile as period targets.
Sample-average approximation
ws4.ecosystem.SampleAverageApproximation evaluates a decision over an ensemble:
from ws4.ecosystem import (
SampleAverageApproximation,
ScenarioSet,
fire_evaluator,
synthetic_grid,
)
from ws4.opt.objective import Target
_, state, edges = synthetic_grid(5, 5)
scenarios = ScenarioSet.from_seeds([11, 12, 13])
train, validation = scenarios.split(0.67)
evaluator = fire_evaluator(state, edges, 10)
targets = [
Target(
"fire.burned_area_ha",
period=10,
maximum=50.0,
max_weight=1.0,
aggregation="cvar_upper",
risk_alpha=0.9,
)
]
saa = SampleAverageApproximation(train, evaluator, targets)
best = saa.best([config_a, config_b])
report = saa.report(best.decision, validation)
report.write("runs/fire-risk")
The evaluator maps a decision and a scenario to an account-values frame. It must use the scenario seed for its tape. The fire evaluator copies the initial cohort state for every run, so a decision does not mutate the landscape.
best is a finite decision-set loop: it evaluates every candidate on the training ensemble
and returns the lowest objective. For a larger decision space, call evaluate from a search
loop instead. report evaluates the chosen decision on the training and validation
ensembles and returns an ws4.ecosystem.OutOfSampleReport.
Out-of-sample report
OutOfSampleReport carries the training and validation evaluations and the generalization
gap:
gap = validation objective - training objective
A positive gap means the decision looks worse out of sample. report.write(out_dir) writes:
train_values.csv— per-scenario training values;validation_values.csv— per-scenario validation values;train_aggregated.csv— risk pseudo-accounts;validation_aggregated.csv— validation risk pseudo-accounts;summary.json— objectives, scenario counts, target penalties, and the gap.
Limits
The scenario overrides are carried but not interpreted by this layer. The evaluator decides how to apply them.
bestevaluates a finite decision set. It is a sample-average-approximation loop, not a replacement for the estate optimizer’s neighbourhood search.Flow targets are not risk-aggregated yet.
A scenario ensemble increases evaluation cost linearly with the number of scenarios. Use common random numbers, a smaller training set, and a held-out validation set.
The risk statistic is only as meaningful as the scenario distribution. A precise CVaR over a poorly chosen ensemble is still a poorly chosen ensemble.