Scenario Ensembles
A sensitivity analysis over this model means running the full 100-year
coupled pipeline once per scenario, where a scenario is one combination
of values from a grid over named RHRunConfig fields. A scenario runs the
rolling-horizon loop by default; planning_mode: single_pass makes it the
one-solve baseline instead (see Paired RH vs Single-Pass Runs below). The
ensemble driver exists to run those scenarios in parallel. This page covers
why the driver exists, the grid syntax, the parallelism and failure models,
performance budgeting, and a worked subsidy flip-point sweep.
One-at-a-Time Sensitivity
The Cartesian ensemble is useful when interactions between parameters are the
question. For one-at-a-time analysis, use sensitivity-run:
fresh-salvage sensitivity-run examples/sensitivity_tsa29.yaml --json
Each configured parameter is run in its own sweep while all other fields stay
at their baseline values. The driver adds a baseline value when it is absent
from a sweep, then writes reports/<id>-results.md: a readable record of
the effective baseline settings, every scenario and its status, the outcome at
each tested value, and the signed change from that parameter’s baseline. The
underlying evidence remains in data/<id>-observations.jsonl and
data/<id>-comparisons.csv; full provenance is written to the sensitivity
manifest.
Why Ensembles
The coupled model has three families of assumptions a policy analysis must sweep rather than fix:
Principal policy. subsidy_rate_per_m3 is the instrument itself —
the analysis question is how salvage volume and program cost respond to
the subsidy level.
Agent profit function. The flat economic fields (green_prices,
burned_price_discount, green_harvest_cost, burned_harvest_cost,
green_transport_cost_per_m3, burned_transport_cost_per_m3,
green_stumpage_rate, burned_stumpage_rate) plus discount_rate
set the agent’s margins; sweeping them stress-tests the calibrated
semi-synthetic surface (see Model Semantics).
Future fire pattern. burn_rate_multiplier scales every MFRI-derived
annual burn rate: 1.0 is the published MFRI table, 0.0 a fire-free
counterfactual control, 2.0 a doubled-fire future.
Every RHRunConfig field except the reserved ones is a valid axis, so the
same mechanism also covers structural sensitivity (horizon, steps,
decay_rate, …).
Grid Syntax
ensemble_id: tsa29-ensemble-smoke
base: # shared RHRunConfig field values
stands_path: outputs/full_tsa/data/tsa29-full-stands.parquet
yields_path: /path/to/femic-tsa29/woodstock_yields.csv
bridge_path: /path/to/femic-tsa29/ws3_bridge
base_year: 2025
horizon: 15
period_length: 10
steps: 10
workers: 8 # per-scenario WS3 threads
axes: # the cartesian grid
subsidy_rate_per_m3: [0.0, 3.0]
burn_rate_multiplier: [1.0, 2.0]
max_workers: 4
output_root: outputs/ensemble_tsa29_smoke
The grid is the cartesian product: one scenario per combination of axis values. Rules enforced at expansion time, before any work starts:
Every
baseandaxeskey must be anRHRunConfigfield name — there is no positional ambiguity and no silent ignore (ensemble_axis_unknown).run_idandoutput_rootare driver-owned: the ensemble assigns them per scenario and rejects them inbase/axes(ensemble_field_reserved).bridge_pathis required inbaseand reserved as an axis: every scenario is bound to the once-prebuilt shared bridge, so abridge_pathaxis would be silently discarded.Every axis needs at least one value (
ensemble_axis_empty); duplicate scenario names fail (ensemble_duplicate_scenario); each expanded scenario must form a validRHRunConfig(ensemble_scenario_invalid).
Scenario names are deterministic: axis-value fragments joined by __
in sorted axis order, normalized for file names (for example
burn_rate_multiplier-1.0__subsidy_rate_per_m3-0.0). An empty axes
mapping yields the single baseline scenario. Records and artifacts are
written in grid order regardless of completion order, so identical inputs
give byte-comparable scenario tables.
Paired RH vs Single-Pass Runs
planning_mode is a flat RHRunConfig field, so it is a valid ensemble
axis and one grid can hold both engines. The scenario worker dispatches
through rh.run_planning, which routes rolling_horizon to
rh.run_rh and single_pass to single_pass.run_single_pass; each
scenario record carries its planning_mode override and each run manifest
its planning_mode, so the arms stay separable. All other axis values are
shared, so the paired difference is the planning engine itself.
ensemble_id: tsa29-rh-vs-single-pass
base:
stands_path: outputs/full_tsa/data/tsa29-full-stands.parquet
yields_path: /path/to/femic-tsa29/woodstock_yields.csv
bridge_path: /path/to/femic-tsa29/ws3_bridge
base_year: 2025
horizon: 15
period_length: 10
steps: 10
workers: 1
subsidy_rate_per_m3: 30.0 # active-subsidy comparison surface
axes:
planning_mode: [rolling_horizon, single_pass]
max_workers: 2
output_root: outputs/ensemble_p8_pair
Every expanded scenario must form a valid RHRunConfig
(ensemble_scenario_invalid otherwise), which constrains mixed grids: a
single-pass arm needs horizon >= steps and the age-lattice preconditions
(see Model Semantics), and burned_stock_reset: true is rejected
with rolling_horizon. So the continuous-carry vs decade-reset comparison
is a single-pass-only grid (planning_mode: [single_pass] crossed with
burned_stock_reset: [false, true]); a cartesian grid that pairs
burned_stock_reset: true with rolling_horizon fails at expansion
time, by design.
Read the pair through the shared decade-record and manifest fields: the
principal/agent objectives, the annual harvest arrays, and
total_area_burned_ha. The summed WS3 objective is not comparable
across modes (one solve vs one per decade, D8), and the single-pass schedule
and run-level offers/decisions replace the per-step artifacts — see
Command Line for both artifact sets.
Parallelism Model
Each scenario runs
rh.run_rhin its own spawn-context worker process (ProcessPoolExecutor) with its own output root<ensemble output_root>/<scenario name>/— no shared mutable state. Spawn workers start from a clean interpreter (no fork-with-threads hazards) and inheritPYTHONPATH, which the ws3 dependency requires.The one shared input is the WS3 bridge: when the configured bridge is the canonical femic bridge, the parent process rebuilds the derived bridge once under the ensemble output root before the pool starts, and every scenario then reads it. The bridge is strictly read-only during the parallel phase; there is no write contention.
max_workers: 1runs scenarios sequentially in-process — the debug and test profile, with identical failure-capture semantics.
Failure Isolation
A scenario failure never kills the ensemble. The worker converts any
exception into a scenario record with status: "failed" and the
structured error code (the underlying RHError.code, or the exception
type name); a worker that dies without returning (pickle failure, hard
crash) is recorded as ensemble_worker_crashed. The ensemble completes
the remaining scenarios and reports:
ok— every scenario succeeded;partial— at least one failed, at least one succeeded;failed— every scenario failed.
All three exit 0: per-scenario failure is data, not a crash — unless
--strict is passed, in which case partial and failed exit 1 and
only ok exits 0. Either way the command
exits non-zero on fatal grid, input, or bridge failures (a missing
stands/yields input is detected during provenance checks before the
bridge prebuild, so it fails cheap). Each successful scenario flushes its
own RH step JSONL and manifest incrementally inside its own output root, so
partial evidence survives even a fatal late failure.
Performance Guidance
The binding constraint is the per-scenario WS3 work: at the 15-period dev horizon the per-step cost is about 15 s all-in (WS3 build plus solve ~11 s) and is serial-bound, so 10 steps lands at roughly 150 s per scenario on the reference 64-core host. Consequences:
Prefer
workers: 1per scenario on large grids and spend cores onmax_workersinstead — scenario parallelism scales; intra-scenario WS3 threads do not (workers 8 vs 64 made no measurable difference at the 15-period horizon; the smoke example’sworkers: 8is the tight default for small grids). Total threads =max_workersxworkers.Hold
workersfixed across any comparison set: WS3 step objectives shift slightly with worker count (±0.014% observed).
Recorded budgets (all on the 64-core host, 100 implemented years per scenario):
Ensemble |
Scenarios |
Settings |
Wall |
|---|---|---|---|
Smoke grid ( |
4 (2 subsidy x 2 fire) |
|
4.41x parallel speedup vs sequential |
Flip sweep, prescribed grid |
26 (13 subsidy x 2 fire) |
|
151.8 s |
Flat-sweep regression (pre-calibration) |
20 |
|
153.6 s |
Thesis-scale projection |
~1,000 |
|
~40 minutes |
Worked Example: The Subsidy Flip-Point Sweep
examples/ensemble_flip_sweep.yaml performs an adaptive binary search for
the fire-active subsidy flip, using subsidy_rate_per_m3 in [20, 30] and
at most eight midpoint iterations. Its tolerance: 0.5 stops the search
once the inactive/active subsidy bracket is no wider than $0.50/m3, so it can
finish before the iteration cap. The final bracket is recorded in
binary_search_result in the JSON summary and ensemble manifest.
max_workers: 1 is required because the search is adaptive:
fresh-salvage ensemble-run examples/ensemble_flip_sweep.yaml --json --strict
Reading the response curve. Join
data/<ensemble>-scenarios.jsonl to each scenario’s RH manifest (the
record carries manifest_path) and plot total_burned_harvest_m3
against the subsidy axis. The recorded curve (burn x1.0; from
planning/phase6-validation-report.md):
Subsidy ($/m3) |
Total salvage, 100 yr (m3) |
Reading |
|---|---|---|
0 - 23.8 |
0.00 |
Below every development type’s breakeven: salvage is a net cost, the agent declines it, and the subsidy is a pure transfer on the offered burned stock (principal objective falls ~33.4k $ per $/m3; agent objective flat at 130,041,680.42). |
23.85 |
61,549.50 |
Turn-on: the lowest-breakeven SPF DT (SPF_ESSF, 23.85) crosses zero margin. |
23.9 - 24.05 |
288,479.53 |
Ramp: SPF_MS joins; the principal’s offer surface re-prices. |
>= 24.1 |
1,338,477.16 |
Saturation: all four SPF DTs are over breakeven and the program reaches the physical maximum — the full fire influx on the offered-but-unharvested slack. |
Three structural facts to check in any sweep you run:
The burn x0.0 control is exactly zero at every subsidy — no fire, no salvage; a nonzero value there indicates a wiring defect, not a result.
Green harvest is subsidy-invariant (36,279,196.26 m3 at burn x1.0, max pairwise spread at solver precision): salvage never displaces green harvest, it recovers volume that would otherwise decay.
The flip location is the volume-weighted SPF breakeven predicted by the calibration (~23.9-24.1 $/m3); the FESBC 14-15 $/m3 benchmark sits well below the turn-on, which is exactly the gap a minimum-subsidy analysis targets. If a future recalibration moves the DT margins, expect the flip to move with them.