Scenario Ensembles

A sensitivity analysis over this model means running the full 100-year coupled pipeline once per scenario, where a scenario is one combination of values from a grid over named RHRunConfig fields. A scenario runs the rolling-horizon loop by default; planning_mode: single_pass makes it the one-solve baseline instead (see Paired RH vs Single-Pass Runs below). The ensemble driver exists to run those scenarios in parallel. This page covers why the driver exists, the grid syntax, the parallelism and failure models, performance budgeting, and a worked subsidy flip-point sweep.

One-at-a-Time Sensitivity

The Cartesian ensemble is useful when interactions between parameters are the question. For one-at-a-time analysis, use sensitivity-run:

fresh-salvage sensitivity-run examples/sensitivity_tsa29.yaml --json

Each configured parameter is run in its own sweep while all other fields stay at their baseline values. The driver adds a baseline value when it is absent from a sweep, then writes reports/<id>-results.md: a readable record of the effective baseline settings, every scenario and its status, the outcome at each tested value, and the signed change from that parameter’s baseline. The underlying evidence remains in data/<id>-observations.jsonl and data/<id>-comparisons.csv; full provenance is written to the sensitivity manifest.

Why Ensembles

The coupled model has three families of assumptions a policy analysis must sweep rather than fix:

Principal policy. subsidy_rate_per_m3 is the instrument itself — the analysis question is how salvage volume and program cost respond to the subsidy level.

Agent profit function. The flat economic fields (green_prices, burned_price_discount, green_harvest_cost, burned_harvest_cost, green_transport_cost_per_m3, burned_transport_cost_per_m3, green_stumpage_rate, burned_stumpage_rate) plus discount_rate set the agent’s margins; sweeping them stress-tests the calibrated semi-synthetic surface (see Model Semantics).

Future fire pattern. burn_rate_multiplier scales every MFRI-derived annual burn rate: 1.0 is the published MFRI table, 0.0 a fire-free counterfactual control, 2.0 a doubled-fire future.

Every RHRunConfig field except the reserved ones is a valid axis, so the same mechanism also covers structural sensitivity (horizon, steps, decay_rate, …).

Grid Syntax

ensemble_id: tsa29-ensemble-smoke
base:                       # shared RHRunConfig field values
  stands_path: outputs/full_tsa/data/tsa29-full-stands.parquet
  yields_path: /path/to/femic-tsa29/woodstock_yields.csv
  bridge_path: /path/to/femic-tsa29/ws3_bridge
  base_year: 2025
  horizon: 15
  period_length: 10
  steps: 10
  workers: 8                # per-scenario WS3 threads
axes:                       # the cartesian grid
  subsidy_rate_per_m3: [0.0, 3.0]
  burn_rate_multiplier: [1.0, 2.0]
max_workers: 4
output_root: outputs/ensemble_tsa29_smoke

The grid is the cartesian product: one scenario per combination of axis values. Rules enforced at expansion time, before any work starts:

  • Every base and axes key must be an RHRunConfig field name — there is no positional ambiguity and no silent ignore (ensemble_axis_unknown).

  • run_id and output_root are driver-owned: the ensemble assigns them per scenario and rejects them in base/axes (ensemble_field_reserved).

  • bridge_path is required in base and reserved as an axis: every scenario is bound to the once-prebuilt shared bridge, so a bridge_path axis would be silently discarded.

  • Every axis needs at least one value (ensemble_axis_empty); duplicate scenario names fail (ensemble_duplicate_scenario); each expanded scenario must form a valid RHRunConfig (ensemble_scenario_invalid).

Scenario names are deterministic: axis-value fragments joined by __ in sorted axis order, normalized for file names (for example burn_rate_multiplier-1.0__subsidy_rate_per_m3-0.0). An empty axes mapping yields the single baseline scenario. Records and artifacts are written in grid order regardless of completion order, so identical inputs give byte-comparable scenario tables.

Paired RH vs Single-Pass Runs

planning_mode is a flat RHRunConfig field, so it is a valid ensemble axis and one grid can hold both engines. The scenario worker dispatches through rh.run_planning, which routes rolling_horizon to rh.run_rh and single_pass to single_pass.run_single_pass; each scenario record carries its planning_mode override and each run manifest its planning_mode, so the arms stay separable. All other axis values are shared, so the paired difference is the planning engine itself.

ensemble_id: tsa29-rh-vs-single-pass
base:
  stands_path: outputs/full_tsa/data/tsa29-full-stands.parquet
  yields_path: /path/to/femic-tsa29/woodstock_yields.csv
  bridge_path: /path/to/femic-tsa29/ws3_bridge
  base_year: 2025
  horizon: 15
  period_length: 10
  steps: 10
  workers: 1
  subsidy_rate_per_m3: 30.0   # active-subsidy comparison surface
axes:
  planning_mode: [rolling_horizon, single_pass]
max_workers: 2
output_root: outputs/ensemble_p8_pair

Every expanded scenario must form a valid RHRunConfig (ensemble_scenario_invalid otherwise), which constrains mixed grids: a single-pass arm needs horizon >= steps and the age-lattice preconditions (see Model Semantics), and burned_stock_reset: true is rejected with rolling_horizon. So the continuous-carry vs decade-reset comparison is a single-pass-only grid (planning_mode: [single_pass] crossed with burned_stock_reset: [false, true]); a cartesian grid that pairs burned_stock_reset: true with rolling_horizon fails at expansion time, by design.

Read the pair through the shared decade-record and manifest fields: the principal/agent objectives, the annual harvest arrays, and total_area_burned_ha. The summed WS3 objective is not comparable across modes (one solve vs one per decade, D8), and the single-pass schedule and run-level offers/decisions replace the per-step artifacts — see Command Line for both artifact sets.

Parallelism Model

  • Each scenario runs rh.run_rh in its own spawn-context worker process (ProcessPoolExecutor) with its own output root <ensemble output_root>/<scenario name>/ — no shared mutable state. Spawn workers start from a clean interpreter (no fork-with-threads hazards) and inherit PYTHONPATH, which the ws3 dependency requires.

  • The one shared input is the WS3 bridge: when the configured bridge is the canonical femic bridge, the parent process rebuilds the derived bridge once under the ensemble output root before the pool starts, and every scenario then reads it. The bridge is strictly read-only during the parallel phase; there is no write contention.

  • max_workers: 1 runs scenarios sequentially in-process — the debug and test profile, with identical failure-capture semantics.

Failure Isolation

A scenario failure never kills the ensemble. The worker converts any exception into a scenario record with status: "failed" and the structured error code (the underlying RHError.code, or the exception type name); a worker that dies without returning (pickle failure, hard crash) is recorded as ensemble_worker_crashed. The ensemble completes the remaining scenarios and reports:

  • ok — every scenario succeeded;

  • partial — at least one failed, at least one succeeded;

  • failed — every scenario failed.

All three exit 0: per-scenario failure is data, not a crash — unless --strict is passed, in which case partial and failed exit 1 and only ok exits 0. Either way the command exits non-zero on fatal grid, input, or bridge failures (a missing stands/yields input is detected during provenance checks before the bridge prebuild, so it fails cheap). Each successful scenario flushes its own RH step JSONL and manifest incrementally inside its own output root, so partial evidence survives even a fatal late failure.

Performance Guidance

The binding constraint is the per-scenario WS3 work: at the 15-period dev horizon the per-step cost is about 15 s all-in (WS3 build plus solve ~11 s) and is serial-bound, so 10 steps lands at roughly 150 s per scenario on the reference 64-core host. Consequences:

  • Prefer workers: 1 per scenario on large grids and spend cores on max_workers instead — scenario parallelism scales; intra-scenario WS3 threads do not (workers 8 vs 64 made no measurable difference at the 15-period horizon; the smoke example’s workers: 8 is the tight default for small grids). Total threads = max_workers x workers.

  • Hold workers fixed across any comparison set: WS3 step objectives shift slightly with worker count (±0.014% observed).

Recorded budgets (all on the 64-core host, 100 implemented years per scenario):

Ensemble

Scenarios

Settings

Wall

Smoke grid (examples/ensemble_tsa29.yaml)

4 (2 subsidy x 2 fire)

max_workers: 4, workers: 8

4.41x parallel speedup vs sequential

Flip sweep, prescribed grid

26 (13 subsidy x 2 fire)

max_workers: 64, workers: 1

151.8 s

Flat-sweep regression (pre-calibration)

20

max_workers: 64, workers: 1

153.6 s

Thesis-scale projection

~1,000

max_workers: 64, workers: 1

~40 minutes

Worked Example: The Subsidy Flip-Point Sweep

examples/ensemble_flip_sweep.yaml performs an adaptive binary search for the fire-active subsidy flip, using subsidy_rate_per_m3 in [20, 30] and at most eight midpoint iterations. Its tolerance: 0.5 stops the search once the inactive/active subsidy bracket is no wider than $0.50/m3, so it can finish before the iteration cap. The final bracket is recorded in binary_search_result in the JSON summary and ensemble manifest. max_workers: 1 is required because the search is adaptive:

fresh-salvage ensemble-run examples/ensemble_flip_sweep.yaml --json --strict

Reading the response curve. Join data/<ensemble>-scenarios.jsonl to each scenario’s RH manifest (the record carries manifest_path) and plot total_burned_harvest_m3 against the subsidy axis. The recorded curve (burn x1.0; from planning/phase6-validation-report.md):

Subsidy ($/m3)

Total salvage, 100 yr (m3)

Reading

0 - 23.8

0.00

Below every development type’s breakeven: salvage is a net cost, the agent declines it, and the subsidy is a pure transfer on the offered burned stock (principal objective falls ~33.4k $ per $/m3; agent objective flat at 130,041,680.42).

23.85

61,549.50

Turn-on: the lowest-breakeven SPF DT (SPF_ESSF, 23.85) crosses zero margin.

23.9 - 24.05

288,479.53

Ramp: SPF_MS joins; the principal’s offer surface re-prices.

>= 24.1

1,338,477.16

Saturation: all four SPF DTs are over breakeven and the program reaches the physical maximum — the full fire influx on the offered-but-unharvested slack.

Three structural facts to check in any sweep you run:

  • The burn x0.0 control is exactly zero at every subsidy — no fire, no salvage; a nonzero value there indicates a wiring defect, not a result.

  • Green harvest is subsidy-invariant (36,279,196.26 m3 at burn x1.0, max pairwise spread at solver precision): salvage never displaces green harvest, it recovers volume that would otherwise decay.

  • The flip location is the volume-weighted SPF breakeven predicted by the calibration (~23.9-24.1 $/m3); the FESBC 14-15 $/m3 benchmark sits well below the turn-on, which is exactly the gap a minimum-subsidy analysis targets. If a future recalibration moves the DT margins, expect the flip to move with them.