Scenario Ensembles ================== A sensitivity analysis over this model means running the full 100-year coupled pipeline once per scenario, where a scenario is one combination of values from a grid over named ``RHRunConfig`` fields. A scenario runs the rolling-horizon loop by default; ``planning_mode: single_pass`` makes it the one-solve baseline instead (see `Paired RH vs Single-Pass Runs`_ below). The ensemble driver exists to run those scenarios in parallel. This page covers why the driver exists, the grid syntax, the parallelism and failure models, performance budgeting, and a worked subsidy flip-point sweep. One-at-a-Time Sensitivity ------------------------- The Cartesian ensemble is useful when interactions between parameters are the question. For one-at-a-time analysis, use ``sensitivity-run``: .. code-block:: bash fresh-salvage sensitivity-run examples/sensitivity_tsa29.yaml --json Each configured parameter is run in its own sweep while all other fields stay at their baseline values. The driver adds a baseline value when it is absent from a sweep, then writes ``reports/-results.md``: a readable record of the effective baseline settings, every scenario and its status, the outcome at each tested value, and the signed change from that parameter's baseline. The underlying evidence remains in ``data/-observations.jsonl`` and ``data/-comparisons.csv``; full provenance is written to the sensitivity manifest. Why Ensembles ------------- The coupled model has three families of assumptions a policy analysis must sweep rather than fix: **Principal policy.** ``subsidy_rate_per_m3`` is the instrument itself — the analysis question is how salvage volume and program cost respond to the subsidy level. **Agent profit function.** The flat economic fields (``green_prices``, ``burned_price_discount``, ``green_harvest_cost``, ``burned_harvest_cost``, ``green_transport_cost_per_m3``, ``burned_transport_cost_per_m3``, ``green_stumpage_rate``, ``burned_stumpage_rate``) plus ``discount_rate`` set the agent's margins; sweeping them stress-tests the calibrated semi-synthetic surface (see :doc:`model_semantics`). **Future fire pattern.** ``burn_rate_multiplier`` scales every MFRI-derived annual burn rate: 1.0 is the published MFRI table, 0.0 a fire-free counterfactual control, 2.0 a doubled-fire future. Every ``RHRunConfig`` field except the reserved ones is a valid axis, so the same mechanism also covers structural sensitivity (``horizon``, ``steps``, ``decay_rate``, ...). Grid Syntax ----------- .. code-block:: yaml ensemble_id: tsa29-ensemble-smoke base: # shared RHRunConfig field values stands_path: outputs/full_tsa/data/tsa29-full-stands.parquet yields_path: /path/to/femic-tsa29/woodstock_yields.csv bridge_path: /path/to/femic-tsa29/ws3_bridge base_year: 2025 horizon: 15 period_length: 10 steps: 10 workers: 8 # per-scenario WS3 threads axes: # the cartesian grid subsidy_rate_per_m3: [0.0, 3.0] burn_rate_multiplier: [1.0, 2.0] max_workers: 4 output_root: outputs/ensemble_tsa29_smoke The grid is the cartesian product: one scenario per combination of axis values. Rules enforced at expansion time, before any work starts: - Every ``base`` and ``axes`` key must be an ``RHRunConfig`` field name — there is no positional ambiguity and no silent ignore (``ensemble_axis_unknown``). - ``run_id`` and ``output_root`` are **driver-owned**: the ensemble assigns them per scenario and rejects them in ``base``/``axes`` (``ensemble_field_reserved``). - ``bridge_path`` is **required in** ``base`` **and reserved as an axis**: every scenario is bound to the once-prebuilt shared bridge, so a ``bridge_path`` axis would be silently discarded. - Every axis needs at least one value (``ensemble_axis_empty``); duplicate scenario names fail (``ensemble_duplicate_scenario``); each expanded scenario must form a valid ``RHRunConfig`` (``ensemble_scenario_invalid``). Scenario names are deterministic: ``axis-value`` fragments joined by ``__`` in sorted axis order, normalized for file names (for example ``burn_rate_multiplier-1.0__subsidy_rate_per_m3-0.0``). An empty ``axes`` mapping yields the single ``baseline`` scenario. Records and artifacts are written in grid order regardless of completion order, so identical inputs give byte-comparable scenario tables. Paired RH vs Single-Pass Runs ----------------------------- ``planning_mode`` is a flat ``RHRunConfig`` field, so it is a valid ensemble axis and one grid can hold both engines. The scenario worker dispatches through ``rh.run_planning``, which routes ``rolling_horizon`` to ``rh.run_rh`` and ``single_pass`` to ``single_pass.run_single_pass``; each scenario record carries its ``planning_mode`` override and each run manifest its ``planning_mode``, so the arms stay separable. All other axis values are shared, so the paired difference is the planning engine itself. .. code-block:: yaml ensemble_id: tsa29-rh-vs-single-pass base: stands_path: outputs/full_tsa/data/tsa29-full-stands.parquet yields_path: /path/to/femic-tsa29/woodstock_yields.csv bridge_path: /path/to/femic-tsa29/ws3_bridge base_year: 2025 horizon: 15 period_length: 10 steps: 10 workers: 1 subsidy_rate_per_m3: 30.0 # active-subsidy comparison surface axes: planning_mode: [rolling_horizon, single_pass] max_workers: 2 output_root: outputs/ensemble_p8_pair Every expanded scenario must form a valid ``RHRunConfig`` (``ensemble_scenario_invalid`` otherwise), which constrains mixed grids: a single-pass arm needs ``horizon >= steps`` and the age-lattice preconditions (see :doc:`model_semantics`), and ``burned_stock_reset: true`` is rejected with ``rolling_horizon``. So the continuous-carry vs decade-reset comparison is a single-pass-only grid (``planning_mode: [single_pass]`` crossed with ``burned_stock_reset: [false, true]``); a cartesian grid that pairs ``burned_stock_reset: true`` with ``rolling_horizon`` fails at expansion time, by design. Read the pair through the shared decade-record and manifest fields: the principal/agent objectives, the annual harvest arrays, and ``total_area_burned_ha``. The summed WS3 objective is **not** comparable across modes (one solve vs one per decade, D8), and the single-pass schedule and run-level offers/decisions replace the per-step artifacts — see :doc:`cli` for both artifact sets. Parallelism Model ----------------- - Each scenario runs ``rh.run_rh`` in its own **spawn-context** worker process (``ProcessPoolExecutor``) with its own output root ``//`` — no shared mutable state. Spawn workers start from a clean interpreter (no fork-with-threads hazards) and inherit ``PYTHONPATH``, which the ws3 dependency requires. - The one shared input is the WS3 bridge: when the configured bridge is the canonical femic bridge, the parent process rebuilds the derived bridge **once** under the ensemble output root before the pool starts, and every scenario then reads it. The bridge is strictly read-only during the parallel phase; there is no write contention. - ``max_workers: 1`` runs scenarios sequentially **in-process** — the debug and test profile, with identical failure-capture semantics. Failure Isolation ----------------- A scenario failure never kills the ensemble. The worker converts any exception into a scenario record with ``status: "failed"`` and the structured error code (the underlying ``RHError.code``, or the exception type name); a worker that dies without returning (pickle failure, hard crash) is recorded as ``ensemble_worker_crashed``. The ensemble completes the remaining scenarios and reports: - ``ok`` — every scenario succeeded; - ``partial`` — at least one failed, at least one succeeded; - ``failed`` — every scenario failed. All three exit 0: per-scenario failure is data, not a crash — unless ``--strict`` is passed, in which case ``partial`` and ``failed`` exit 1 and only ``ok`` exits 0. Either way the command exits non-zero on fatal grid, input, or bridge failures (a missing stands/yields input is detected during provenance checks **before** the bridge prebuild, so it fails cheap). Each successful scenario flushes its own RH step JSONL and manifest incrementally inside its own output root, so partial evidence survives even a fatal late failure. Performance Guidance -------------------- The binding constraint is the per-scenario WS3 work: at the 15-period dev horizon the per-step cost is about 15 s all-in (WS3 build plus solve ~11 s) and is serial-bound, so 10 steps lands at roughly **150 s per scenario** on the reference 64-core host. Consequences: - Prefer ``workers: 1`` per scenario on large grids and spend cores on ``max_workers`` instead — scenario parallelism scales; intra-scenario WS3 threads do not (workers 8 vs 64 made no measurable difference at the 15-period horizon; the smoke example's ``workers: 8`` is the tight default for small grids). Total threads = ``max_workers`` x ``workers``. - Hold ``workers`` fixed across any comparison set: WS3 step objectives shift slightly with worker count (±0.014% observed). Recorded budgets (all on the 64-core host, 100 implemented years per scenario): .. list-table:: :header-rows: 1 * - Ensemble - Scenarios - Settings - Wall * - Smoke grid (``examples/ensemble_tsa29.yaml``) - 4 (2 subsidy x 2 fire) - ``max_workers: 4``, ``workers: 8`` - 4.41x parallel speedup vs sequential * - Flip sweep, prescribed grid - 26 (13 subsidy x 2 fire) - ``max_workers: 64``, ``workers: 1`` - 151.8 s * - Flat-sweep regression (pre-calibration) - 20 - ``max_workers: 64``, ``workers: 1`` - 153.6 s * - Thesis-scale projection - ~1,000 - ``max_workers: 64``, ``workers: 1`` - ~40 minutes Worked Example: The Subsidy Flip-Point Sweep -------------------------------------------- ``examples/ensemble_flip_sweep.yaml`` performs an adaptive binary search for the fire-active subsidy flip, using ``subsidy_rate_per_m3`` in [20, 30] and at most eight midpoint iterations. Its ``tolerance: 0.5`` stops the search once the inactive/active subsidy bracket is no wider than $0.50/m3, so it can finish before the iteration cap. The final bracket is recorded in ``binary_search_result`` in the JSON summary and ensemble manifest. ``max_workers: 1`` is required because the search is adaptive: .. code-block:: bash fresh-salvage ensemble-run examples/ensemble_flip_sweep.yaml --json --strict Reading the response curve. Join ``data/-scenarios.jsonl`` to each scenario's RH manifest (the record carries ``manifest_path``) and plot ``total_burned_harvest_m3`` against the subsidy axis. The recorded curve (burn x1.0; from ``planning/phase6-validation-report.md``): .. list-table:: :header-rows: 1 * - Subsidy ($/m3) - Total salvage, 100 yr (m3) - Reading * - 0 - 23.8 - 0.00 - Below every development type's breakeven: salvage is a net cost, the agent declines it, and the subsidy is a pure transfer on the offered burned stock (principal objective falls ~33.4k $ per $/m3; agent objective flat at 130,041,680.42). * - 23.85 - 61,549.50 - Turn-on: the lowest-breakeven SPF DT (SPF_ESSF, 23.85) crosses zero margin. * - 23.9 - 24.05 - 288,479.53 - Ramp: SPF_MS joins; the principal's offer surface re-prices. * - >= 24.1 - 1,338,477.16 - Saturation: all four SPF DTs are over breakeven and the program reaches the physical maximum — the full fire influx on the offered-but-unharvested slack. Three structural facts to check in any sweep you run: - **The burn x0.0 control is exactly zero** at every subsidy — no fire, no salvage; a nonzero value there indicates a wiring defect, not a result. - **Green harvest is subsidy-invariant** (36,279,196.26 m3 at burn x1.0, max pairwise spread at solver precision): salvage never displaces green harvest, it recovers volume that would otherwise decay. - The flip location is the volume-weighted SPF breakeven predicted by the calibration (~23.9-24.1 $/m3); the FESBC 14-15 $/m3 benchmark sits well below the turn-on, which is exactly the gap a minimum-subsidy analysis targets. If a future recalibration moves the DT margins, expect the flip to move with them.