Stage 01a: Strata, VDYP Curves, and TIPSY Input Generation

Scope

Stage 01a is the per-management-unit compile phase. It builds top strata, creates SI-level AU splits, runs VDYP sampling/fits, and emits BatchTIPSY input parameter files.

Key Workflow Steps

  1. Select top strata by cumulative area coverage / target-N strategy.

  2. Alias sparse strata to dominant strata where configured.

  3. Define SI bins (L/M/H) and collapse sparse bins when thresholds demand it.

  4. Run VDYP (sampling mode: auto/all/fixed-N).

  5. Smooth fitted curves and publish fit diagnostics.

  6. Generate 03_input-*.csv + workbook handoff for unattended BTC/BatchTIPSY.

Selected-AU Curve-Family Contract

Stage 01a distinguishes static AU assignment from the smaller canonical curve-family universe:

  • static AU assignment may classify every stand or polygon into the full BEC/species/SI AU universe used for provenance and review;

  • only the selected top-area strata/AU bins are normally compiled, plotted, and published as natural/untreated VDYP curve families; and

  • non-selected AU bins or stands must be remapped/imputed to one of the selected canonical curve families through the reviewed lexicographic stratum-name matching audit before downstream bundle/export work consumes curve IDs.

Do not multiply curve families to match every sparse static AU bin unless the case-specific roadmap explicitly accepts that broader curve compilation. The standard FEMIC teaching-instance pattern is:

  1. rank strata by area and select the smallest top-area set needed to meet the configured coverage target;

  2. overlay L/M/H SI classes on that selected set to define the canonical curve families;

  3. compile VDYP and managed/TIPSY curves for those selected curve families;

  4. preserve an auditable remap table from every non-selected AU bin to its selected curve-family target; and

  5. carry both the raw/static AU and selected curve-family target into the model input bundle so the downstream runtime can use stable curve IDs without losing provenance.

The TFL 6 instance is a concrete example: the reviewed static universe has 384 AU bins, but the accepted selected curve-family universe has 77 AU bins. Non-selected bins are not separate curve families; they are remapped to the selected canonical curve set in the instance remap audit. MKRF follows the same selected-AU publication pattern for runtime normalization.

For TSR-style bundles, this selected-curve contract applies to the AFLB / forested model universe after spatial netdown/resultant processing, not only to final THLB area. Fragments outside THLB are still inside the model as unmanaged/full-retention forest. They still need untreated VDYP curve assignments so retained inventory can grow and report correctly. THLB/NTHLB status must be carried as treatment-eligibility and retention attributes, not as a reason to omit non-THLB forest from Stage 01a curve coverage.

VDYP Fitting and SI Splits

  • SI split definitions are policy-driven and can vary by case.

  • For small management units, bin-collapse thresholds are required to avoid unstable regressions.

  • Tail handling and outlier controls are needed when right-tail flattening or early-age anomalies appear in binned medians.

  • Before each VDYP batch is launched, FEMIC now drops sampled polygon rows that lack matching layer rows so the polygon/layer CSV pair stays strictly feature-aligned at the external handoff seam.

  • Default per-case/per-FMU smoothing exceptions now live in config/vdyp_fit_policy.yaml.

  • Instance-specific overlays can add or adjust those rules with config/vdyp_fit_policy.yaml inside the active --instance-root checkout.

  • Override precedence is: runtime kwarg_overrides_for_tsa -> instance-local YAML overlay -> FEMIC-level YAML defaults -> narrow code fallback if the shared default YAML is missing or malformed.

TIPSY Input Boundary

  • FEMIC writes canonical BTC MSYT.csv handoff files plus workbook mirrors.

  • 03_input-*.csv is the canonical BTC/BatchTIPSY input artifact used by the unattended /TSR seam; tipsy_params_tsa*.xlsx is a human-readable mirror generated from the same table payload.

  • When FEMIC emits BTC rows with planted_percent < 100, the same canonical handoff must also carry explicit natural_species* and natural_density* payload; mixed-share rows with blank natural-ingress fields are now treated as a FEMIC contract error before BTC launch.

  • Legacy 02_input-*.dat remains a compatibility artifact only.

  • Species code mapping and SI fallback behavior should be explicit in config/tipsy/tsa*.yaml (legacy filename pattern retained for compatibility).

Operator QA Checklist

  • Confirm non-empty top strata with expected abundance coverage.

  • Confirm SI distribution plots are plausible before VDYP fitting.

  • Confirm the selected curve-family count, full static AU count, and non-selected remap count are all reported separately.

  • Confirm AU count and labels are stable and interpretable.

  • Confirm every non-selected AU bin maps to a selected curve-family target before bundle-table generation.

  • Confirm 03_input-*.csv aligns with the expected BTC MSYT.csv schema before exporting across systems.

Known Failure Signatures

  • Empty SI bins despite adequate stand counts: inspect quantile logic and collapse thresholds.

  • Flat/degenerate or wildly oscillating VDYP curves: inspect bin medians, sample size, fit overrides, and whether the sampled polygon/layer batch lost feature alignment before VDYP.

  • Raw vdyp_ply_* / vdyp_lyr_* / vdyp_out_* / vdyp_err_* spill: these now belong under vdyp_io/scratch/; if they are still collecting in the top-level vdyp_io/ root, treat that as a runtime-layout regression.

  • BTC / BatchTIPSY parse failures: usually input-schema mismatch, unsupported species/FIZ combinations, or incompatible report-template seams.

Primary Legacy Notebook Coverage

See traceability mapping for markdown cells in 01a_run-tsa.ipynb and cross-referenced driver cells in 00_data-prep.ipynb.

K3Z Teaching Baseline Notes

  • K3Z baseline managed curves now come from real BatchTIPSY output driven by VDYP-derived SI.

  • The low-yield CWHvm_CW+YC and CWHvm_CW+PLC strata are intentionally excluded from the treated/TIPSY pathway and retained out of THLB via RETENTION = 1.0.

  • Remaining treated AUs use the simplified teaching planting logic: - FD-pair AUs: 900 FD + 3100 HW - CW-pair AUs: 900 CW + 3100 HW - all other remaining treated AUs: 600 CW + 300 FD + 3100 HW

Linux Source-Checkout Prerequisites

Before running Stage 01a from a fresh Linux source checkout, ensure:

python -m pip install -r requirements-dev.txt
git submodule update --init --recursive
git -C external/femic-public-data annex enableremote arbutus-s3
datalad get -r external/femic-public-data/data
export FEMIC_EXTERNAL_DATA_ROOT=$PWD/external/femic-public-data/data

For isolated --instance-root clones, FEMIC falls back to source-root runtime assets when local copies are missing (for example data/tipsy_params_columns, vdyp_io/VDYP_CFG, and vdyp_io/VDYP.INI).

Known-Good Windows K3Z Hand-Off

On the validated Patchworks workstation, the intended K3Z Stage 01a path is:

$env:FEMIC_EXTERNAL_DATA_ROOT="$PWD\external\femic-public-data\data"
python -m femic prep validate-case --instance-root external/femic-k3z-instance --run-config config/run_profile.k3z.yaml
python -m femic run --instance-root external/femic-k3z-instance --run-config config/run_profile.k3z.yaml --run-id k3z_windows_cleanstart

Expected outcome:

  • native Windows VDYP runs using the bundled VDYP7Console.exe

  • SiteProd geoprocessing can fall back through ArcGIS Pro when needed

  • FEMIC stops intentionally at the BTC freshness boundary after writing: - external/femic-k3z-instance/data/03_input-tsak3z.csv - external/femic-k3z-instance/data/tipsy_params_tsak3z.xlsx or a timestamped fallback workbook - optionally external/femic-k3z-instance/data/02_input-tsak3z.dat as a legacy mirror

At that point, do not rerun Stage 01a unless the TIPSY handoff really needs to be regenerated. Stage 01b freshness is BTC-CSV-content based, so unchanged 03_input content can reuse existing BTC output, but real canonical input changes require a refreshed 04_output.

Curve Rescue Guard

Stage 01a curve smoothing now keeps the late fit-quality rescue pass aligned with earlier candidate-selection policy. In particular, a raw tail_blend candidate that was already rejected during tail_blend_selection is no longer allowed to re-enter later only because it clears early_overshoot_exceeds_gate. If the primary/current fit still fails the gate and no previously accepted candidate resolves that failure, Stage 01a now logs the unresolved-gate warning and keeps the better current fit instead of reviving the rejected tail blend.