Stage 01b: Post-TIPSY Integration and Comparison

Scope

Stage 01b ingests BTC/BatchTIPSY output, aligns managed-track curves with VDYP untreated-track curves, and writes downstream bundle artifacts.

Required Input

  • 03_input-<unit>.csv generated by Stage 01a (canonical BTC input).

  • tipsy_params_tsa<unit>.xlsx generated by Stage 01a (human-readable mirror).

  • 04_output-<unit>.csv and 04_error-<unit>.csv generated by unattended BTC.

  • Corresponding Stage 01a outputs (AUs, VDYP curves, handoff metadata).

Legacy compatibility seam:

  • 02_input-<unit>.dat and 04_output-<unit>.out remain supported only for older manual BatchTIPSY workflows. They are no longer the default FEMIC contract.

Freshness guard

Stage 01b treats 03_input-<unit>.csv as canonical on the default BTC seam and uses input-content fingerprints for stale detection:

  • If the current input hash differs from the hash previously paired with 04_output-<unit>.csv, Stage 01b fails fast.

  • If no hash sidecar exists yet, Stage 01b falls back to input-vs-output mtime and then performs an input/output coherence check (AU/table coverage): - coherent pairs warn-and-continue by default (development-friendly mode), - incoherent pairs still fail fast.

  • Set FEMIC_STRICT_TIPSY_TIMESTAMP_MISMATCH=1 to escalate coherent timestamp mismatch back to a hard error.

  • tipsy_params_tsa<unit>.xlsx is only a human-readable mirror and is not authoritative for freshness.

  • Use FEMIC_ALLOW_STALE_TIPSY_OUTPUT=1 only for explicit debugging on the legacy DAT/OUT seam.

Legacy DAT/OUT seam note:

  • When resuming from 02_input-<unit>.dat + 04_output-<unit>.out, Stage 01b keeps the older DAT-first freshness behavior for compatibility.

Managed-curve mode note: when managed_curve_mode != tipsy (for example vdyp_transform), Stage 01b skips the BatchTIPSY freshness guard because the managed curve path does not depend on refreshed BatchTIPSY output.

Core Responsibilities

  • Parse TIPSY output tables into model-ready curve points.

  • Align/compare managed and untreated curves by AU.

  • Generate per-AU comparison plots for QA and tuning.

  • Publish updated bundle tables for export stages.

  • Apply any configured post-TIPSY yield assumptions before writing the final bundle tables. For TSA29, this includes the later section 7.1.5 rule that removes broadleaf volume from conifer-leading untreated curves while leaving THLB step 015 as the separate broadleaf-leading area exclusion.

Optional TSR Yield Assumptions

Stage 01b post-TIPSY bundling can apply a narrow instance-local TSR yield_assumptions.yaml file before writing data/model_input_bundle.

  • CLI seam:

    • femic tsa post-tipsy --yield-assumptions-path <path>

    • femic tsa btc-post-tipsy --yield-assumptions-path <path>

  • Run-profile seam:

    • modes.yield_assumptions_path: config/tsr/yield_assumptions.yaml

  • Default behavior:

    • if config/tsr/yield_assumptions.yaml exists under the instance root, the post-TIPSY workflow uses it automatically;

    • otherwise no later yield-assumption adjustment is applied.

TSA29 broadleaf rule split:

  • THLB step 015 still removes broadleaf-leading stands from THLB area.

  • TSA29 section 7.1.5 is a later bundle/yield assumption:

    • for conifer-leading untreated AUs only,

    • remove the broadleaf share from the untreated total curve,

    • zero untreated broadleaf species-proportion sidecars, and

    • renormalize the remaining untreated conifer sidecars to sum to 1.0.

Treated curves are left unchanged.

Interpretation Guide

  • Expect coherent relative behavior between untreated and managed trajectories according to scenario assumptions.

  • Identify AUs where managed curves are implausibly weak/strong and route them to parameter tuning or managed-curve transform workflows.

Exit Criteria

  • Non-empty managed and untreated curve sets for all intended AUs.

  • QA plots generated without parse/fit failures.

  • Bundle tables ready for Patchworks/Woodstock export.

Primary Legacy Notebook Coverage

Derived from 01b_run-tsa.ipynb plus parent orchestration notes in 00_data-prep.ipynb.

Linux Source-Checkout Prerequisites

Before running Stage 01b from a fresh Linux source checkout, ensure:

python -m pip install -r requirements-dev.txt
git submodule update --init --recursive
git -C external/femic-public-data annex enableremote arbutus-s3
datalad get -r external/femic-public-data/data
export FEMIC_EXTERNAL_DATA_ROOT=$PWD/external/femic-public-data/data

Known-Good Windows K3Z Resume Path

Once unattended BTC has refreshed the canonical output files:

  • external/femic-k3z-instance/data/04_output-tsak3z.csv

  • external/femic-k3z-instance/data/04_error-tsak3z.csv

resume only the downstream path:

$env:FEMIC_EXTERNAL_DATA_ROOT="$PWD\external\femic-public-data\data"
python -m femic tsa btc-post-tipsy --instance-root external/femic-k3z-instance --run-config config/run_profile.k3z.yaml --tsa k3z --run-id k3z_windows_cleanstart

The command group and flag still use the legacy tsa naming seam for compatibility. Read them generically as the selected FMU/code target.

Then continue into Patchworks export/build as needed, for example:

python -m femic patchworks build-blocks --instance-root external/femic-k3z-instance --config config/patchworks.runtime.windows.yaml
python -m femic patchworks matrix-build --instance-root external/femic-k3z-instance --config config/patchworks.runtime.windows.yaml --run-id k3z_windows_cleanstart

This is the intended Windows clean-start/restart boundary for K3Z: upstream Stage 01a writes the BTC handoff, unattended /TSR runs under FEMIC, then tsa btc-post-tipsy resumes downstream without rerunning the expensive GIS/VDYP work.