femic.pipeline.tipsy Module

The femic.pipeline.tipsy module owns FEMIC’s BTC/BatchTIPSY handoff seam. It translates smoothed VDYP outputs into per-AU TIPSY parameter tables, writes the canonical 03_input-*.csv handoff plus workbook mirrors, manages BTC report templates and unattended /TSR execution, and validates or parses returned BTC/TIPSY outputs during Stage 01b.

If you are debugging why FEMIC generated the wrong BTC input rows, why a report template or unattended BTC run produced the wrong output, or why Stage 01b is refusing to accept an existing returned file, this is the first module to read. In practice it owns:

  • the BTC MSYT.csv input schema and writer

  • candidate evaluation and AU/SI selection for TIPSY parameter generation

  • writing the canonical BTC handoff plus the human-readable XLSX mirror

  • BTC custom report template parsing/building/writing

  • unattended BTC runner argument assembly and manifest support

  • fingerprinting and freshness validation for returned BTC or legacy BatchTIPSY output

  • coherence-based stale-output acceptance logic for repeated dev/test reruns

Start Here If…

Use this page first if you are trying to:

  • understand why 03_input-*.csv is treated as canonical while tipsy_params_tsa*.xlsx is only a mirror built on the legacy tsa filename seam

  • debug BTC parse failures caused by input-schema mismatch or unsafe report templates

  • inspect why a stratum/SI candidate was excluded from TIPSY parameter generation

  • trace why Stage 01b accepted or rejected an older returned BTC/TIPSY output

  • understand how managed_curve_mode=vdyp_transform changes the BTC/TIPSY boundary behavior

Typical maintenance path:

  1. Start with build_tipsy_params_for_tsa() and evaluate_tipsy_candidate() if the issue is about which AU/SI/species combinations make it into the handoff.

  2. Move to build_tipsy_input_table(), write_tipsy_input_exports(), and write_btc_input_csv() if the problem is about canonical handoff generation.

  3. Read run_btc_cli(), parse_btc_custom_report_template(), and write_btc_custom_report_template() when the failure is visible at the unattended BTC runtime boundary.

  4. Read validate_tipsy_output_is_fresh(), assess_tipsy_input_output_coherence(), write_tipsy_output_input_fingerprint(), and parse_btc_tsr_transposed_output() when the failure is visible at Stage 01b resume.

Typical Usage

The common operator-facing pattern is to let Stage 01a write the canonical BTC handoff, run unattended BTC, and then parse the returned output before Stage 01b resumes:

from pathlib import Path
from femic.pipeline.tipsy import run_btc_cli

run_btc_cli(
    input_csv_path=Path("data/03_input-tsa08.csv"),
    output_path=Path("data/04_output-tsa08.csv"),
    error_path=Path("data/04_error-tsa08.csv"),
)

How This Fits Into The Pipeline

This module owns the default unattended BTC seam plus the remaining legacy BatchTIPSY compatibility boundary described in:

At a high level, the owning sequence is:

  1. Stage 01a selects eligible AU/SI candidates and builds TIPSY parameter rows

  2. FEMIC writes 03_input-*.csv and tipsy_params_tsa*.xlsx

  3. BTC runs unattended under FEMIC and returns 04_output-*.csv / 04_error-*.csv

  4. Stage 01b validates/parses that output against the current canonical BTC handoff

That means this module is both a data-shaping layer and a workflow boundary guard. It now runs BTC directly, while still carrying the older DAT/OUT freshness rules as a compatibility seam.

Key Entry Surfaces

The highest-value entrypoints in this module are:

  • build_tipsy_params_for_tsa() Generate the per-AU TIPSY parameter payloads from smoothed VDYP outputs.

  • evaluate_tipsy_candidate() Decide whether one stratum/SI candidate is eligible and why.

  • build_tipsy_input_table() Turn per-AU parameter payloads into the tabular export surface.

  • write_tipsy_input_exports() Write the canonical BTC handoff and workbook mirror for one selected FMU/code target. The exported workbook filename still follows the legacy tsa pattern for compatibility.

  • write_btc_input_csv() Write the canonical MSYT.csv-style BTC input file for one selected FMU/code target.

  • run_btc_cli() Launch unattended TIPSYbtc.exe /TSR against a canonical BTC handoff.

  • validate_tipsy_output_is_fresh() Enforce the Stage 01b freshness guard against the canonical input seam.

  • assess_tipsy_input_output_coherence() Decide whether an older output still looks structurally coherent with the current input workbook.

  • write_tipsy_output_input_fingerprint() Persist the canonical input SHA256 sidecar paired with an accepted output file.

  • parse_btc_custom_report_template() Read an existing BTC .rpt custom report into a structured template.

  • build_btc_custom_report_template() Build a curated BTC report template from a preset or an existing template.

  • write_btc_custom_report_template() Write a BTC .rpt report file back to disk.

  • parse_btc_tsr_transposed_output() Parse the vetted unattended /TSR transposed CSV output back into FEMIC managed-curve rows.

BTC Report Template Support

This module now also carries the first FEMIC-side utilities for BTC custom report templates. That work is still part of the broader Phase 48 BTC cutover, but it already supports a useful maintenance pattern:

  1. parse an existing BTC .rpt file,

  2. clone or extend its column list in Python, and

  3. write a vetted replacement template back out.

The first built-in unattended preset is the transposed TSR mashup that safely combines:

  • merchantable volume

  • height

  • gross volume

  • crown closure

That preset exists because live local probes showed that /TSR is report-coupled: replacing TimberSupply.rpt with a compatible transposed report template changes what /TSR emits. Not every report type is a safe drop-in replacement, so FEMIC should prefer vetted compatible templates over arbitrary all-fields output experiments.

Critical /TSR Overlay Precedence Insight

One critical reverse-engineering result must not be lost:

  • plain installed TIPSYbtc.exe /TSR consults the user-overlay report under the current user’s Windows Documents folder: - <Documents>\BatchTIPSY Composer\TimberSupply.rpt before falling back to the stock installed TimberSupply.rpt

  • a broken overlay can therefore make stock-looking /TSR runs fail even when the installed BTC report under Program Files is fine

  • removing the overlay restores stock fallback behavior

  • replacing the overlay with a stock-based safe enhanced TSR template lets plain installed /TSR run successfully while still extending the output surface

For unattended FEMIC BTC work, this live user-overlay path is the only known-valid `/TSR` seam. Treat that as an operating constraint, not as a soft preference.

One adjacent negative result is also now important enough to document here:

  • the undocumented BTC /No_GUI switch is not a proven FEMIC runtime seam

  • runtime probes and direct decompilation both indicate that /No_GUI acts as a visibility toggle rather than a useful execution trigger

  • plain /No_GUI <project>.btc loads passive project state into a hidden BTC session but does not automatically process or export anything

  • /TSR and /FLP remain the only proven useful command-line execution triggers for unattended FEMIC BTC work

  • if another hidden execution seam exists, it is more likely to be another startup trigger than a post-launch control channel

Treat that /No_GUI result as a stable operating conclusion, not as an invitation to keep probing it during ordinary FEMIC maintenance.

This matters because early copied-install/generated-template probes were too pessimistic. They were useful clues, but they were not exercising the most faithful live /TSR seam. The safest unattended extension path is now:

  1. start from the actual stock TimberSupply.rpt structure

  2. extend it conservatively through the live user-overlay seam

  3. test plain installed /TSR

Do not assume a clean-room generated replacement template is equivalent to the stock report contract just because the visible fields look similar.

Do not treat the following as decision-making proof for unattended FEMIC /TSR behavior:

  • copied-install-local TimberSupply.rpt overrides

  • stock-report swaps done outside the live user Documents overlay

  • probes that do not explicitly pass through <Documents>\BatchTIPSY Composer\TimberSupply.rpt

FEMIC now resolves that overlay path generically from the current user’s Windows Documents directory instead of relying on a machine-specific OneDrive path assumption.

The current stock-based unattended patch path also forces the TSR horizon to:

  • TableRange=0-350:10|# MAX=350 INC=10

so the unattended BTC output timeline lines up with FEMIC’s longer VDYP curve timeline instead of stopping at the stock 120-year range.

Why This Matters For Richer Indicator Probing

This same overlay insight overturned the first bleak stand-table conclusion. When the first-batch candidates were re-probed through the real overlay seam instead of a stand-alone generated replacement template, all of these columns passed cleanly:

  • MAI

  • BasalArea:000

  • DBHg:000

  • SPH:000

  • StemCount000

  • StemCount125

  • StemCount175

So the main compatibility rule appears to be structural:

  • preserving the hidden stock TimberSupply.rpt contract matters a great deal

  • some earlier failures were seam-mismatch artifacts, not proof that the columns were impossible through unattended /TSR

  • when BTC exposes the same metric at 000, 125, and 175 top- diameter merchantable cutoffs, FEMIC should treat that triplet as an atomic bank-design unit so downstream forest-model users can compare the delta between thresholds rather than being stranded with only one cutoff surface

Optional Unattended Indicator Banks

FEMIC now has real optional BTC indicator-bank switches on top of the core unattended /TSR seam:

  • --indicator-bank stand-structure-basic

  • --indicator-bank stand-structure-threshold-raw

  • --indicator-bank yield-and-age-core

  • --indicator-bank genetics-fertilization-and-oaf

  • --indicator-bank tass-and-site-index-raw

  • --indicator-bank log-grades

  • --indicator-bank lumber-2-or-better

  • --indicator-bank lumber-graded

  • --indicator-bank lumber-degraded

  • --indicator-bank industrial-logs

  • --indicator-bank residual-fibre

  • --indicator-bank mortality-summary

  • --indicator-bank crop250-stand-quality

  • --indicator-bank crown-and-fire

  • --indicator-bank biomass-live

  • --indicator-bank biomass-dead

  • --indicator-bank carbon

  • --indicator-bank co2e

  • --indicator-bank mortality-size-classes

  • --indicator-bank diameter-class-stems

  • --indicator-bank diameter-class-volume

  • --indicator-bank diameter-class-vpt

Current bank contents:

  • stand-structure-basic: - MAI - BasalArea:000 - DBHg:000 - SPH:000 - StemCount000 - StemCount125 - StemCount175

  • stand-structure-threshold-raw: - Volume000 - Volume125 - Volume175 - BasalArea000 - BasalArea125 - BasalArea175 - MeanDBHg000 - MeanDBHg125 - MeanDBHg175 - MAI000 - MAI125 - MAI175 - VPT000 - VPT125 - VPT175 - Juvenille_Volume000 - Juvenille_Volume125 - Juvenille_Volume175 - Juvenille_Percent000 - Juvenille_Percent125 - Juvenille_Percent175

  • yield-and-age-core: - Year - TotalAge - BHAge - StandAge - HeightSindex - Height - Volume - VPT - HeightTassTop - HeightTassMean - HeightTassPredom

  • genetics-fertilization-and-oaf: - GWgain - FertGain - OAFremoval - OAFmortality - OAFimpact - OAF

  • tass-and-site-index-raw: - YearTASS_Base - HeightSindex_Base - YearTASS_Full - HeightSindex_Full

  • log-grades: - Logs_Grade_D - Logs_Grade_F - Logs_Grade_H - Logs_Grade_I - Logs_Grade_J - Logs_Grade_U - Logs_Grade_X - Logs_Grade_Y - default shipped bank excludes Logs_Grade_All because that BTC field is a separate scaled-log metric rather than an additive member of the explicit merchantable-grade partition - downstream compile recipes may still opt Logs_Grade_All back in explicitly when a model wants that separate metric

  • lumber-2-or-better: - Lumber_2_or_Better_2x4 - Lumber_2_or_Better_2x6 - Lumber_2_or_Better_2x8 - Lumber_2_or_Better_2x10 - Lumber_2_or_Better_All - LRF_2_or_Better_All

  • lumber-graded: - Lumber_Graded_SS_2x4 - Lumber_Graded_SS_2x6 - Lumber_Graded_SS_2x8 - Lumber_Graded_SS_2x10 - Lumber_Graded_1_2x4 - Lumber_Graded_1_2x6 - Lumber_Graded_1_2x8 - Lumber_Graded_1_2x10 - Lumber_Graded_2_2x4 - Lumber_Graded_2_2x6 - Lumber_Graded_2_2x8 - Lumber_Graded_2_2x10 - Lumber_Graded_3_2x4 - Lumber_Graded_3_2x6 - Lumber_Graded_3_2x8 - Lumber_Graded_3_2x10 - Lumber_Graded_4_2x4 - Lumber_Graded_4_2x6 - Lumber_Graded_4_2x8 - Lumber_Graded_4_2x10 - Lumber_Graded_All - LRF_Graded_All

  • lumber-degraded: - Lumber_Degraded_SS_2x4 - Lumber_Degraded_SS_2x6 - Lumber_Degraded_SS_2x8 - Lumber_Degraded_SS_2x10 - Lumber_Degraded_1_2x4 - Lumber_Degraded_1_2x6 - Lumber_Degraded_1_2x8 - Lumber_Degraded_1_2x10 - Lumber_Degraded_2_2x4 - Lumber_Degraded_2_2x6 - Lumber_Degraded_2_2x8 - Lumber_Degraded_2_2x10 - Lumber_Degraded_3_2x4 - Lumber_Degraded_3_2x6 - Lumber_Degraded_3_2x8 - Lumber_Degraded_3_2x10 - Lumber_Degraded_4_2x4 - Lumber_Degraded_4_2x6 - Lumber_Degraded_4_2x8 - Lumber_Degraded_4_2x10 - Lumber_Degraded_All - LRF_Degraded_All

  • industrial-logs: - Industrial_Logs_D38L13 - Industrial_Logs_D38L11 - Industrial_Logs_D38L8 - Industrial_Logs_D30L13 - Industrial_Logs_D30L11 - Industrial_Logs_D30L8 - Industrial_Logs_D20L13 - Industrial_Logs_D20L11 - Industrial_Logs_D20L8 - Industrial_Logs_D125L13 - Industrial_Logs_D125L11 - Industrial_Logs_D125L8 - Industrial_Logs_D125L63 - Industrial_Logs_D125L51 - Industrial_Logs_D125L5 - Industrial_Logs_D305 - Industrial_Logs_D254 - Industrial_Logs_D203 - Industrial_Logs_D178 - Industrial_Logs_D152

  • residual-fibre: - Residual_Chips - Residual_Sawdust - Residual_Shavings - Residual_Trim - Residual_Bark

  • mortality-summary: - Mortality_Stems - Mortality_DBHg_Mean - Mortality_Height_Mean - Mortality_Basal_Area - Mortality_Volume_Total

  • crop250-stand-quality: - Crop250VolUtil125 - Crop250DBHgMean - Crop250LiveCrown

  • crown-and-fire: - CrownCover - mean_height_to_crown_base - mean_crown_length - Crown_Bulk_Density

  • biomass-live: - Biomass_Live_Wood - Biomass_Live_Bark - Biomass_Live_Foliar - Biomass_Live_Branch - Biomass_Live_Roots - Biomass_Live_Total - Biomass_Live_Above

  • biomass-dead: - Biomass_Dead_Wood - Biomass_Dead_Bark - Biomass_Dead_Foliar - Biomass_Dead_Branch - Biomass_Dead_Roots - Biomass_Dead_Total - Biomass_Dead_Above

  • carbon: - Carbon_Live_Wood - Carbon_Live_Bark - Carbon_Live_Foliar - Carbon_Live_Branch - Carbon_Live_Roots - Carbon_Live_Total - Carbon_Live_Above - Carbon_Dead_Wood - Carbon_Dead_Bark - Carbon_Dead_Foliar - Carbon_Dead_Branch - Carbon_Dead_Roots - Carbon_Dead_Total - Carbon_Dead_Above

  • co2e: - CO2e_Live_Wood - CO2e_Live_Bark - CO2e_Live_Foliar - CO2e_Live_Branch - CO2e_Live_Roots - CO2e_Live_Total - CO2e_Live_Above - CO2e_Dead_Wood - CO2e_Dead_Bark - CO2e_Dead_Foliar - CO2e_Dead_Branch - CO2e_Dead_Roots - CO2e_Dead_Total - CO2e_Dead_Above

  • mortality-size-classes: - Mortality_Stems_Size_Class_{5,15,25,35,45,55,65} - Mortality_Volume_Size_Class_{5,15,25,35,45,55,65} - Mortality_VPT_Size_Class_{5,15,25,35,45,55,65}

  • diameter-class-stems: - Stems_Diameter_Class_{0,5,10,...,90}

  • diameter-class-volume: - Volume_Diameter_Class_{0,5,10,...,90}

  • diameter-class-vpt: - VPT_Diameter_Class_{0,5,10,...,90}

Important runtime detail:

  • the working implementation patches the real per-user overlay report path under <Documents>\BatchTIPSY Composer\TimberSupply.rpt with backup/restore;

  • relying only on a copied-install-local TimberSupply.rpt is not enough, because the live overlay can silently shadow that local file and make the run appear successful while dropping the requested bank columns from the returned output.

  • BTC/TIPSY runtime artifacts now default under tipsy_io/logs and tipsy_io/scratch so operator supervision is not visually mixed into the VDYP runtime namespace.

  • live unattended /TSR overlay smokes must be run sequentially, not in parallel, because they share the same per-user TimberSupply.rpt overlay.

Live smoke proof now exists for:

  • femic tipsy run-btc <MSYT.csv> --indicator-bank stand-structure-basic

  • femic tipsy run-btc <MSYT.csv> --indicator-bank stand-structure-threshold-raw

  • femic tipsy run-btc <MSYT.csv> --indicator-bank yield-and-age-core

  • femic tipsy run-btc <MSYT.csv> --indicator-bank genetics-fertilization-and-oaf

  • femic tipsy run-btc <MSYT.csv> --indicator-bank tass-and-site-index-raw

  • femic tipsy run-btc <MSYT.csv> --indicator-bank log-grades

  • femic tipsy run-btc <MSYT.csv> --indicator-bank lumber-2-or-better

  • femic tipsy run-btc <MSYT.csv> --indicator-bank lumber-graded

  • femic tipsy run-btc <MSYT.csv> --indicator-bank lumber-degraded

  • femic tipsy run-btc <MSYT.csv> --indicator-bank industrial-logs

  • femic tipsy run-btc <MSYT.csv> --indicator-bank residual-fibre

  • femic tipsy run-btc <MSYT.csv> --indicator-bank mortality-summary

  • femic tipsy run-btc <MSYT.csv> --indicator-bank crop250-stand-quality

  • femic tipsy run-btc <MSYT.csv> --indicator-bank crown-and-fire

  • femic tipsy run-btc <MSYT.csv> --indicator-bank biomass-live

  • femic tipsy run-btc <MSYT.csv> --indicator-bank biomass-dead

  • femic tipsy run-btc <MSYT.csv> --indicator-bank carbon

  • femic tipsy run-btc <MSYT.csv> --indicator-bank co2e

  • femic tipsy run-btc <MSYT.csv> --indicator-bank mortality-size-classes

  • femic tipsy run-btc <MSYT.csv> --indicator-bank diameter-class-stems

  • femic tipsy run-btc <MSYT.csv> --indicator-bank diameter-class-volume

  • femic tipsy run-btc <MSYT.csv> --indicator-bank diameter-class-vpt

That returned a single unattended output CSV with:

The only remaining canonical BTC outputs that do not currently fit cleanly into the optional-bank rollout are the non-threshold Juvenille_Volume and Juvenille_Percent totals, which still trigger live-overlay BTC modal failures. Their threshold-specific 000/125/175 variants are already shipped through stand-structure-threshold-raw.

  • the default conservative families: - MVcon_* - MVdec_* - HTcon_* - HTdec_* - gVol_* - CC_*

  • plus the first stand-structure bank: - MAI_* - BasalArea000_* - DBHg000_* - SPH000_* - StemCount000_* - StemCount125_* - StemCount175_*

  • plus the log-grade bank: - Logs_Grade_D_* - Logs_Grade_F_* - Logs_Grade_H_* - Logs_Grade_I_* - Logs_Grade_J_* - Logs_Grade_U_* - Logs_Grade_X_* - Logs_Grade_Y_* - Logs_Grade_All_* only when the bank is explicitly configured to include the separate all-grades metric

  • plus the lumber-2-or-better bank: - Lumber_2_or_Better_2x4_* - Lumber_2_or_Better_2x6_* - Lumber_2_or_Better_2x8_* - Lumber_2_or_Better_2x10_* - Lumber_2_or_Better_All_* - LRF_2_or_Better_All_*

  • plus the mortality-summary bank: - Mortality_Stems_* - Mortality_DBHg_Mean_* - Mortality_Height_Mean_* - Mortality_Basal_Area_* - Mortality_Volume_Total_*

  • plus the crop250-stand-quality bank: - Crop250VolUtil125_* - Crop250DBHgMean_* - Crop250LiveCrown_*

  • plus the crown-and-fire bank: - CrownCover_* - mean_height_to_crown_base_* - mean_crown_length_* - Crown_Bulk_Density_*

  • plus the biomass-live bank: - Biomass_Live_Wood_* - Biomass_Live_Bark_* - Biomass_Live_Foliar_* - Biomass_Live_Branch_* - Biomass_Live_Roots_* - Biomass_Live_Total_* - Biomass_Live_Above_*

  • plus the biomass-dead bank: - Biomass_Dead_Wood_* - Biomass_Dead_Bark_* - Biomass_Dead_Foliar_* - Biomass_Dead_Branch_* - Biomass_Dead_Roots_* - Biomass_Dead_Total_* - Biomass_Dead_Above_*

  • plus the carbon bank: - Carbon_Live_Wood_* - Carbon_Live_Bark_* - Carbon_Live_Foliar_* - Carbon_Live_Branch_* - Carbon_Live_Roots_* - Carbon_Live_Total_* - Carbon_Live_Above_* - Carbon_Dead_Wood_* - Carbon_Dead_Bark_* - Carbon_Dead_Foliar_* - Carbon_Dead_Branch_* - Carbon_Dead_Roots_* - Carbon_Dead_Total_* - Carbon_Dead_Above_*

  • plus the co2e bank: - CO2e_Live_Wood_* - CO2e_Live_Bark_* - CO2e_Live_Foliar_* - CO2e_Live_Branch_* - CO2e_Live_Roots_* - CO2e_Live_Total_* - CO2e_Live_Above_* - CO2e_Dead_Wood_* - CO2e_Dead_Bark_* - CO2e_Dead_Foliar_* - CO2e_Dead_Branch_* - CO2e_Dead_Roots_* - CO2e_Dead_Total_* - CO2e_Dead_Above_*

  • plus the lumber-graded bank: - Lumber_Graded_SS_2x4_* - Lumber_Graded_1_2x4_* - Lumber_Graded_2_2x4_* - Lumber_Graded_3_2x4_* - Lumber_Graded_4_2x4_* - Lumber_Graded_All_* - LRF_Graded_All_*

  • plus the lumber-degraded bank: - Lumber_Degraded_SS_2x4_* - Lumber_Degraded_1_2x4_* - Lumber_Degraded_2_2x4_* - Lumber_Degraded_3_2x4_* - Lumber_Degraded_4_2x4_* - Lumber_Degraded_All_* - LRF_Degraded_All_*

  • plus the industrial-logs bank: - Industrial_Logs_D38L13_* - Industrial_Logs_D30L13_* - Industrial_Logs_D20L13_* - Industrial_Logs_D125L13_* - Industrial_Logs_D125L5_* - Industrial_Logs_D305_* - Industrial_Logs_D152_*

  • plus the residual-fibre bank: - Residual_Chips_* - Residual_Sawdust_* - Residual_Shavings_* - Residual_Trim_* - Residual_Bark_*

while still honoring the 350-year unattended TSR timeline.

The small dataclasses in this module are also useful because they define the candidate/freshness contracts explicitly:

  • TIPSYCandidateEvaluation

  • TipsyInputOutputCoherence

Canonical Artifacts And Contracts

The most important operator/runtime contracts in this module are:

  • 03_input-*.csv is the canonical BTC/BatchTIPSY input artifact

  • tipsy_params_tsa*.xlsx is a human-readable mirror of the same content, not the authoritative freshness source

  • 04_output-*.csv / 04_error-*.csv are the default returned BTC artifacts for Stage 01b

  • legacy 02_input-*.dat / 04_output-*.out remain supported only for compatibility with older manual BatchTIPSY workflows

  • when a returned output is accepted, FEMIC can store an input SHA256 sidecar so later reruns know which exact handoff produced that output

These rules are why this module is so sensitive: a seemingly small field-width change or a misunderstood stale-output policy can silently distort downstream managed-curve comparisons.

Freshness And Coherence Policy

The key freshness behavior in this module is:

  • if allow_stale is enabled, the hard freshness guard is bypassed entirely

  • otherwise FEMIC prefers canonical-input-based validation over workbook timestamp checks

  • if a fingerprint sidecar exists and its stored canonical input SHA256 differs from the current canonical input SHA256, Stage 01b fails fast

  • if output timestamps are older than the current canonical input, FEMIC now performs a structural coherence check using AU/table coverage before deciding whether to stop

  • coherent timestamp mismatch warns and continues by default

  • strict_timestamp_mismatch converts that coherent warning path back into a hard error

  • when managed_curve_mode != tipsy the broader workflow may skip this boundary because managed curves are no longer driven by refreshed TIPSY output

This is the code-level owner of the guidance documented in the Stage 01b guide. If the docs and runtime ever seem inconsistent about stale 04_output reuse, inspect this module first.

Failure Seams To Watch

The common failure boundaries in this module are:

  • BTC handoff/report regressions schema drift, unsafe report columns, or report-template mismatches can make BTC reject the handoff or crash during batch processing

  • candidate exclusion surprises low-volume, low-SI, excluded-leading-species, or no-species-candidate paths can remove rows operators expected to see in the handoff

  • stale output confusion the most common Stage 01b operator error is reusing an old 04_output file after the canonical input content changed materially

  • coherence false assumptions timestamp mismatch does not always mean the output is invalid; this module explicitly distinguishes structurally coherent reruns from real stale-output drift

  • workbook-only reasoning code or docs that treat the XLSX mirror as canonical will eventually disagree with Stage 01b’s canonical-input-first logic

Cross-References

Guides and references that pair especially closely with this module:

Related API pages:

Reusable TIPSY parameter helper utilities.

class femic.pipeline.tipsy.BTCColumnProbeResult(candidate_token, status, accepted_column_tokens, run_id, exit_code=None, error_message=None, manifest_path=None, output_csv_path=None, error_csv_path=None, output_created=None, error_created=None, dialog_auto_closed=None, dialog_close_attempted=None, failure_classification=None, report_type=None, identifier_mode=None, output_format=None, probe_token=None, probe_header1_override=None, probe_header2_override=None, probe_units_override=None, variant_id=None, variant_label=None, variant_source_report=None, variant_source_kind=None, attempted_variants=(), clues=None)[source]

Bases: object

Result for one incremental BTC report-column compatibility probe.

Parameters:

  • candidate_token (str)

  • status (str)

  • accepted_column_tokens (tuple[str, ...])

  • run_id (str)

  • exit_code (int | None)

  • error_message (str | None)

  • manifest_path (Path | None)

  • output_csv_path (Path | None)

  • error_csv_path (Path | None)

  • output_created (bool | None)

  • error_created (bool | None)

  • dialog_auto_closed (bool | None)

  • dialog_close_attempted (bool | None)

  • failure_classification (str | None)

  • report_type (str | None)

  • identifier_mode (str | None)

  • output_format (str | None)

  • probe_token (str | None)

  • probe_header1_override (str | None)

  • probe_header2_override (str | None)

  • probe_units_override (str | None)

  • variant_id (str | None)

  • variant_label (str | None)

  • variant_source_report (str | None)

  • variant_source_kind (str | None)

  • attempted_variants (tuple[str, ...])

  • clues (Mapping[str, Any] | None)

accepted_column_tokens: tuple[str, ...]
attempted_variants: tuple[str, ...] = ()
candidate_token: str
clues: Mapping[str, Any] | None = None
dialog_auto_closed: bool | None = None
dialog_close_attempted: bool | None = None
error_created: bool | None = None
error_csv_path: Path | None = None
error_message: str | None = None
exit_code: int | None = None
failure_classification: str | None = None
identifier_mode: str | None = None
manifest_path: Path | None = None
output_created: bool | None = None
output_csv_path: Path | None = None
output_format: str | None = None
probe_header1_override: str | None = None
probe_header2_override: str | None = None
probe_token: str | None = None
probe_units_override: str | None = None
report_type: str | None = None
run_id: str
status: str
variant_id: str | None = None
variant_label: str | None = None
variant_source_kind: str | None = None
variant_source_report: str | None = None
class femic.pipeline.tipsy.BTCCustomReportColumn(token, width=0, header1_override='', header2_override='', units_override='', raw_line='')[source]

Bases: object

One column entry in a BTC custom report template.

Parameters:

  • token (str)

  • width (int)

  • header1_override (str)

  • header2_override (str)

  • units_override (str)

  • raw_line (str)

header1_override: str = ''
header2_override: str = ''
raw_line: str = ''
render()[source]
Return type:

str

token: str
units_override: str = ''
width: int = 0
class femic.pipeline.tipsy.BTCCustomReportTemplate(name, icon_id=13, identifier='FirstIDcolumn', identifier_integer=True, report_type='databaseByStand', output_format='TAB', border=500, header_height=250, footer_height=250, header_flags=None, columns=())[source]

Bases: object

Structured BTC custom report template.

Parameters:

  • name (str)

  • icon_id (int)

  • identifier (str)

  • identifier_integer (bool)

  • report_type (str)

  • output_format (str)

  • border (int)

  • header_height (int)

  • footer_height (int)

  • header_flags (Mapping[str, str] | None)

  • columns (Sequence[BTCCustomReportColumn])

border: int = 500
columns: Sequence[BTCCustomReportColumn] = ()
footer_height: int = 250
header_flags: Mapping[str, str] | None = None
header_height: int = 250
icon_id: int = 13
identifier: str = 'FirstIDcolumn'
identifier_integer: bool = True
name: str
output_format: str = 'TAB'
render()[source]
Return type:

str

report_type: str = 'databaseByStand'
class femic.pipeline.tipsy.BTCProbeColumnVariant(variant_id, label, column, source_report=None, source_report_type=None, source_kind='generic')[source]

Bases: object

One concrete report-line variant for probing a candidate BTC token.

Parameters:

  • variant_id (str)

  • label (str)

  • column (BTCCustomReportColumn)

  • source_report (str | None)

  • source_report_type (str | None)

  • source_kind (str)

column: BTCCustomReportColumn
label: str
source_kind: str = 'generic'
source_report: str | None = None
source_report_type: str | None = None
variant_id: str
class femic.pipeline.tipsy.BTCRunResult(run_id, mode, command, manifest_path, stdout_log_path, stderr_log_path, output_csv_path, error_csv_path, executable_path, install_root, working_dir, copied_install, exit_code, duration_sec, report_template_path, uses_live_overlay=False)[source]

Bases: object

Result payload for one supervised BTC CLI run.

Parameters:

  • run_id (str)

  • mode (str)

  • command (tuple[str, ...])

  • manifest_path (Path)

  • stdout_log_path (Path)

  • stderr_log_path (Path)

  • output_csv_path (Path)

  • error_csv_path (Path)

  • executable_path (Path)

  • install_root (Path)

  • working_dir (Path)

  • copied_install (bool)

  • exit_code (int)

  • duration_sec (float)

  • report_template_path (Path | None)

  • uses_live_overlay (bool)

command: tuple[str, ...]
copied_install: bool
duration_sec: float
error_csv_path: Path
executable_path: Path
exit_code: int
install_root: Path
manifest_path: Path
mode: str
output_csv_path: Path
report_template_path: Path | None
run_id: str
stderr_log_path: Path
stdout_log_path: Path
uses_live_overlay: bool = False
working_dir: Path
class femic.pipeline.tipsy.BTCRuntimeDiscovery(executable_path, source)[source]

Bases: object

Resolved BTC executable discovery result.

Parameters:

  • executable_path (Path)

  • source (str)

executable_path: Path
source: str
class femic.pipeline.tipsy.BTCRuntimePreparation(executable_path, install_root, working_dir, staged_input_csv, copied_install, report_template_path, uses_live_overlay=False)[source]

Bases: object

Prepared BTC runtime layout for one supervised CLI run.

Parameters:

  • executable_path (Path)

  • install_root (Path)

  • working_dir (Path)

  • staged_input_csv (Path)

  • copied_install (bool)

  • report_template_path (Path | None)

  • uses_live_overlay (bool)

copied_install: bool
executable_path: Path
install_root: Path
report_template_path: Path | None
staged_input_csv: Path
uses_live_overlay: bool = False
working_dir: Path
class femic.pipeline.tipsy.TIPSYCandidateEvaluation(eligible, reason, species_map, leading_species, bec, max_vol, min_vol, operable_years, si_vri_iqrlo, si_spr_iqrlo, si_vri_med, si_spr_med, min_si)[source]

Bases: object

Eligibility outcome and derived metrics for one stratum+SI candidate.

Parameters:

  • eligible (bool)

  • reason (str | None)

  • species_map (Mapping[str, Any])

  • leading_species (str | None)

  • bec (str)

  • max_vol (float)

  • min_vol (float)

  • operable_years (float)

  • si_vri_iqrlo (float)

  • si_spr_iqrlo (float)

  • si_vri_med (float)

  • si_spr_med (float)

  • min_si (float | None)

bec: str
eligible: bool
leading_species: str | None
max_vol: float
min_si: float | None
min_vol: float
operable_years: float
reason: str | None
si_spr_iqrlo: float
si_spr_med: float
si_vri_iqrlo: float
si_vri_med: float
species_map: Mapping[str, Any]
class femic.pipeline.tipsy.TipsyInputOutputCoherence(coherent: 'bool', summary: 'str', expected_au_count: 'int', expected_table_count: 'int', observed_table_count: 'int')[source]

Bases: object

Parameters:

  • coherent (bool)

  • summary (str)

  • expected_au_count (int)

  • expected_table_count (int)

  • observed_table_count (int)

coherent: bool
expected_au_count: int
expected_table_count: int
observed_table_count: int
summary: str
femic.pipeline.tipsy.apply_btc_indicator_banks(*, template, indicator_bank_names)[source]

Append vetted indicator-bank columns to an existing BTC report template.

Parameters:

  • template (BTCCustomReportTemplate)

  • indicator_bank_names (Sequence[str])

Return type:

BTCCustomReportTemplate

femic.pipeline.tipsy.assess_tipsy_input_output_coherence(*, tipsy_input_excel_path, tipsy_output_path)[source]

Assess whether TIPSY input and output look structurally coherent.

Parameters:

  • tipsy_input_excel_path (str | Path)

  • tipsy_output_path (str | Path)

Return type:

TipsyInputOutputCoherence

femic.pipeline.tipsy.btc_indicator_bank_columns(name, *, options=None)[source]

Return the vetted column set for one optional BTC indicator bank.

Parameters:

  • name (str)

  • options (Mapping[str, Any] | None)

Return type:

tuple[BTCCustomReportColumn, …]

femic.pipeline.tipsy.btc_msyt_input_csv_path(*, tsa, input_root='data', filename_template='03_input-tsa{tsa}.csv')[source]

Build canonical per-TSA BTC MSYT.csv handoff path.

Parameters:

  • tsa (str)

  • input_root (str | Path)

  • filename_template (str)

Return type:

Path

femic.pipeline.tipsy.btc_report_template_preset(name)[source]

Return a vetted built-in BTC custom report template preset.

Parameters:

name (str)

Return type:

BTCCustomReportTemplate

femic.pipeline.tipsy.build_btc_cli_command(*, executable_path, mode, input_csv, output_csv, error_csv, extra_executable_args=())[source]

Build the concrete BTC CLI command for /TSR or /FLP execution.

Parameters:

  • executable_path (str | Path)

  • mode (str)

  • input_csv (str | Path)

  • output_csv (str | Path)

  • error_csv (str | Path)

  • extra_executable_args (Sequence[str | Path])

Return type:

list[str]

femic.pipeline.tipsy.build_btc_custom_report_template(*, name, source_template=None, columns=None, header_flags=None, icon_id=None, identifier=None, identifier_integer=None, report_type=None, output_format=None, border=None, header_height=None, footer_height=None)[source]

Build a BTC custom report template from a preset or existing template.

Parameters:

  • name (str)

  • source_template (BTCCustomReportTemplate | None)

  • columns (Sequence[BTCCustomReportColumn] | None)

  • header_flags (Mapping[str, str] | None)

  • icon_id (int | None)

  • identifier (str | None)

  • identifier_integer (bool | None)

  • report_type (str | None)

  • output_format (str | None)

  • border (int | None)

  • header_height (int | None)

  • footer_height (int | None)

Return type:

BTCCustomReportTemplate

femic.pipeline.tipsy.build_btc_msyt_input_table(*, tipsy_table, natural_tipsy_table=None, pd_module)[source]

Build BTC MSYT.csv input rows from TIPSY planted and natural-side payloads.

Parameters:

  • tipsy_table (Any)

  • natural_tipsy_table (Any | None)

  • pd_module (Any)

Return type:

Any

femic.pipeline.tipsy.build_tipsy_input_table(*, tipsy_params_for_tsa, tipsy_params_columns, pd_module, table_key='f')[source]

Build TIPSY input table rows from per-AU parameter payloads.

Parameters:

  • tipsy_params_for_tsa (Mapping[int, Mapping[str, Mapping[str, Any]]])

  • tipsy_params_columns (Sequence[str])

  • pd_module (Any)

  • table_key (str)

Return type:

Any

femic.pipeline.tipsy.build_tipsy_params_for_tsa(*, tsa, results_for_tsa, si_levels, vdyp_curves_smooth_tsa, vdyp_results_for_tsa, exclusion, tipsy_param_builder, vdyp_curve_events_path=None, append_jsonl_fn=None, min_operable_years=50.0, si_iqrlo_quantile=0.5, si_merge_enabled=True, si_merge_max_relative_gap=0.08, si_merge_max_window_nrmse=0.12, si_merge_min_common_ages=5, si_merge_age_min=30, si_merge_age_max=250, verbose=True, message_fn=<built-in function print>)[source]

Select eligible strata+SI combos and build TIPSY params for one TSA.

Parameters:

  • tsa (str)

  • results_for_tsa (Sequence[tuple[int, str, Mapping[str, Any]]])

  • si_levels (Sequence[str])

  • vdyp_curves_smooth_tsa (Any)

  • vdyp_results_for_tsa (Mapping[int, Mapping[str, Any]])

  • exclusion (Mapping[str, Any])

  • tipsy_param_builder (Any)

  • vdyp_curve_events_path (Any)

  • append_jsonl_fn (Any)

  • min_operable_years (float)

  • si_iqrlo_quantile (float)

  • si_merge_enabled (bool)

  • si_merge_max_relative_gap (float)

  • si_merge_max_window_nrmse (float)

  • si_merge_min_common_ages (int)

  • si_merge_age_min (int)

  • si_merge_age_max (int)

  • verbose (bool)

  • message_fn (Any)

Return type:

tuple[dict[tuple[str, str], int], dict[int, tuple[str, str]], dict[int, dict[str, dict[str, Any]]]]

femic.pipeline.tipsy.build_tipsy_warning_event(*, tsa, stratumi, sc, si_level, au, reason)[source]

Build standardized warning payload for TIPSY-input stage issues.

Parameters:

  • tsa (str)

  • stratumi (int)

  • sc (str)

  • si_level (str | None)

  • au (int | None)

  • reason (str)

Return type:

dict[str, Any]

femic.pipeline.tipsy.compute_file_sha256(path)[source]

Compute deterministic SHA256 digest for file content.

Parameters:

path (str | Path)

Return type:

str

femic.pipeline.tipsy.compute_vdyp_oaf1(vdyp_out)[source]

Compute OAF1 from mean VDYP % Stk values, handling malformed tables.

Parameters:

vdyp_out (Mapping[Any, Any])

Return type:

float

femic.pipeline.tipsy.compute_vdyp_site_index(vdyp_out, *, ndigits=1)[source]

Compute mean SI across VDYP output tables, rounded for TIPSY input.

Parameters:

  • vdyp_out (Mapping[Any, Any])

  • ndigits (int)

Return type:

float

femic.pipeline.tipsy.evaluate_tipsy_candidate(*, sc, vdyp_curve_df, result_si, exclusion, min_operable_years, si_iqrlo_quantile, siteprod_si_fallback_by_species=None)[source]

Evaluate whether a stratum+SI candidate is usable for TIPSY parameter generation.

Parameters:

  • sc (str)

  • vdyp_curve_df (Any)

  • result_si (Mapping[str, Any])

  • exclusion (Mapping[str, Any])

  • min_operable_years (float)

  • si_iqrlo_quantile (float)

  • siteprod_si_fallback_by_species (Mapping[str, float] | None)

Return type:

TIPSYCandidateEvaluation

femic.pipeline.tipsy.parse_btc_custom_report_template(template_path)[source]

Parse a BTC .rpt custom report file into a structured template.

Parameters:

template_path (str | Path)

Return type:

BTCCustomReportTemplate

femic.pipeline.tipsy.parse_btc_tsr_transposed_output(*, output_csv, pd_module)[source]

Parse unattended BTC /TSR transposed output into legacy long-curve rows.

Parameters:

  • output_csv (str | Path)

  • pd_module (Any)

Return type:

Any

femic.pipeline.tipsy.prepare_btc_runtime(*, executable_path, input_csv, scratch_root, mode, report_template=None, report_preset_name=None, indicator_bank_names=(), copy_install=False, prefer_user_overlay=False)[source]

Stage a writable BTC runtime root and input CSV for one run.

Parameters:

  • executable_path (str | Path)

  • input_csv (str | Path)

  • scratch_root (str | Path)

  • mode (str)

  • report_template (BTCCustomReportTemplate | str | Path | None)

  • report_preset_name (str | None)

  • indicator_bank_names (Sequence[str])

  • copy_install (bool)

  • prefer_user_overlay (bool)

Return type:

BTCRuntimePreparation

femic.pipeline.tipsy.probe_btc_indicator_banks(*, input_csv, indicator_bank_names, mode='TSR', executable_path=None, source_template=None, source_preset_name='tsr-unattended-default', copy_install=False, scratch_root=None, log_dir=PosixPath('tipsy_io/logs'), run_id_prefix='btc_bank_probe', env=None, compatibility_json=None, fallback_to_column_ratchet=True, variant_strategy='default', alias_overrides=None, attempt_timeout_seconds=6.0)[source]

Probe whole BTC indicator banks in single runs, with ratchet fallback.

Parameters:

  • input_csv (str | Path)

  • indicator_bank_names (Sequence[str])

  • mode (str)

  • executable_path (str | Path | None)

  • source_template (BTCCustomReportTemplate | str | Path | None)

  • source_preset_name (str | None)

  • copy_install (bool)

  • scratch_root (str | Path | None)

  • log_dir (str | Path)

  • run_id_prefix (str)

  • env (Mapping[str, str] | None)

  • compatibility_json (str | Path | None)

  • fallback_to_column_ratchet (bool)

  • variant_strategy (str)

  • alias_overrides (Mapping[str, Sequence[str]] | None)

  • attempt_timeout_seconds (float)

Return type:

tuple[list[BTCColumnProbeResult], BTCCustomReportTemplate]

femic.pipeline.tipsy.probe_btc_report_columns(*, input_csv, candidate_tokens, mode='TSR', executable_path=None, source_template=None, source_preset_name='tsr-unattended-default', copy_install=False, scratch_root=None, log_dir=PosixPath('tipsy_io/logs'), run_id_prefix='btc_probe', env=None, compatibility_json=None, variant_strategy='default', alias_overrides=None, attempt_timeout_seconds=6.0)[source]

Probe BTC report-column compatibility one candidate token at a time.

Starts from the current safe template, adds one candidate token, and keeps only the additions that survive a real BTC run. Failures are recorded but not retained in the rolling accepted template.

Parameters:

  • input_csv (str | Path)

  • candidate_tokens (Sequence[str | BTCCustomReportColumn])

  • mode (str)

  • executable_path (str | Path | None)

  • source_template (BTCCustomReportTemplate | str | Path | None)

  • source_preset_name (str | None)

  • copy_install (bool)

  • scratch_root (str | Path | None)

  • log_dir (str | Path)

  • run_id_prefix (str)

  • env (Mapping[str, str] | None)

  • compatibility_json (str | Path | None)

  • variant_strategy (str)

  • alias_overrides (Mapping[str, Sequence[str]] | None)

  • attempt_timeout_seconds (float)

Return type:

tuple[list[BTCColumnProbeResult], BTCCustomReportTemplate]

femic.pipeline.tipsy.resolve_btc_executable(*, executable_path=None, env=None)[source]

Resolve the BatchTIPSY BTC executable path on Windows-first hosts.

Parameters:

  • executable_path (str | Path | None)

  • env (Mapping[str, str] | None)

Return type:

BTCRuntimeDiscovery

femic.pipeline.tipsy.run_btc_cli(*, input_csv, mode='TSR', output_csv=None, error_csv=None, executable_path=None, report_template=None, report_preset_name=None, indicator_bank_names=(), copy_install=None, scratch_root=None, log_dir=PosixPath('tipsy_io/logs'), run_id=None, env=None, extra_executable_args=(), timeout_seconds=None)[source]

Run BTC /TSR or /FLP in a supervised writable scratch environment.

Parameters:

  • input_csv (str | Path)

  • mode (str)

  • output_csv (str | Path | None)

  • error_csv (str | Path | None)

  • executable_path (str | Path | None)

  • report_template (BTCCustomReportTemplate | str | Path | None)

  • report_preset_name (str | None)

  • indicator_bank_names (Sequence[str])

  • copy_install (bool | None)

  • scratch_root (str | Path | None)

  • log_dir (str | Path)

  • run_id (str | None)

  • env (Mapping[str, str] | None)

  • extra_executable_args (Sequence[str | Path])

  • timeout_seconds (float | None)

Return type:

BTCRunResult

femic.pipeline.tipsy.tipsy_output_input_fingerprint_path(*, tipsy_output_path)[source]

Return sidecar path storing the canonical input fingerprint for TIPSY output.

Parameters:

tipsy_output_path (str | Path)

Return type:

Path

femic.pipeline.tipsy.tipsy_params_excel_path(*, tsa, tipsy_params_path_prefix)[source]

Build legacy per-TSA TIPSY parameter workbook path.

Parameters:

  • tsa (str)

  • tipsy_params_path_prefix (str | Path)

Return type:

Path

femic.pipeline.tipsy.tipsy_stage_output_paths(*, tsa, output_root='data')[source]

Build legacy 01b per-TSA output CSV paths.

Parameters:

  • tsa (str)

  • output_root (str | Path)

Return type:

tuple[Path, Path]

femic.pipeline.tipsy.validate_tipsy_output_is_fresh(*, tipsy_input_excel_path=None, btc_input_csv_path=None, tipsy_output_path, allow_stale=False, strict_timestamp_mismatch=False)[source]

Fail fast when BatchTIPSY output is stale against canonical BTC CSV input.

Parameters:

  • tipsy_input_excel_path (str | Path | None)

  • btc_input_csv_path (str | Path | None)

  • tipsy_output_path (str | Path)

  • allow_stale (bool)

  • strict_timestamp_mismatch (bool)

Return type:

None

femic.pipeline.tipsy.write_btc_custom_report_template(*, output_path, template)[source]

Write a BTC custom report template to disk.

Parameters:

  • output_path (str | Path)

  • template (BTCCustomReportTemplate)

Return type:

Path

femic.pipeline.tipsy.write_btc_msyt_input_csv(*, btc_msyt_table, tsa, output_root='data', filename_template='03_input-tsa{tsa}.csv')[source]

Write the BTC canonical MSYT.csv handoff for one TSA.

Parameters:

  • btc_msyt_table (Any)

  • tsa (str)

  • output_root (str | Path)

  • filename_template (str)

Return type:

Path

femic.pipeline.tipsy.write_tipsy_input_exports(*, tipsy_table, tsa, tipsy_params_path_prefix)[source]

Write the human-readable TIPSY parameter workbook for one TSA.

Parameters:

  • tipsy_table (Any)

  • tsa (str)

  • tipsy_params_path_prefix (str)

Return type:

str

femic.pipeline.tipsy.write_tipsy_output_input_fingerprint(*, btc_input_csv_path, tipsy_output_path)[source]

Persist canonical input CSV SHA256 used for the accepted BatchTIPSY output.

Parameters:

  • btc_input_csv_path (str | Path | None)

  • tipsy_output_path (str | Path)

Return type:

Path | None