ForestModelXML recipe
WS4 reads the established ForestModelXML model format as its recipe, and compiles it with a fragment inventory into the compiled matrix dataset (see notes/adr/0011 and notes/design/ws4-matrix-builder-and-interop.md). The compiler is a matrix builder.
What the parser reads
ws4.io.forest_model_xml.parse_recipe reads:
the root
ForestModelattributeshorizon,year, andmatch;curveelements and their points;defineelements, split into field-to-column bindings and constants;the
inputelement’sblock,area, andageexpressions;the
outputelement;selectblocks, each with a statement, features, products, and a track with treatments;treatment
minage,maxage,retain, andtransitionassigns;attribute sources: a
curvereference or anexpression.
Select statements are evaluated by ws4.io.expressions: comparisons (eq, ne, lt, le,
gt, ge, in), the boolean operators (and, or, not), string concatenation and
arithmetic, parentheses, and the functions startswith, endswith, substring, int,
Number, length, lastindexof, if, lookupTable, curveId, attribute, and
operable. An expression the evaluator cannot parse falls back to a term matcher that skips the
terms it cannot read, so a compile is best-effort for models that use a function the evaluator lacks.
Compilation
The matrix is a Model II model. State variables (themes) define a stratum; a track is a unique trajectory over time between two treatments; and a treatment resets one or more state variables, which moves a block to the stratum of the new values and therefore onto that stratum’s track.
ws4.build.compiler.build_canonical_dataset builds the matrix with a Model II expansion (ws4.build.expansion):
the state variables are the recipe’s non-constant define fields;
each fragment’s state is read from its columns, and the reachable strata are enumerated from the inventory and the treatment transitions, to a fixed point;
each stratum becomes a track (
tracknames), named by its state values;a stratum is governed by every select whose statement matches it, and features, products, and treatments accumulate across those selects;
a treatment’s
new_trackis the track of the stratum reached by applying its transition (a literal, a constant, a field reference such asau = auf, or a concatenation such as'thn_'+au);a
<retention factor="RET">rule splits a fragment’s area into a retained portion (the factor, moved to the assigned state, for exampleIFM = unmanaged) and a base portion (the remainder); a<features>element nested inside theretentiondescribes the retained fraction and attaches to the stratum the rule moves the area to;an
expression-sourced attribute resolves to a curve name, or, when it evaluates to a number, to a generated constant curve;attribute('label')resolves to a label already emitted for the same stratum, so an attribute can be derived from another (for exampleoperable(attribute('…')));a block with fragments or portions in more than one stratum is emitted once per portion (
blocks).
Validation
The builder is validated against several reference compiled matrices. On each, it reproduces the structural counts exactly:
Model |
tracks / strata |
treatments |
blocks |
|---|---|---|---|
a 218-fragment instance (two variants) |
351 / 351 |
263 |
399 |
the same instance, intensive variants |
732 / 732 and 553 / 553 |
963 and 605 |
427 and 427 |
a 421,945-fragment instance |
524 / 524 |
210 |
(not published) |
On the MKRF canonical rebuild recipe and its fragments (1763 records), the builder reproduces the
reference matrix exactly: tracks 7048/7048, strata 7048/7048, treatments 10014/10014, blocks
3183/3183, features 104311/104311, products 420588/420588, and protoaccounts 91/91 (row counts and
label sets; scripts/acceptance.py). The reference is the model’s own canonical tracks/ directory.
A matrix from a different model variant is not a valid comparison.
Legacy ingestion
ws4.build.legacy.ingest_legacy_matrix reads an existing matrix directory plus a fragments table, maps the legacy column names to the schema, and records the mapping in the manifest.
ws4 ingest --matrix legacy/matrix --fragments fragments.csv --out build/compiled
Reference output
A reference compiled matrix for a real, moderately complex model (a UBC research forest) sets the target for the builder:
tracknamesandstrata: 2116 tracks, one per development stratum;treatments: 2024 rows (treatments expand across the tracks they apply to);blocks: 3183 rows, with a block split into a managed and an unmanaged portion;accounts: 60 accounts;messages: no rows (a clean compile).
The reference is the model’s own Tracks/ directory. Compare a builder run against the matrix from
the same recipe and fragments; a matrix from a different model variant is not a valid comparison.
Known gaps
The following remain and are tracked in the backlog (notes/backlog.md):
the wider expression function library (functions beyond those listed above);
successionandproducesemantics beyond treatment transitions;legacy-ingestion layouts beyond the known columns, and the road model;
a replay of a run validated against a reference schedule.
API
- ws4.io.forest_model_xml.parse_recipe(path: Path) Recipe[source]
Parse and validate a ForestModelXML recipe.
- Parameters:
path (Path) – Path to the recipe.
- Returns:
Recipe – The parsed recipe.
- Raises:
ForestModelXMLError – When the recipe violates the required structure.
- ws4.build.compiler.build_canonical_dataset(xml_path: Path, fragments_path: Path, out_dir: Path) Path[source]
Compile a ForestModelXML recipe and a fragment inventory into a matrix dataset.
- Parameters:
xml_path (Path) – Path to the ForestModelXML recipe.
fragments_path (Path) – Path to the fragment inventory, as CSV or Parquet.
out_dir (Path) – Directory to write the compiled dataset into.
- Returns:
Path – The dataset directory.
- ws4.build.legacy.ingest_legacy_matrix(matrix_dir: Path, fragments_path: Path, out_dir: Path) Path[source]
Ingest a legacy matrix dataset into the WS4 schema.
- Parameters:
matrix_dir (Path) – Directory holding the legacy matrix CSV files.
fragments_path (Path) – Path to the fragment inventory (CSV or Parquet), with the block, area, and age columns.
out_dir (Path) – Directory to write the ingested dataset into.
- Returns:
Path – The dataset directory.