The library path

WS4 is two ways in, not one. The other notebooks here drive the ws4 command; this one never leaves Python: it compiles the sample estate, opens the model as data, simulates schedules in memory, runs a search, and reads the result back as frames. That is the path to take when a study has to be automated — a sweep, a custom diagnostic, WS4 inside something else.

It complements two pages rather than replacing them: docs/getting-started/python-api.rst walks the same steps one call at a time, and docs/howto/automate-a-study.rst collects the patterns. What is here is a single session with its real output, start to finish.

This one is the library path end to end: no ws4 command is run anywhere.

[1]:
from pathlib import Path

import pandas as pd
from ws4.build.compiler import build_canonical_dataset

work = Path("tmp/examples/library")
work.mkdir(parents=True, exist_ok=True)
dataset = build_canonical_dataset(
    "examples/estate/recipe.xml", "examples/estate/fragments.csv", work / "compiled"
)
print(dataset)
tmp/examples/library/compiled

Open the model as data

The compiled dataset is the contract between ingestion and execution: a directory of Parquet tables plus a manifest. Model holds one frame per table, which is what makes a diagnostic a one-liner instead of a file-format problem.

[2]:
from ws4.model.api import Model

model = Model.from_dataset(dataset)
counts = pd.DataFrame(
    [{"table": name, "rows": len(model.table(name))} for name in sorted(model.tables)]
)
print(counts.to_string(index=False))
        table  rows
     accounts     4
       blocks    64
       curves    12
     features     4
    fragments    64
       groups    64
     messages     1
     products     2
protoaccounts     4
       strata     2
       tracks     2
   treatments     2

Simulate in memory: no run directory

Cutting the eight western blocks at offset 0 opens a patch and moves volume; leaving the estate alone does not. Both are simulated here and the difference is read straight out of the returned frame.

[3]:
from ws4.sim.engine import Simulator

years = 20
empty = pd.DataFrame({"block_id": [], "treatment": [], "offset": []})
cut = pd.DataFrame({"block_id": [1, 2, 9, 10, 57, 58, 63, 64], "treatment": ["CC"] * 8, "offset": [0] * 8})
simulator = Simulator.from_dataset(dataset)
before = simulator.simulate(empty, years)
after = simulator.simulate(cut, years)
both = before.merge(after, on=["offset", "account"], suffixes=("_untreated", "_cut"))
print(both[both.account == "feature.Seral.regen"].to_string(index=False))
 offset             account  value_untreated  value_cut
     -1 feature.Seral.regen              0.0        0.0
      0 feature.Seral.regen              0.0       81.9
      1 feature.Seral.regen              0.0       81.9
      2 feature.Seral.regen              0.0       81.9
      3 feature.Seral.regen              0.0       81.9
      4 feature.Seral.regen              0.0       81.9
      5 feature.Seral.regen              0.0       81.9
      6 feature.Seral.regen              0.0       81.9
      7 feature.Seral.regen              0.0       81.9
      8 feature.Seral.regen              0.0       81.9
      9 feature.Seral.regen              0.0       81.9
     10 feature.Seral.regen              0.0       81.9
     11 feature.Seral.regen              0.0       81.9
     12 feature.Seral.regen              0.0       81.9
     13 feature.Seral.regen              0.0       81.9
     14 feature.Seral.regen              0.0       81.9
     15 feature.Seral.regen              0.0       81.9
     16 feature.Seral.regen              0.0       81.9
     17 feature.Seral.regen              0.0       81.9
     18 feature.Seral.regen              0.0       81.9
     19 feature.Seral.regen              0.0       81.9

The frame that comes back is the same one the command line writes to account_values.csv; here it never reaches the disk. Anything a frame can do is therefore available mid-study — a pivot, a plot, an assertion that fails a sweep early.

Search, then read the result as data

A target is a floor or a ceiling on an account in a year. This one asks for 5 000 units harvested in the first ten-year period, and the search is run with the estate’s topology and its patch account so that spatial structure is priced rather than assumed.

[4]:
(work / "targets.yaml").write_text(
    "targets:\n"
    "  - account: product.Harvested.CC\n"
    "    period: 1\n"
    "    minimum: 5000\n"
    "    period_width: 10\n"
)
from ws4.opt.engine import OptimizeConfig, Optimizer

optimizer = Optimizer.from_dataset(
    dataset,
    OptimizeConfig(
        years=100,
        iterations=2000,
        seed=1,
        threads=1,
        targets_path=str(work / "targets.yaml"),
        topology_path="examples/estate/topology.csv",
        patch_accounts_path="examples/estate/patches.yaml",
    ),
)
summary = optimizer.run(work / "run")
print(pd.Series(summary)[["incumbent_objective", "blocks_scheduled", "iterations", "seed", "status"]].to_string())
incumbent_objective     0.0
blocks_scheduled          8
iterations             2000
seed                      1
status                   ok

run is the whole of a command-line run as one call: it searches, writes the run directory, and returns the summary it wrote. The target table says whether the floor was met, and what it reports is the account summed over the target’s period — so it can be checked against the values themselves rather than trusted.

[5]:
targets = pd.read_csv(work / "run" / "report_targets.csv")
print(targets.to_string(index=False))

values = pd.read_csv(work / "run" / "account_values.csv")
harvest = values[values.account == "product.Harvested.CC"]
period_total = harvest[harvest.offset < 10]["value"].sum()
print()
print(f"the same period, read from the values: {period_total:,.2f}")
             account  period  year       value  minimum  maximum  raw_penalty  penalty  scale normalization  priority  kind status
product.Harvested.CC       1    10 5549.857143     5000      NaN          0.0      0.0    1.0          none         1 level    met

the same period, read from the values: 5,549.86

Two schedules, one frame each — checked against the run

The run directory’s values are the simulator’s own; simulating the schedule it recorded has to give the same numbers. That is the kind of check a study wants to make for itself, and the library makes it short.

[6]:
from ws4.sim.engine import Simulator

incumbent = pd.read_csv(work / "run" / "schedule.csv")
replay = Simulator.from_dataset(dataset).simulate(incumbent, years=100)
account = "product.Harvested.CC"
reported = values[values.account == account].sort_values("offset")["value"].to_numpy()
mine = replay[replay.account == account].sort_values("offset")["value"].to_numpy()
print(f"{account}: {len(reported)} periods, matches within 1e-9: {bool(abs(reported - mine).max() < 1e-9)}")
product.Harvested.CC: 14 periods, matches within 1e-9: True

Where the files land

Everything above wrote exactly one directory of inputs and one run directory, and nothing else: the in-memory schedules were never files.

This notebook

The equivalent command

build_canonical_dataset(...)

ws4 ingest --recipe ... --fragments ... --out ...

Simulator.from_dataset(...).simulate(...)

ws4 simulate --dataset ... --schedule ... --out ...

Optimizer.from_dataset(...).run(...)

ws4 optimize --dataset ... --targets ... --out ...

The commands call this same code, so the two paths cannot drift apart in what they compute — only in what they leave on disk.

Where to look next

  • docs/getting-started/python-api.rst is the step-by-step library walkthrough, including the in-memory evaluator and a hand-driven search.

  • docs/howto/automate-a-study.rst collects the patterns: sweeps over seeds and targets, custom diagnostics, and resuming a command-line run in Python.

  • docs/reference/optimizer.rst specifies the objective, the operators and the islands.

  • 020_even_flow and 060_parameter_sweep are the other two notebooks that use the library.