{ "cells": [ { "cell_type": "markdown", "id": "de4a9e80", "metadata": {}, "source": [ "# The library path\n", "\n", "WS4 is two ways in, not one. The other notebooks here drive the `ws4` command; this one never leaves\n", "Python: it compiles the sample estate, opens the model as data, simulates schedules in memory, runs a\n", "search, and reads the result back as frames. That is the path to take when a study has to be automated\n", "— a sweep, a custom diagnostic, WS4 inside something else.\n", "\n", "It complements two pages rather than replacing them: `docs/getting-started/python-api.rst` walks the\n", "same steps one call at a time, and `docs/howto/automate-a-study.rst` collects the patterns. What is\n", "here is a single session with its real output, start to finish.\n", "\n", "This one is the **library** path end to end: no `ws4` command is run anywhere.\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "2f8ff6df", "metadata": { "execution": { "iopub.execute_input": "2026-10-18T05:00:46.915629Z", "iopub.status.busy": "2026-10-18T05:00:46.915202Z", "iopub.status.idle": "2026-10-18T05:00:47.450664Z", "shell.execute_reply": "2026-10-18T05:00:47.449358Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tmp/examples/library/compiled\n" ] } ], "source": [ "from pathlib import Path\n", "\n", "import pandas as pd\n", "from ws4.build.compiler import build_canonical_dataset\n", "\n", "work = Path(\"tmp/examples/library\")\n", "work.mkdir(parents=True, exist_ok=True)\n", "dataset = build_canonical_dataset(\n", " \"examples/estate/recipe.xml\", \"examples/estate/fragments.csv\", work / \"compiled\"\n", ")\n", "print(dataset)" ] }, { "cell_type": "markdown", "id": "e1711440", "metadata": {}, "source": [ "## Open the model as data\n", "\n", "The compiled dataset is the contract between ingestion and execution: a directory of Parquet tables\n", "plus a manifest. `Model` holds one frame per table, which is what makes a diagnostic a one-liner\n", "instead of a file-format problem." ] }, { "cell_type": "code", "execution_count": 2, "id": "9d80f7f5", "metadata": { "execution": { "iopub.execute_input": "2026-10-18T05:00:47.454179Z", "iopub.status.busy": "2026-10-18T05:00:47.453964Z", "iopub.status.idle": "2026-10-18T05:00:47.503833Z", "shell.execute_reply": "2026-10-18T05:00:47.502474Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " table rows\n", " accounts 4\n", " blocks 64\n", " curves 12\n", " features 4\n", " fragments 64\n", " groups 64\n", " messages 1\n", " products 2\n", "protoaccounts 4\n", " strata 2\n", " tracks 2\n", " treatments 2\n" ] } ], "source": [ "from ws4.model.api import Model\n", "\n", "model = Model.from_dataset(dataset)\n", "counts = pd.DataFrame(\n", " [{\"table\": name, \"rows\": len(model.table(name))} for name in sorted(model.tables)]\n", ")\n", "print(counts.to_string(index=False))" ] }, { "cell_type": "markdown", "id": "69623494", "metadata": {}, "source": [ "## Simulate in memory: no run directory\n", "\n", "Cutting the eight western blocks at offset 0 opens a patch and moves volume; leaving the estate alone\n", "does not. Both are simulated here and the difference is read straight out of the returned frame.\n" ] }, { "cell_type": "code", "execution_count": 3, "id": "34b5d30a", "metadata": { "execution": { "iopub.execute_input": "2026-10-18T05:00:47.506879Z", "iopub.status.busy": "2026-10-18T05:00:47.506687Z", "iopub.status.idle": "2026-10-18T05:00:47.841195Z", "shell.execute_reply": "2026-10-18T05:00:47.839465Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " offset account value_untreated value_cut\n", " -1 feature.Seral.regen 0.0 0.0\n", " 0 feature.Seral.regen 0.0 81.9\n", " 1 feature.Seral.regen 0.0 81.9\n", " 2 feature.Seral.regen 0.0 81.9\n", " 3 feature.Seral.regen 0.0 81.9\n", " 4 feature.Seral.regen 0.0 81.9\n", " 5 feature.Seral.regen 0.0 81.9\n", " 6 feature.Seral.regen 0.0 81.9\n", " 7 feature.Seral.regen 0.0 81.9\n", " 8 feature.Seral.regen 0.0 81.9\n", " 9 feature.Seral.regen 0.0 81.9\n", " 10 feature.Seral.regen 0.0 81.9\n", " 11 feature.Seral.regen 0.0 81.9\n", " 12 feature.Seral.regen 0.0 81.9\n", " 13 feature.Seral.regen 0.0 81.9\n", " 14 feature.Seral.regen 0.0 81.9\n", " 15 feature.Seral.regen 0.0 81.9\n", " 16 feature.Seral.regen 0.0 81.9\n", " 17 feature.Seral.regen 0.0 81.9\n", " 18 feature.Seral.regen 0.0 81.9\n", " 19 feature.Seral.regen 0.0 81.9\n" ] } ], "source": [ "from ws4.sim.engine import Simulator\n", "\n", "years = 20\n", "empty = pd.DataFrame({\"block_id\": [], \"treatment\": [], \"offset\": []})\n", "cut = pd.DataFrame({\"block_id\": [1, 2, 9, 10, 57, 58, 63, 64], \"treatment\": [\"CC\"] * 8, \"offset\": [0] * 8})\n", "simulator = Simulator.from_dataset(dataset)\n", "before = simulator.simulate(empty, years)\n", "after = simulator.simulate(cut, years)\n", "both = before.merge(after, on=[\"offset\", \"account\"], suffixes=(\"_untreated\", \"_cut\"))\n", "print(both[both.account == \"feature.Seral.regen\"].to_string(index=False))" ] }, { "cell_type": "markdown", "id": "898bc958", "metadata": {}, "source": [ "The frame that comes back is the same one the command line writes to `account_values.csv`; here it\n", "never reaches the disk. Anything a frame can do is therefore available mid-study — a pivot, a plot, an\n", "assertion that fails a sweep early.\n" ] }, { "cell_type": "markdown", "id": "fc3661ff", "metadata": {}, "source": [ "## Search, then read the result as data\n", "\n", "A target is a floor or a ceiling on an account in a year. This one asks for 5 000 units harvested in\n", "the first ten-year period, and the search is run with the estate's topology and its patch account so\n", "that spatial structure is priced rather than assumed.\n" ] }, { "cell_type": "code", "execution_count": 4, "id": "364299b3", "metadata": { "execution": { "iopub.execute_input": "2026-10-18T05:00:47.844539Z", "iopub.status.busy": "2026-10-18T05:00:47.844320Z", "iopub.status.idle": "2026-10-18T05:00:50.015491Z", "shell.execute_reply": "2026-10-18T05:00:50.013798Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "incumbent_objective 0.0\n", "blocks_scheduled 8\n", "iterations 2000\n", "seed 1\n", "status ok\n" ] } ], "source": [ "(work / \"targets.yaml\").write_text(\n", " \"targets:\\n\"\n", " \" - account: product.Harvested.CC\\n\"\n", " \" period: 1\\n\"\n", " \" minimum: 5000\\n\"\n", " \" period_width: 10\\n\"\n", ")\n", "from ws4.opt.engine import OptimizeConfig, Optimizer\n", "\n", "optimizer = Optimizer.from_dataset(\n", " dataset,\n", " OptimizeConfig(\n", " years=100,\n", " iterations=2000,\n", " seed=1,\n", " threads=1,\n", " targets_path=str(work / \"targets.yaml\"),\n", " topology_path=\"examples/estate/topology.csv\",\n", " patch_accounts_path=\"examples/estate/patches.yaml\",\n", " ),\n", ")\n", "summary = optimizer.run(work / \"run\")\n", "print(pd.Series(summary)[[\"incumbent_objective\", \"blocks_scheduled\", \"iterations\", \"seed\", \"status\"]].to_string())" ] }, { "cell_type": "markdown", "id": "34a17c43", "metadata": {}, "source": [ "`run` is the whole of a command-line run as one call: it searches, writes the run directory, and\n", "returns the summary it wrote. The target table says whether the floor was met, and what it reports is\n", "the account summed over the target's period — so it can be checked against the values themselves\n", "rather than trusted.\n" ] }, { "cell_type": "code", "execution_count": 5, "id": "975161a1", "metadata": { "execution": { "iopub.execute_input": "2026-10-18T05:00:50.019260Z", "iopub.status.busy": "2026-10-18T05:00:50.018877Z", "iopub.status.idle": "2026-10-18T05:00:50.029340Z", "shell.execute_reply": "2026-10-18T05:00:50.028114Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " account period year value minimum maximum raw_penalty penalty scale normalization priority kind status\n", "product.Harvested.CC 1 10 5549.857143 5000 NaN 0.0 0.0 1.0 none 1 level met\n", "\n", "the same period, read from the values: 5,549.86\n" ] } ], "source": [ "targets = pd.read_csv(work / \"run\" / \"report_targets.csv\")\n", "print(targets.to_string(index=False))\n", "\n", "values = pd.read_csv(work / \"run\" / \"account_values.csv\")\n", "harvest = values[values.account == \"product.Harvested.CC\"]\n", "period_total = harvest[harvest.offset < 10][\"value\"].sum()\n", "print()\n", "print(f\"the same period, read from the values: {period_total:,.2f}\")" ] }, { "cell_type": "markdown", "id": "40fff5ae", "metadata": {}, "source": [ "## Two schedules, one frame each — checked against the run\n", "\n", "The run directory's values are the simulator's own; simulating the schedule it recorded has to give the\n", "same numbers. That is the kind of check a study wants to make for itself, and the library makes it\n", "short.\n" ] }, { "cell_type": "code", "execution_count": 6, "id": "cf2af334", "metadata": { "execution": { "iopub.execute_input": "2026-10-18T05:00:50.032541Z", "iopub.status.busy": "2026-10-18T05:00:50.032360Z", "iopub.status.idle": "2026-10-18T05:00:50.169890Z", "shell.execute_reply": "2026-10-18T05:00:50.168609Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "product.Harvested.CC: 14 periods, matches within 1e-9: True\n" ] } ], "source": [ "from ws4.sim.engine import Simulator\n", "\n", "incumbent = pd.read_csv(work / \"run\" / \"schedule.csv\")\n", "replay = Simulator.from_dataset(dataset).simulate(incumbent, years=100)\n", "account = \"product.Harvested.CC\"\n", "reported = values[values.account == account].sort_values(\"offset\")[\"value\"].to_numpy()\n", "mine = replay[replay.account == account].sort_values(\"offset\")[\"value\"].to_numpy()\n", "print(f\"{account}: {len(reported)} periods, matches within 1e-9: {bool(abs(reported - mine).max() < 1e-9)}\")" ] }, { "cell_type": "markdown", "id": "04fcc113", "metadata": {}, "source": [ "## Where the files land\n", "\n", "Everything above wrote exactly one directory of inputs and one run directory, and nothing else: the\n", "in-memory schedules were never files.\n", "\n", "| This notebook | The equivalent command |\n", "| --- | --- |\n", "| `build_canonical_dataset(...)` | `ws4 ingest --recipe ... --fragments ... --out ...` |\n", "| `Simulator.from_dataset(...).simulate(...)` | `ws4 simulate --dataset ... --schedule ... --out ...` |\n", "| `Optimizer.from_dataset(...).run(...)` | `ws4 optimize --dataset ... --targets ... --out ...` |\n", "\n", "The commands call this same code, so the two paths cannot drift apart in what they compute — only in\n", "what they leave on disk.\n" ] }, { "cell_type": "markdown", "id": "e1c172c2", "metadata": {}, "source": [ "## Where to look next\n", "\n", "* `docs/getting-started/python-api.rst` is the step-by-step library walkthrough, including the\n", " in-memory evaluator and a hand-driven search.\n", "* `docs/howto/automate-a-study.rst` collects the patterns: sweeps over seeds and targets, custom\n", " diagnostics, and resuming a command-line run in Python.\n", "* `docs/reference/optimizer.rst` specifies the objective, the operators and the islands.\n", "* `020_even_flow` and `060_parameter_sweep` are the other two notebooks that use the library.\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "python3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }