Automate a study
The command line is the fastest way to run one study. The Python API is the way to run many: sweeps over seeds, targets, and weights; custom diagnostics mid-search; results collected into a frame; WS4 embedded in something else. This guide is the set of patterns that make that practical.
For a worked session rather than a list of patterns, The library path runs the whole of one in the gallery.
Prerequisites
The API walkthrough read once (Quickstart: the Python API).
A compiled dataset and a targets file you are happy with.
Patterns
1. Run many configurations and collect the summaries. Every run writes the same files, so a sweep is a loop:
import pandas as pd
from pathlib import Path
from ws4.opt.engine import OptimizeConfig, Optimizer
rows = []
for seed in range(1, 11):
out = Path(f"tmp/study/seed-{seed}")
optimizer = Optimizer.from_dataset(
dataset,
OptimizeConfig(
years=100,
iterations=5000,
seed=seed,
targets_path="tmp/howto/targets.yaml",
starts=1,
),
)
summary = optimizer.run(out)
rows.append({"seed": seed, "objective": summary["incumbent_objective"],
"blocks": summary["blocks_scheduled"]})
print(pd.DataFrame(rows).describe())
The objective spread across seeds tells you whether a single run is worth reporting.
2. Build targets in memory. A sweep over a target value does not need a file per case:
from ws4.opt.objective import Target
for floor in (20_000, 30_000, 40_000):
targets = [
Target("product.Harvested.CC", period * 10, minimum=floor, period_width=10)
for period in range(10)
]
evaluator = optimizer.delta_evaluator(targets)
# ... drive a solver, or hand the evaluator to your own loop
3. Price a move yourself. The evaluator holds the running totals and returns the exact change a move would make, without re-simulating. That is what makes a custom acceptance rule, a diagnostic, or a different solver cheap:
import random
from ws4.opt.solvers import solve_sa_delta
candidates = optimizer.candidates()
evaluator = optimizer.delta_evaluator(targets)
best, objective = solve_sa_delta(
evaluator,
random.Random(1),
iterations=20_000,
temperature=1000.0,
cooling=0.99,
candidates=candidates,
registry=optimizer.registry,
)
Change random.Random(1) for any random source, and swap solve_sa_delta for solve_sa,
solve_tabu, or solve_ils to try another metaheuristic on the same problem.
4. Resume, warm-start, and watch. solve_sa_delta and its siblings take an initial
schedule, which is how a study fragments a long run — prime, then add a term, then continue:
primed, _ = solve_sa_delta(evaluator, rng, iterations=20_000, ..., initial=None)
even, _ = solve_sa_delta(evaluator_with_flow, rng, iterations=20_000, ..., initial=primed)
5. Scale out. A single search is one process. Two configurations scale:
Setting |
What it does |
|---|---|
|
Run independent multi-starts across worker processes and keep the best. Reproducible: the parallel path returns what the serial path returns for the same seed. |
|
Run the island model: several islands in parallel, migrating the global elite between episodes. The result is deterministic for a given seed. |
optimizer = Optimizer.from_dataset(
dataset,
OptimizeConfig(years=100, iterations=3000, islands=8, episodes=2, workers=8,
targets_path="tmp/howto/targets.yaml", seed=1),
)
summary = optimizer.run(Path("tmp/study/islands"))
Threads do not help the numeric core: the curve evaluation holds the GIL and is already close to the NumPy floor (notes/performance.md in the repository records the measurement). Process-level parallelism is the lever.
6. Read the run back. The model and run outputs are ordinary files and frames, so a study’s results go straight into analysis:
from ws4.model.api import Model
model = Model.from_dataset(dataset)
schedule = pd.read_csv("tmp/study/islands/schedule.csv")
values = pd.read_csv("tmp/study/islands/account_values.csv")
report = pd.read_csv("tmp/study/islands/report_targets.csv")
print(report.groupby("status").size())
Troubleshooting
The API returns nothing useful for account values — the run’s own files are the reliable source:
account_values.csv,report_targets.csv,report_accounts.csv.A sweep is slower than the CLI — the CLI sets threads for BLAS and the search; a script should set
threadsinOptimizeConfigthe same way it does, or run withworkers/islandsand keep each process single-threaded.Results differ between machines — the search is deterministic for a given seed and the same code path; a different
workersorislandssetting is a different search, not a different answer to the same question. Pin those in a study’s configuration and record them next to the results.