Synthetic Datasets
==================

FHOPS ships with a small library of synthetic scenarios that mirror the structure of our teaching
and benchmarking datasets. Each bundle is a self-contained directory under
``examples/synthetic/`` with a ``scenario.yaml`` and the CSV tables that back it. The bundles are
generated by :func:`fhops.scenario.synthetic.generate_random_dataset` so they follow the same data
contract validation as user-supplied scenarios.

Reference bundles
-----------------

Three ready-to-use bundles are versioned with the project. The table below summarises their scale;
see ``examples/synthetic/metadata.yaml`` for the full metadata (random seeds, blackout windows, and
shift calendars).

.. list-table:: Reference bundle statistics
   :header-rows: 1
   :widths: 15 12 12 12 12 12 25

   * - Bundle
     - Blocks
     - Machines
     - Landings
     - Days
     - Shifts/day
     - Seed / Notes
   * - ``small``
     - 4
     - 2
     - 1
     - 6
     - 1
     - Seed ``101`` — single-shift schedule without blackouts.
   * - ``medium``
     - 8
     - 4
     - 2
     - 12
     - 1
     - Seed ``202`` — includes short blackout windows for downtime realism.
   * - ``large``
     - 16
     - 6
     - 3
     - 18
     - 2
     - Seed ``303`` — two-shift calendar with extended blackout periods.

The example README (``examples/synthetic/README.md``) gives a short overview and links back to the
metadata file if you need exact counts or blackout windows for reproducibility.

Block enrichment, crews, and blackouts
--------------------------------------

Each bundle ships with richer signals to support evaluation/benchmarking:

* ``blocks.csv`` now contains ``terrain`` and ``prescription`` columns sampled from tier-aware pools
  (e.g., ``gentle``/``mixed`` for the small instance, ``steep``/``snow`` for the large one).
* ``machines.csv`` adds a ``crew`` column; the generated ``crew_assignments.csv`` table records the
  crew → machine mapping with capability notes so validators and scenario loaders can reconstruct
  assignments.
* Crew capabilities are sampled from the ``capability_pool`` while respecting the tier-specific span
  (single-skill crews in ``small``, multi-skill teams in ``large``).
* Blackout windows are produced with tier defaults: zero-probability for ``small`` bundles, short gaps
  for ``medium``, and multi-day outages for ``large``. Metadata captures the sampled windows so smoke
  tests can diff future changes.
* Weighted terrain/prescription distributions and blackout bias windows can be overridden via
  ``SyntheticDatasetConfig`` (see ``terrain_weights`` / ``prescription_weights`` and ``blackout_biases``)
  to concentrate sampling on seasonal or topographical hotspots. Harvest-system mixes are also recorded
  when bundled systems are supplied.

If you regenerate the library, the metadata YAML under each tier (and the aggregate metadata file in
``examples/synthetic/``) records the terrain/prescription mix, crew capabilities, blackout windows,
and the seed that produced the dataset.

CLI walkthrough
---------------

Use the bundles anywhere a normal scenario is accepted. Typical entry points:

* Validate the data contract:

  .. code-block:: bash

     fhops validate examples/synthetic/medium/scenario.yaml

* Run the simulated annealing baseline and capture the schedule:

  .. code-block:: bash

     fhops solve-heur examples/synthetic/medium/scenario.yaml --out /tmp/medium_sa.csv --seed 99

* Generate shift/day summaries with KPI exports:

  .. code-block:: bash

     fhops eval-playback examples/synthetic/medium/scenario.yaml \\
                         --assignments /tmp/medium_sa.csv \\
                         --shift-out /tmp/medium_shift.csv \\
                         --day-out /tmp/medium_day.csv \\
                         --summary-md /tmp/medium_playback.md

* Benchmark solver presets across the reference datasets:

  .. code-block:: bash

     fhops bench suite --scenario examples/synthetic/large/scenario.yaml \\
                       --operator-preset explore \\
                       --out-dir tmp/bench_synth_large

* Generate a fresh bundle (preview or write to disk):

  .. code-block:: bash

     fhops synth generate tmp/synth_medium_custom --tier medium --seed 321 --blocks 10:14 --preview

  Drop ``--preview`` (and optionally add ``--overwrite``) to write the generated CSV/YAML bundle. The
  command accepts YAML/TOML configs via ``--config`` when you need full control over the
  ``SyntheticDatasetConfig`` fields.

  Common flags you can mix in:

  - ``--machines 6:8`` / ``--landings 2`` / ``--days 10:14`` to resample sizes.
  - ``--shifts-per-day 2`` to simulate multi-shift calendars.
  - ``--machine-daily-hours 20`` when you want the generated ``machines.csv`` to reflect partial-day
    availability (default is 24, matching the data-contract baseline).

  .. note::
     When you write directly to ``examples/synthetic/<tier>`` the CLI automatically refreshes the aggregate
     ``examples/synthetic/metadata.yaml`` so documentation and automation stay in sync.

* Create several bundles via a batch plan:

  .. code-block:: bash

     fhops synth batch docs/examples/synthetic_batch_plan.yaml --overwrite

  The plan file can be YAML/TOML/JSON and contains a list of entries, each mirroring the options
  accepted by ``fhops synth generate`` (``tier``, ``seed``, ``config``, overrides, etc.). This is handy
  when preparing datasets for experiments or CI warmups. See
  ``docs/examples/synthetic_batch_plan.yaml`` (and the referenced ``synthetic_batch_custom.yaml``) for
  a ready-to-run example.

These commands reuse the same CLI surfaces already documented in :doc:`evaluation` and
:doc:`../reference/cli`, but the synthetic bundles keep the inputs lightweight enough for quick
iteration and teaching exercises.

The loader automatically ingests ``crew_assignments.csv`` when present, so downstream analytics can
group KPIs by crew. Pair the bundle metadata with playback exports to slice utilisation or downtime
by terrain/prescription class.

Regenerating bundles
--------------------

The generator can be scripted to refresh or extend the library. Seeds are recorded in
``metadata.yaml`` so you can rebuild a bundle exactly:

.. code-block:: python

   from pathlib import Path

   from fhops.scenario.synthetic import SyntheticDatasetConfig, generate_random_dataset

   config = SyntheticDatasetConfig(
       name="synthetic-medium",
       tier="medium",
       num_blocks=(8, 10),
       num_days=12,
       num_machines=4,
       num_landings=2,
       shifts_per_day=1,
       shift_hours=(9.5, 10.5),
       terrain_pool=["rolling", "mixed", "steep"],
       crew_pool=["crew-alpha", "crew-beta", "crew-gamma"],
       capability_pool=["harvester", "forwarder", "processor"],
       crew_capability_span=(1, 2),
   )

   bundle = generate_random_dataset(config, seed=202)
   bundle.write(Path("examples/synthetic/medium"))

You can customise the config ranges (blocks, production rates, blackout probabilities, and role
assignments) or pass a ``systems`` dictionary to incorporate predefined harvest system templates via
``generate_random_dataset(..., systems=default_system_registry())``.

For small teaching examples, the deterministic helper :func:`fhops.scenario.synthetic.generate_basic`
creates a scenario with a single landing and no randomisation; for system-aware datasets use
:func:`fhops.scenario.synthetic.generate_with_systems`.

Where to go next
----------------

* ``tests/test_synthetic_dataset.py`` illustrates the expected invariants and is a good starting
  point if you extend the generator.
* The benchmarking how-to (:doc:`benchmarks`) explains how to wire new scenarios into the benchmarking
  harness once you are happy with the dataset characteristics.