Architecture and data flow

The package has four layers: source tables, builder, state cache, and consumer.

Data flow

carbon package --------+
compiled dataset ------+--> builder (WS4 CarbonLedger) --> state arrays
treatments/transitions +                                      |
HWP config                                                    v
                                                     manifest + index
                                                            |
                                                            v
                                                   fresh_cbm.StateCache
                                                            |
                               +----------------------------+------------------+
                               |                                               |
                               v                                               v
                     WS4 state-cache consumer                     direct client lookup
                               |
                               v
                    block area weighting + target cells
                               |
                               v
                      exact ws4 carbon replay

Components

Builder

scripts/build_state_cache.py uses WS4’s exact CarbonLedger stand-state path. It enumerates the declared track/age states, prices each chain offset, and writes one array per chain.

The builder is intentionally outside the runtime package. It needs WS4 and libcbm; the client does not.

Cache

A cache directory contains:

  • manifest.json — versioned identity and schema;

  • index.parquet — row id to track/age mapping;

  • values/empty.npy — untreated states;

  • values/offset_XX.npy — one-event CC states.

Each value array has shape (states, years + 1, accounts).

Client

fresh_cbm.StateCache validates the manifest, maps (track_id, age) to a row, and reads the appropriate memmap. It returns account vectors in the manifest’s declared order.

Consumer

fresh_cbm.effects.build_block_effects applies fragment area weights and maps cached states to target cells. WS4’s ws4 carbon-removal –state-cache uses that projection for carbon cells and combines it with non-carbon cells from the WS4 simulator. It creates an exact per-block effect table, searches it, and replays the winning schedule through the exact carbon path.

State identity

The logical key is:

package fingerprint
+ track/classifier identity
+ age
+ treatment chain
+ boundary variant
+ HWP / fertilization config
+ CBM defaults and libcbm version
+ horizon and account schema

The manifest is the machine-readable identity. A consumer must reject a cache that does not match its run.

Exactness and fallback

The cache is exact for its declared state family. It is not a complete model of all possible states. A consumer must fall back to libcbm or the exact WS4 ledger when:

  • a state, age, track, or offset is missing;

  • a chain has unsupported events;

  • the package or CBM defaults differ;

  • the account schema or horizon differs; or

  • the cache file fails checksum or shape validation.

Performance model

The cache changes the cost profile:

  • build: one-time CBM state construction, parallel over states;

  • lookup: metadata validation, a memmap read, and array indexing;

  • consumer: block area sums and target-cell projection.

For the published TSA41 cache, the full landscape lookup removed a 976 s table build. The remaining full-run cost was exact replay.

Security and provenance

  • Release values are hashed in SHA256SUMS.

  • The manifest records package fingerprint, CBM defaults hash, and libcbm version.

  • Public manifests must not contain machine-specific paths or private data.

  • The client validates the schema but does not replace a checksum audit.