Compiled corridor dataset
The compiled dataset is the boundary between ingestion and execution. The simulator and the planner read
the dataset and nothing else from the raw sources. A dataset directory contains a manifest.json and
one file per table.
Schema version: brushline.dataset.v1.
Directory layout
dataset/
manifest.json
segments.parquet
The first release requires the segments table. P1.2 and P1.3 add the corridor, state, treatment,
observation, access, and structure tables. P1.4 adds the plan and run contracts, which live in the run
directory rather than the compiled dataset.
Manifest fields
Field |
Meaning |
|---|---|
|
|
|
Stable dataset identifier. |
|
Human-readable identity. |
|
Timezone-aware creation time. |
|
Coordinate reference system, for example |
|
Named units, such as |
|
One |
|
Source, version, licence, creator, configuration, and model cards. |
|
SHA-256 of the canonical content fields. |
Table fields
Field |
Meaning |
|---|---|
|
Logical table name. |
|
Dataset-relative file path. Absolute paths and |
|
|
|
One or more columns that must be present, non-null, and unique. |
|
Column contracts. |
|
Expected row count. |
|
SHA-256 of the table file. |
Column fields
Field |
Meaning |
|---|---|
|
Column name. |
|
Logical type: |
|
Whether the column must be present. Defaults to true. |
|
Physical unit, when the column has one. |
|
Short human description. |
The segments table
The P1.1 contract requires four columns:
Column |
Type |
Meaning |
|---|---|---|
|
string |
Primary key. |
|
string |
Corridor that owns the segment. |
|
float64 |
Segment length in metres. |
|
string |
Segment geometry as WKT. |
P1.2 adds the span, treatment-unit, site, access, structure, and service-standard fields.
Content digest
The content digest covers the schema version, the dataset id, the CRS, and the table contracts: name,
path, format, primary key, row count, checksum, and column names, types, required flags, and units. It
does not cover created_utc, name, or description, so a rebuild with the same content keeps the
same digest.
Validation
Run:
brushline validate --dataset build/compiled
brushline validate --dataset build/compiled --json
The validator checks:
the manifest exists and is valid JSON;
the schema version is
brushline.dataset.v1;every table path is relative and stays inside the dataset;
every table file exists and can be read;
every required column is present and has the declared logical type;
the primary key is present, non-null, and unique;
the row count matches;
the SHA-256 matches;
the content digest matches;
the required
segmentstable is present.
Errors use stable prefixes: DATASET-MANIFEST, DATASET-SCHEMA, DATASET-PATH,
DATASET-TABLE, DATASET-COLUMN, DATASET-DTYPE, DATASET-KEY, DATASET-ROW-COUNT,
DATASET-CHECKSUM, and DATASET-DIGEST. An undeclared extra column is a warning
(DATASET-EXTRA-COLUMN), not an error.
Example
{
"schema_version": "brushline.dataset.v1",
"dataset_id": "synthetic-corridor-v1",
"name": "Synthetic corridor",
"description": "A two-corridor contract fixture.",
"created_utc": "2026-11-14T00:00:00Z",
"crs": "EPSG:3005",
"units": {"length": "m", "area": "ha"},
"tables": [
{
"name": "segments",
"path": "segments.parquet",
"format": "parquet",
"primary_key": ["segment_id"],
"columns": [
{"name": "segment_id", "dtype": "string", "required": true},
{"name": "corridor_id", "dtype": "string", "required": true},
{"name": "length_m", "dtype": "float64", "required": true, "unit": "m"},
{"name": "geometry_wkt", "dtype": "string", "required": true}
],
"row_count": 2,
"sha256": "0000000000000000000000000000000000000000000000000000000000000000"
}
],
"provenance": {
"source": "synthetic",
"source_version": "1.0",
"licence": "CC0-1.0",
"created_by": "brushline"
},
"content_digest": "0000000000000000000000000000000000000000000000000000000000000000"
}
The example digests are placeholders. A real manifest carries the actual file checksum and the computed content digest.