INV-01 TSA 23 Full Run
This guide reproduces the TSA 23 point-cloud feature run used for the INV-01 pilot. It is a research runbook, not an operational scheduler.
Architecture
Sockeye compute nodes have no public internet. Public LidarBC point clouds are downloaded to Sockeye scratch, processed there, and archived to Chinook through Globus. The compute phase reads scratch, not Chinook S3.
LidarBC HTTPS
-> Sockeye scratch raw/laz + raw/dem
-> per-batch Parquet parts
-> local reduce + model cards
-> GeoTIFF / inventory table
Chinook (Globus):
inv01/raw/laz
inv01/raw/dem
inv01/parts2
inv01/products
Runbook
The executable examples are in
examples/inv01_tsa23_full:
build_manifest.pypairs LidarBC point-cloud and 1 m DEM tiles.split_manifest.pycreates download shards and 64-tile batches.download_tiles.pydownloads and resumes tiles.process_batch.pyderives first-return metrics on the global grid.compute_batches.sbatchruns the batch array.reduce_full.pyclips to the TSA boundary, applies the IDF cards, and optionally writes the 25 m GeoTIFF.
Build the manifest:
python examples/inv01_tsa23_full/build_manifest.py \
--selection lidarbc_selection_full.json \
--boundary tsa23_boundary.gpkg \
--output tile_manifest_full.csv
Split it:
python examples/inv01_tsa23_full/split_manifest.py \
--manifest tile_manifest_full.csv \
--download-shards 2 \
--batch-size 64 \
--output-dir "$INV01_ROOT/manifests"
Download:
python examples/inv01_tsa23_full/download_tiles.py \
"$INV01_ROOT/manifests/download_a.csv" \
"$INV01_ROOT/raw/laz" "$INV01_ROOT/raw/dem" 8
Process batches after setting INV01_ROOT, PYTHON, FEMIC_SRC, and
RUNBOOK_DIR:
export INV01_ROOT=/scratch/st-gep-1/gep/inv01
export PYTHON=/arc/project/st-gep-1/gep/venv-inv01-312/bin/python
export FEMIC_SRC=/arc/project/st-gep-1/gep/inv01/femic-src/src
export RUNBOOK_DIR=$PWD/examples/inv01_tsa23_full
cd "$INV01_ROOT"
sbatch "$RUNBOOK_DIR/compute_batches.sbatch"
The global 25 m grid used for this run is in UTM zone 10N:
origin
(549275, 5784675)cell size
25 mminimum
10above-2 m first returnsminimum cover
10%
Reduce locally:
python examples/inv01_tsa23_full/reduce_full.py \
--parts /srv/shared-data/gep/ws4/tmp/inv01-data/full-output/parts \
--boundary tsa23_boundary.gpkg \
--models config/inventory/models \
--out-dir /srv/shared-data/gep/ws4/tmp/inv01-data/full-output \
--raster
Results and limits
The 2026-10-30 run produced:
5,899 point-cloud/DEM tile pairs
93 batches
9,523,476 TSA-clipped cells
6,573,196 height cells
2,942,755 BA/volume cells
mean height 16.09 m, BA 23.87 m2/ha, volume 139.33 m3/ha
Coverage was 64% of TSA 23 by tile footprint and 48% by modelled cell area. 907 point-cloud tiles had no matching 1 m DEM; 24 tiles had no usable first returns. The model cards are the 14-plot IDF pilot cards, not an operational inventory. No independent P48/P49 validation was available.
The detailed evidence, timings, hashes, and provenance are in
notes/reports/inv01-tsa23-full-mosaic.md.