Geospatial and Runtime Bootstrap

Why This Matters

FEMIC depends on more than Python packages. A usable workstation needs a combination of:

geospatial Python libraries (Fiona/GDAL)
Git + git-annex + DataLad for annex-backed public-data payloads
platform-specific external tools such as VDYP, Patchworks, Java, ArcGIS Pro, and Wine where applicable

This guide records the currently known-good bootstrap rituals for both Windows and Linux, with Windows treated as the active reference host for end-to-end Patchworks validation.

For the canonical source-checkout developer ritual, see docs/guides/developer-environment-bootstrap.rst.

Authoritative Platform Runtime Surfaces

Windows

Treat Windows as authoritative for:

native Patchworks launch and Matrix Builder
native Java runtime for Patchworks
native VDYP (VDYP7Console.exe)
ArcGIS Pro fallback for SiteProd geoprocessing when canonical siteprod.tif + siteprod.bandmap.json are unavailable
Git + git-annex + DataLad access to external/femic-public-data

Linux

Treat Linux as authoritative for:

normal Python/FEMIC development workflow
upstream FEMIC stages that do not require native Patchworks
Wine-wrapped VDYP execution where native Windows VDYP is unavailable
ArcRasterRescue executable invocation via the documented patched fork build (or explicit FEMIC_ARC_RASTER_RESCUE_EXE override)

Core Executables and Services

Windows workstation checklist:

python
git
git annex
.venvScriptsdatalad.exe
Java for Patchworks
Patchworks installation / patchworks.jar
native VDYP7Console.exe
ArcGIS Pro Python (propy.bat) available by explicit path if not on PATH

Linux workstation checklist:

python
git
git annex
datalad
java
wine / wine64
Linux geospatial runtime (gdal-bin, libgdal-dev, Fiona-compatible stack)

Linux VDYP runtime note:

When running with --instance-root (including temporary /tmp clones), FEMIC now stages missing legacy VDYP runtime assets (vdyp_io/VDYP_CFG and vdyp_io/VDYP.INI) from FEMIC_SOURCE_ROOT before Wine dispatch.
Keep the source checkout runtime payloads intact under $FEMIC_SOURCE_ROOT/vdyp_io (or $FEMIC_SOURCE_ROOT/VDYP7/VDYP7).

Windows Bootstrap Ritual

Upgrade packaging tools:

python -m pip install --upgrade pip setuptools wheel

Install/refresh the local virtual environment dependencies:

python -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install -r requirements-dev.txt

Confirm the Windows runtime baseline:

git --version
git annex version
.venv\Scripts\datalad.exe --version
java --version

Materialize the annex-backed public data you need:

git submodule update --init --recursive
git -C external/femic-public-data annex enableremote arbutus-s3
.venv\Scripts\datalad.exe get -r external/femic-public-data/data

Validate the case before long runs:

femic prep validate-case --instance-root external/femic-k3z-instance --run-config config/run_profile.k3z.yaml
femic prep geospatial-preflight

For the known-good K3Z Windows path, expect the following runtime pattern:

native VDYP
canonical pre-stacked SiteProd TIFF + band-map by default, with ArcGIS Pro fallback only when those artifacts are unavailable
default unattended BTC handoff at the 03_input-*.csv / 04_output-*.csv boundary
native Patchworks / Matrix Builder after post-TIPSY

Linux Bootstrap Ritual

Install system geospatial dependencies first:

sudo apt-get update
sudo apt-get install -y gdal-bin libgdal-dev

Install/refresh the virtual environment:

python -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip setuptools wheel
python -m pip install -r requirements-dev.txt

Confirm the Linux runtime baseline:

git --version
git annex version
datalad --version
java --version
wine --version

Materialize the annex-backed public data you need:

git submodule update --init --recursive
git -C external/femic-public-data annex enableremote arbutus-s3
datalad get -r external/femic-public-data/data

Validate the case before long runs:

export FEMIC_EXTERNAL_DATA_ROOT=$PWD/external/femic-public-data/data
femic prep validate-case --instance-root external/femic-k3z-instance --run-config config/run_profile.k3z.yaml
femic prep geospatial-preflight

DataLad / git-annex Smoke Checks

These checks are lightweight and worth running before a clean-start pipeline rerun:

.venv\Scripts\datalad.exe get external/femic-public-data/data/misc.thlb.tif
Test-Path external\femic-public-data\data\misc.thlb.tif

A healthy Windows checkout should also report:

git -C external/femic-public-data annex version
.venv\Scripts\datalad.exe status external/femic-public-data

If the payload is present and the repo responds normally, the Windows public-data bootstrap is good enough for FEMIC pipeline runs.

Native Windows clones may still materialize some annexed raster worktree paths as tiny pointer stubs instead of ordinary TIFF files. FEMIC now resolves those pointer-style paths at the direct THLB/SiteProd raster-open seams before calling rasterio.open(...), so Linux behavior stays unchanged while Windows clones remain usable without extra manual checkout-mode tweaking.

Verify Runtime Readiness

Run FEMIC geospatial preflight after install:

femic prep geospatial-preflight

This checks:

Fiona import
GDAL version visibility
basic shapefile write/read smoke test

This is intentionally a generic runtime smoke, not a case-aware annex or FileGDB materialization check. On Windows, passing femic prep geospatial-preflight does not prove that the canonical annex-backed TSA boundary geodatabase is readable in the active FEMIC case. Use femic prep validate-case for that.

Troubleshooting

If git or git annex is missing on Windows, fix the user PATH first and restart the shell.
If datalad is available only in .venv, use .venvScriptsdatalad.exe explicitly instead of relying on PATH.
If annex-backed payloads show only pointer files, run git -C external/femic-public-data annex enableremote arbutus-s3 and then datalad get -r external/femic-public-data/data before rerunning FEMIC.
If femic prep validate-case fails on external/femic-public-data/data/bc/tsa/FADM_TSA.gdb, do not jump straight to reinstalling GDAL. First treat it as a likely public-data materialization seam and run the canonical open-source recovery sequence:
```
git -C external/femic-public-data annex enableremote arbutus-s3
.venv\Scripts\datalad.exe get -r external/femic-public-data/data
git -C external/femic-public-data annex unlock data/bc/tsa/FADM_TSA.gdb
python -m femic prep validate-case --instance-root external/femic-k3z-instance --run-config config/run_profile.k3z.yaml
```
A successful open-source recovery should end with the canonical layer WHSE_ADMIN_BOUNDARIES_FADM_TSA becoming readable again through the same validate-case seam.
If Fiona imports but shapefile smoke fails, verify GDAL shared-library resolution and recreate the virtual environment.
If the open-source recovery path is still blocked but ArcGIS Pro is available, treat arcpy as a fallback recovery leg for exporting the TSA boundary to a GeoPandas-friendly artifact. It is a fallback, not a required primary FEMIC runtime dependency.
If ArcGIS Pro fallback is required, treat propy.bat as a path-resolved tool, not something guaranteed to be on PATH.
For manual GIS review on Windows, femic prep arcgis-review-project can emit a ready-to-open ArcGIS Pro project from the instance’s local .shp and .gpkg layers. This is an inspection aid only: it does not replace FEMIC’s canonical geoprocessing/runtime pipeline, all emitted layers default to visible = off so the review project opens as a quiet workspace, and GeoPackage-backed layers can be staged as helper shapefiles under the chosen output directory when ArcGIS compatibility requires it.
If Linux VDYP runs but Windows does not, check the Windows-native VDYP config directory and parameter-file resolution before rerunning the full pipeline.
If Stage 00 cannot find ArcRasterRescue, set FEMIC_ARC_RASTER_RESCUE_EXE explicitly (or restore the documented sibling-checkout layout) rather than changing SiteProd extraction design.