nemora.ingest
Nemora’s ingest package wraps reusable dataset abstractions and pipelines that
convert raw inventory releases (FAIB PSP, FIA Datamart, etc.) into the shared
nemora.inventory.v1 tables consumed by nemora.fit, nemora.sampling, and
downstream tooling.
The key entry points mirror the concepts introduced in the
Ingest Module how-to:
DatasetSource/DatasetFetcherdescribe how to locate and cache raw files.TransformPipelineorchestrates composable DataFrame transformations.Submodules (
faib,fia,hps) provide dataset-specific helpers and CLI-ready pipelines for stand-table and HPS tally generation.
See also
docs/howto/ingest.mdfor step-by-step ingest workflows.docs/howto/inventory_schema.mdfor the sharednemora.inventory.v1release contract and HPS migration.nemora.clifor Typer commands such asingest-faib,faib-manifest, andingest-faib-hpsthat wrap these helpers.
Package API
.. py:module:: nemora.ingest
Ingestion/ETL scaffolding for Nemora.
This module defines lightweight abstractions that describe raw inventory
datasets (DatasetSource) and the transformation pipelines (TransformPipeline)
that convert them into the tidy stand tables consumed by other Nemora modules.
Concrete connectors for BC FAIB, FIA, and other inventories will extend these
primitives in upcoming revisions.
.. py:class:: DatasetFetcher(*args, **kwargs) :module: nemora.ingest
Bases: :py:class:~typing.Protocol
Callable that retrieves one or more artifacts for a dataset source.
.. py:class:: DatasetSource(name, description, uri=None, metadata=
Bases: :py:class:object
Describe a raw inventory dataset that can be ingested by Nemora.
:type name: :sphinx_autodoc_typehints_type:\:py\:class\:\str` :param name: Human-readable identifier for the dataset. :type description: :sphinx_autodoc_typehints_type::py:class:`str` :param description: Short summary of the dataset contents (region, sampling design, etc.). :type uri: :sphinx_autodoc_typehints_type::py:class:`str` | :py:obj:`None` :param uri: Optional canonical URI (open data portal link, DataLad URL, etc.). :type metadata: :sphinx_autodoc_typehints_type::py:class:`dict`\ \[:py:class:`str`, :py:data:`~typing.Any`] :param metadata: Arbitrary extra fields (licensing, citation info, cache preferences). :type fetcher: :sphinx_autodoc_typehints_type::py:class:`~nemora.ingest.DatasetFetcher` | :py:obj:`None``
:param fetcher: Optional callable able to retrieve the dataset artifacts when invoked.
.. py:attribute:: DatasetSource.description :module: nemora.ingest :type: str
.. py:method:: DatasetSource.fetch() :module: nemora.ingest
Return artifacts for this dataset via the configured fetcher.
:rtype: :sphinx_autodoc_typehints_type:`\:py\:class\:\`\~collections.abc.Iterable\`\\ \\\[\:py\:class\:\`\~pathlib.Path\`\]`
.. py:attribute:: DatasetSource.fetcher :module: nemora.ingest :type: ~nemora.ingest.DatasetFetcher | None
.. py:attribute:: DatasetSource.metadata :module: nemora.ingest :type: dict[str, ~typing.Any]
.. py:attribute:: DatasetSource.name :module: nemora.ingest :type: str
.. py:attribute:: DatasetSource.uri :module: nemora.ingest :type: str | None
.. py:class:: ForestPlotTarget(name, latitude, longitude, radius_km, bec_codes, min_projected_age=60.0, max_projected_age=None, sample_type=’F’, first_visit_only=True, min_distance_km=0.0) :module: nemora.ingest :canonical: nemora.ingest.selection.ForestPlotTarget
Bases: :py:class:object
Describe a forest validation target and its plot filters.
:type name: :sphinx_autodoc_typehints_type:\:py\:class\:\str` :param name: Human-readable target name. :type latitude: :sphinx_autodoc_typehints_type::py:class:`float` :param latitude: Target centre in WGS84 decimal degrees. :type longitude: :sphinx_autodoc_typehints_type::py:class:`float` :param longitude: Target centre in WGS84 decimal degrees. :type radius_km: :sphinx_autodoc_typehints_type::py:class:`float` :param radius_km: Great-circle search radius. :type bec_codes: :sphinx_autodoc_typehints_type::py:class:`tuple`\ \[:py:class:`str`, :py:data:`…<Ellipsis>`] :param bec_codes: Accepted BEC codes, such as ``("SBSdw1", "ICHmk3")``. :type min_projected_age: :sphinx_autodoc_typehints_type::py:class:`float` :param min_projected_age: Minimum ``PROJ_AGE_1`` in years. :type sample_type: :sphinx_autodoc_typehints_type::py:class:`str` | :py:obj:`None` :param sample_type: Optional FAIB ``SAMP_TYP`` value to keep. :type first_visit_only: :sphinx_autodoc_typehints_type::py:class:`bool``
:param first_visit_only: Keep only visits marked FIRST_MSMT == "Y".
.. py:attribute:: ForestPlotTarget.bec_codes :module: nemora.ingest :type: tuple[str, …]
.. py:attribute:: ForestPlotTarget.first_visit_only :module: nemora.ingest :type: bool
.. py:attribute:: ForestPlotTarget.latitude :module: nemora.ingest :type: float
.. py:attribute:: ForestPlotTarget.longitude :module: nemora.ingest :type: float
.. py:attribute:: ForestPlotTarget.max_projected_age :module: nemora.ingest :type: float | None
.. py:attribute:: ForestPlotTarget.min_distance_km :module: nemora.ingest :type: float
.. py:attribute:: ForestPlotTarget.min_projected_age :module: nemora.ingest :type: float
.. py:attribute:: ForestPlotTarget.name :module: nemora.ingest :type: str
.. py:attribute:: ForestPlotTarget.radius_km :module: nemora.ingest :type: float
.. py:attribute:: ForestPlotTarget.sample_type :module: nemora.ingest :type: str | None
.. py:class:: InventoryProvenance(source_id, source_version, retrieved_utc, source_uri=None, licence=None, checksum=None, notes=None) :module: nemora.ingest :canonical: nemora.ingest.schema.InventoryProvenance
Bases: :py:class:object
Release-level source and retrieval metadata.
.. py:attribute:: InventoryProvenance.checksum :module: nemora.ingest :type: str | None
.. py:attribute:: InventoryProvenance.licence :module: nemora.ingest :type: str | None
.. py:attribute:: InventoryProvenance.notes :module: nemora.ingest :type: str | None
.. py:attribute:: InventoryProvenance.retrieved_utc :module: nemora.ingest :type: str
.. py:attribute:: InventoryProvenance.source_id :module: nemora.ingest :type: str
.. py:attribute:: InventoryProvenance.source_uri :module: nemora.ingest :type: str | None
.. py:attribute:: InventoryProvenance.source_version :module: nemora.ingest :type: str
.. py:class:: InventoryRelease(release_id, provenance, plots, trees, species=
Bases: :py:class:object
A validated, tidy inventory release.
.. py:attribute:: InventoryRelease.crs :module: nemora.ingest :type: str | None
.. py:attribute:: InventoryRelease.metadata :module: nemora.ingest :type: dict[str, ~typing.Any]
.. py:attribute:: InventoryRelease.plots :module: nemora.ingest :type: ~pandas.DataFrame
.. py:attribute:: InventoryRelease.provenance :module: nemora.ingest :type: ~nemora.ingest.schema.InventoryProvenance
.. py:attribute:: InventoryRelease.release_id :module: nemora.ingest :type: str
.. py:property:: InventoryRelease.schema_version :module: nemora.ingest :type: str
Return the schema version written by this release.
.. py:attribute:: InventoryRelease.species :module: nemora.ingest :type: ~pandas.DataFrame
.. py:method:: InventoryRelease.to_manifest() :module: nemora.ingest
Return the JSON-serializable release manifest.
:rtype: :sphinx_autodoc_typehints_type:`\:py\:class\:\`dict\`\\ \\\[\:py\:class\:\`str\`\, \:py\:data\:\`\~typing.Any\`\]`
.. py:attribute:: InventoryRelease.trees :module: nemora.ingest :type: ~pandas.DataFrame
.. py:attribute:: InventoryRelease.units :module: nemora.ingest :type: ~nemora.ingest.schema.InventoryUnits
.. py:class:: InventoryUnits(dbh=’cm’, height=’m’, tally=’count’, baf=’m2_per_ha’) :module: nemora.ingest :canonical: nemora.ingest.schema.InventoryUnits
Bases: :py:class:object
Canonical units for the inventory tables.
.. py:attribute:: InventoryUnits.baf :module: nemora.ingest :type: str
.. py:attribute:: InventoryUnits.dbh :module: nemora.ingest :type: str
.. py:attribute:: InventoryUnits.height :module: nemora.ingest :type: str
.. py:attribute:: InventoryUnits.tally :module: nemora.ingest :type: str
.. py:exception:: InventoryValidationError(errors) :module: nemora.ingest :canonical: nemora.ingest.schema.InventoryValidationError
Bases: :py:class:ValueError
Raised when an inventory release does not match the shared schema.
.. py:class:: PlotInventorySpec(release_id, provenance, plot_columns=
Bases: :py:class:object
Describe how source tables map to the canonical inventory schema.
Column mappings use canonical names as keys and source-column names as
values. A canonical name that already exists in the source frame is used
without a mapping entry. Missing optional columns are filled from the
corresponding defaults or with NA.
:type release_id: :sphinx_autodoc_typehints_type:\:py\:class\:\str` :param release_id: Stable identifier written to the release manifest. :type provenance: :sphinx_autodoc_typehints_type::py:class:`~nemora.ingest.schema.InventoryProvenance` :param provenance: Source and retrieval metadata required by ``nemora.inventory.v1``. :type plot_columns: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Mapping`\ \[:py:class:`str`, :py:class:`str`] :param plot_columns: Mapping from canonical plot columns to source column names. :type tree_columns: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Mapping`\ \[:py:class:`str`, :py:class:`str`] :param tree_columns: Mapping from canonical tree columns to source column names. :type species_columns: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Mapping`\ \[:py:class:`str`, :py:class:`str`] :param species_columns: Mapping from canonical species columns to source column names. :type plot_defaults: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Mapping`\ \[:py:class:`str`, :py:class:`object`] :param plot_defaults: Values used when a canonical plot column is absent. :type tree_defaults: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Mapping`\ \[:py:class:`str`, :py:class:`object`] :param tree_defaults: Values used when a canonical tree column is absent. :type species_defaults: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Mapping`\ \[:py:class:`str`, :py:class:`object`] :param species_defaults: Values used when a canonical species column is absent. :type units: :sphinx_autodoc_typehints_type::py:class:`~nemora.ingest.schema.InventoryUnits` :param units: Canonical release units. :type crs: :sphinx_autodoc_typehints_type::py:class:`str` | :py:obj:`None` :param crs: CRS for plot coordinates, when present. :type metadata: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Mapping`\ \[:py:class:`str`, :py:class:`object`] :param metadata: Additional JSON-serializable release metadata. :type derive_tree_count: :sphinx_autodoc_typehints_type::py:class:`bool``
:param derive_tree_count: Fill a missing plots.tree_count with the number of tree rows per
plot. This is a row count, not a sum of expansion tallies.
.. py:attribute:: PlotInventorySpec.crs :module: nemora.ingest :type: str | None
.. py:attribute:: PlotInventorySpec.derive_tree_count :module: nemora.ingest :type: bool
.. py:attribute:: PlotInventorySpec.metadata :module: nemora.ingest :type: ~collections.abc.Mapping[str, object]
.. py:attribute:: PlotInventorySpec.plot_columns :module: nemora.ingest :type: ~collections.abc.Mapping[str, str]
.. py:attribute:: PlotInventorySpec.plot_defaults :module: nemora.ingest :type: ~collections.abc.Mapping[str, object]
.. py:attribute:: PlotInventorySpec.provenance :module: nemora.ingest :type: ~nemora.ingest.schema.InventoryProvenance
.. py:attribute:: PlotInventorySpec.release_id :module: nemora.ingest :type: str
.. py:attribute:: PlotInventorySpec.species_columns :module: nemora.ingest :type: ~collections.abc.Mapping[str, str]
.. py:attribute:: PlotInventorySpec.species_defaults :module: nemora.ingest :type: ~collections.abc.Mapping[str, object]
.. py:attribute:: PlotInventorySpec.tree_columns :module: nemora.ingest :type: ~collections.abc.Mapping[str, str]
.. py:attribute:: PlotInventorySpec.tree_defaults :module: nemora.ingest :type: ~collections.abc.Mapping[str, object]
.. py:attribute:: PlotInventorySpec.units :module: nemora.ingest :type: ~nemora.ingest.schema.InventoryUnits
.. py:class:: TransformPipeline(name, steps=
Bases: :py:class:object
A sequence of callables that transform raw dataframes into Nemora tables.
.. py:method:: TransformPipeline.add_step(step) :module: nemora.ingest
Append a transformation step to the pipeline.
:rtype: :sphinx_autodoc_typehints_type:`\:py\:obj\:\`None\``
.. py:attribute:: TransformPipeline.metadata :module: nemora.ingest :type: dict[str, ~typing.Any]
.. py:attribute:: TransformPipeline.name :module: nemora.ingest :type: str
.. py:method:: TransformPipeline.run(frame) :module: nemora.ingest
Apply every transformation step to the supplied dataframe.
:rtype: :sphinx_autodoc_typehints_type:`\:py\:class\:\`\~pandas.DataFrame\``
.. py:attribute:: TransformPipeline.steps :module: nemora.ingest :type: list[~collections.abc.Callable[[~pandas.DataFrame], ~pandas.DataFrame]]
.. py:function:: assign_holdout_split(plots, *, holdout_fraction=0.25, seed=20261030) :module: nemora.ingest
Assign a deterministic train/hold-out split.
The split uses a SHA-256 key over seed and plot_id. It is stable
across machines and Python versions and does not depend on row order.
:type plots: :sphinx_autodoc_typehints_type:\:py\:class\:\~pandas.DataFrame` :param plots: Selected plot frame with a ``plot_id`` column. :type holdout_fraction: :sphinx_autodoc_typehints_type::py:class:`float` :param holdout_fraction: Fraction of plots assigned to the hold-out set. :type seed: :sphinx_autodoc_typehints_type::py:class:`int``
:param seed: Integer split seed.
:returns: A copy with a split column containing train or holdout.
:rtype: pandas.DataFrame
:raises ValueError: If there are fewer than two plots or the fraction is outside (0, 1).
.. py:function:: build_inventory_release(plots, trees, *, spec, species=None) :module: nemora.ingest
Resolve source tables into a validated nemora.inventory.v1 release.
:type plots: :sphinx_autodoc_typehints_type:\:py\:class\:\~pandas.DataFrame` :param plots: Source plot table. It must provide, or map to, ``plot_id``. :type trees: :sphinx_autodoc_typehints_type::py:class:`~pandas.DataFrame` :param trees: Source tree or tally-class table. It must provide, or map to, ``plot_id``, ``dbh_cm``, and ``tally``. :type spec: :sphinx_autodoc_typehints_type::py:class:`~nemora.ingest.plots.PlotInventorySpec` :param spec: Release metadata, column mappings, and defaults. :type species: :sphinx_autodoc_typehints_type::py:class:`~pandas.DataFrame` | :py:obj:`None``
:param species: Optional species lookup table.
:returns: A validated canonical release. :rtype: InventoryRelease
:raises InventoryValidationError: If a mapping names an unknown source column or the resolved release is invalid.
.. py:function:: inventory_schema() :module: nemora.ingest
Return the portable schema document for external consumers.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\dict`\ \[:py:class:`str`, :py:data:`~typing.Any`]`
.. py:function:: materialize_tally_trees(release, *, tree_id_prefix=’T’) :module: nemora.ingest
Expand tally classes into individual tree rows with stable IDs.
Rows that already have a tree_id are preserved. Rows without one are
expanded into tally copies with tally=1 and deterministic
<prefix><plot_id>-<sequence> identifiers. Fractional tallies are
rejected because materializing them requires an explicit expansion rule.
:type release: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.schema.InventoryRelease` :param release: A validated ``nemora.inventory.v1`` release. :type tree_id_prefix: :sphinx_autodoc_typehints_type::py:class:`str``
:param tree_id_prefix: Prefix for generated tree identifiers.
:returns: A validated release at the individual-tree grain. :rtype: InventoryRelease
:raises InventoryValidationError: If the input release is invalid or a tally is not a non-negative integer.
.. py:function:: read_inventory_release(directory) :module: nemora.ingest
Read and validate a canonical release directory.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.schema.InventoryRelease``
.. py:function:: resolve_tree_detail(tree_detail_source, selected_plots, *, live_only=True, chunk_size=200000) :module: nemora.ingest
Resolve FAIB tree detail for selected plots into canonical tree rows.
:type tree_detail_source: :sphinx_autodoc_typehints_type:\:py\:class\:\str` | :py:class:`~pathlib.Path` | :py:class:`~pandas.DataFrame` :param tree_detail_source: Path to ``faib_tree_detail.csv`` or an equivalent DataFrame. :type selected_plots: :sphinx_autodoc_typehints_type::py:class:`~pandas.DataFrame` :param selected_plots: Output of :func:select_forest_plots (or a frame with the canonical ``plot_id``, ``cluster_id``, ``visit_number``, and ``plot_number`` columns). :type live_only: :sphinx_autodoc_typehints_type::py:class:`bool` :param live_only: Keep only ``LV_D == "L"`` records. When false, dead records are kept with ``status="dead"``. :type chunk_size: :sphinx_autodoc_typehints_type::py:class:`int``
:param chunk_size: CSV chunk size for streaming reads.
:returns: Canonical nemora.inventory.v1 tree columns only.
:rtype: pandas.DataFrame
:raises KeyError: If required selection or tree columns are missing.
:raises ValueError: If chunk_size is not positive.
.. py:function:: select_forest_plots(site_header, plot_header, sample_by_visit, *, target) :module: nemora.ingest
Select FAIB plots for one target.
:type site_header: :sphinx_autodoc_typehints_type:\:py\:class\:\~pandas.DataFrame` :param site_header: FAIB site-level table with coordinates and BEC fields. :type plot_header: :sphinx_autodoc_typehints_type::py:class:`~pandas.DataFrame` :param plot_header: FAIB plot-level table with cluster, visit, and plot numbers. :type sample_by_visit: :sphinx_autodoc_typehints_type::py:class:`~pandas.DataFrame` :param sample_by_visit: FAIB visit-level table with first-measurement, date, age, and sample type. :type target: :sphinx_autodoc_typehints_type::py:class:`~nemora.ingest.selection.ForestPlotTarget``
:param target: Target and filter definition.
:returns: One row per selected plot with canonical identity, distance, BEC, age, and provenance-relevant metadata. The frame is sorted by distance then plot id. :rtype: pandas.DataFrame
:raises KeyError: If a required source column is missing. :raises ValueError: If no plots match.
.. py:function:: selection_manifest(selected, *, target, holdout_fraction, split_seed) :module: nemora.ingest
Build a location-light manifest for a selected plot set.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\dict`\ \[:py:class:`str`, :py:data:`~typing.Any`]`
.. py:function:: validate_inventory_release(release) :module: nemora.ingest
Validate a release against nemora.inventory.v1.
:raises InventoryValidationError: If the release id, schema tables, units, or provenance are invalid.
:rtype: :sphinx_autodoc_typehints_type:\:py\:obj\:\None``
.. py:function:: write_inventory_release(release, directory) :module: nemora.ingest
Validate and write a canonical release directory.
The directory contains release.json, plots.csv, trees.csv, and
species.csv. CSV columns are always the full schema, so a minimal input
table still produces a stable artifact.
:rtype: :sphinx_autodoc_typehints_type:\:py\:obj\:\None``
.. py:function:: write_selection_manifest(manifest, path) :module: nemora.ingest
Write a selection manifest as stable, indented JSON.
:rtype: :sphinx_autodoc_typehints_type:\:py\:obj\:\None``
Dataset helpers
FAIB (nemora.ingest.faib)
.. py:module:: nemora.ingest.faib
Helpers for working with BC FAIB ground sample datasets.
.. py:class:: DataDictionary(sheets) :module: nemora.ingest.faib
Bases: :py:class:object
Structured representation of FAIB data dictionary entries.
.. py:method:: DataDictionary.get_table_schema(table) :module: nemora.ingest.faib
Return the schema for a specific table.
:rtype: :sphinx_autodoc_typehints_type:`\:py\:class\:\`\~pandas.DataFrame\``
.. py:attribute:: DataDictionary.sheets :module: nemora.ingest.faib :type: ~collections.abc.Mapping[str, ~pandas.DataFrame]
.. py:class:: FAIBManifestResult(manifest_path, tables, bafs, truncated_flags, downloaded) :module: nemora.ingest.faib
Bases: :py:class:object
Summary of outputs produced by :func:generate_faib_manifest.
.. py:attribute:: FAIBManifestResult.bafs :module: nemora.ingest.faib :type: list[float]
.. py:attribute:: FAIBManifestResult.downloaded :module: nemora.ingest.faib :type: list[~pathlib.Path]
.. py:attribute:: FAIBManifestResult.manifest_path :module: nemora.ingest.faib :type: ~pathlib.Path
.. py:attribute:: FAIBManifestResult.tables :module: nemora.ingest.faib :type: list[~pathlib.Path]
.. py:attribute:: FAIBManifestResult.truncated_flags :module: nemora.ingest.faib :type: dict[~pathlib.Path, bool]
.. py:function:: aggregate_stand_table(tree_detail, plot_info, *, baf, dbh_col=’DBH_CM’, expansion_col=’TREE_EXP’, baf_col=’BAF’, group_keys=(‘CLSTR_ID’, ‘VISIT_NUMBER’, ‘PLOT’)) :module: nemora.ingest.faib
Aggregate tree detail records into a stand table for a given BAF.
:type tree_detail: :sphinx_autodoc_typehints_type:\:py\:class\:\~pandas.DataFrame` :param tree_detail: Raw FAIB tree detail records. :type plot_info: :sphinx_autodoc_typehints_type::py:class:`~pandas.DataFrame` :param plot_info: Plot-level records containing BAF metadata (sample-by-visit or plot header). :type baf: :sphinx_autodoc_typehints_type::py:class:`float` :param baf: Target basal area factor to filter (e.g., 12). :type dbh_col: :sphinx_autodoc_typehints_type::py:class:`str` :param dbh_col: Column containing diameter at breast height in centimetres. :type expansion_col: :sphinx_autodoc_typehints_type::py:class:`str` :param expansion_col: Column representing tree expansion weights. :type group_keys: :sphinx_autodoc_typehints_type::py:class:`tuple`\ \[:py:class:`str`, :py:data:`…<Ellipsis>`]`
:param group_keys: Keys used to join tree detail with sample-by-visit metadata.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~pandas.DataFrame``
.. py:function:: auto_select_bafs(root, count=3, *, plot_file=’faib_plot_header.csv’, sample_file=’faib_sample_byvisit.csv’) :module: nemora.ingest.faib
Select representative BAF values from FAIB metadata.
:type root: :sphinx_autodoc_typehints_type:\:py\:class\:\str` | :py:class:`~pathlib.Path` :param root: Directory containing FAIB CSV extracts. :type count: :sphinx_autodoc_typehints_type::py:class:`int` :param count: Number of representative BAF values to return. :type plot_file: :sphinx_autodoc_typehints_type::py:class:`str` :param plot_file: CSV filenames to inspect for BAF metadata (plot header preferred). :type sample_file: :sphinx_autodoc_typehints_type::py:class:`str``
:param sample_file: CSV filenames to inspect for BAF metadata (plot header preferred).
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\list`\ \[:py:class:`float`]`
.. py:function:: build_faib_dataset_source(dataset=’psp’, *, destination, filenames=None, overwrite=False) :module: nemora.ingest.faib
Create a :class:DatasetSource for FAIB CSV extracts.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.DatasetSource``
.. py:function:: build_faib_stand_table_pipeline(plot_info, *, baf, dbh_col, expansion_col, baf_col, group_keys=(‘CLSTR_ID’, ‘VISIT_NUMBER’, ‘PLOT’)) :module: nemora.ingest.faib
Create a :class:TransformPipeline that aggregates FAIB stand tables.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.TransformPipeline``
.. py:function:: build_stand_table_from_csvs(root, baf, *, tree_file=’faib_tree_detail.csv’, sample_file=’faib_sample_byvisit.csv’, plot_file=None, dbh_col=None, expansion_col=None, baf_col=None) :module: nemora.ingest.faib
Load FAIB CSV extracts from root and build a stand table for baf.
:type root: :sphinx_autodoc_typehints_type:\:py\:class\:\str` | :py:class:`~pathlib.Path` :param root: Directory containing the FAIB CSV extracts. :type baf: :sphinx_autodoc_typehints_type::py:class:`float` :param baf: Desired basal area factor to filter. :type tree_file: :sphinx_autodoc_typehints_type::py:class:`str` :param tree_file: Filename for the tree detail CSV within ``root``. :type sample_file: :sphinx_autodoc_typehints_type::py:class:`str` | :py:obj:`None``
:param sample_file: Filename for the sample-by-visit CSV within root.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~pandas.DataFrame``
.. py:function:: download_faib_csvs(destination, dataset=’psp’, *, overwrite=False, filenames=None) :module: nemora.ingest.faib
Download FAIB CSV extracts via FTP into destination.
:type destination: :sphinx_autodoc_typehints_type:\:py\:class\:\str` | :py:class:`~pathlib.Path` :param destination: Directory where files will be written. :type dataset: :sphinx_autodoc_typehints_type::py:class:`str``
:param dataset: Either "psp" or "non_psp".
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\list`\ \[:py:class:`~pathlib.Path`]`
.. py:function:: generate_faib_manifest(destination, *, dataset=’psp’, source=None, fetch=False, overwrite=False, bafs=None, auto_count=None, max_rows=None, write_parquet=True) :module: nemora.ingest.faib
Fetch FAIB extracts, build stand tables, and emit a manifest.
:type destination: :sphinx_autodoc_typehints_type:\:py\:class\:\str` | :py:class:`~pathlib.Path` :param destination: Directory where the manifest and stand tables will be written. :type dataset: :sphinx_autodoc_typehints_type::py:class:`str` :param dataset: FAIB dataset to process (``"psp"`` or ``"non_psp"``). :type source: :sphinx_autodoc_typehints_type::py:class:`str` | :py:class:`~pathlib.Path` | :py:obj:`None` :param source: Optional directory containing pre-downloaded FAIB CSV files. When omitted, files will be fetched into ``destination / "raw"`` if ``fetch`` is true. :type fetch: :sphinx_autodoc_typehints_type::py:class:`bool` :param fetch: When set, download the FAIB CSV files before building stand tables. :type overwrite: :sphinx_autodoc_typehints_type::py:class:`bool` :param overwrite: Force re-download of CSV files even when they already exist locally. :type bafs: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Sequence`\ \[:py:class:`float`] | :py:obj:`None` :param bafs: Explicit BAF values to build stand tables for. :type auto_count: :sphinx_autodoc_typehints_type::py:class:`int` | :py:obj:`None` :param auto_count: When provided, automatically select ``auto_count`` representative BAF values instead of using ``bafs``. :type max_rows: :sphinx_autodoc_typehints_type::py:class:`int` | :py:obj:`None` :param max_rows: Optional limit on the number of rows retained in each stand table. :type write_parquet: :sphinx_autodoc_typehints_type::py:class:`bool``
:param write_parquet: When true (default), persist a Parquet copy of the manifest alongside the CSV.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.faib.FAIBManifestResult``
.. py:function:: load_data_dictionary(url) :module: nemora.ingest.faib
Download and parse a FAIB data dictionary XLSX file.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.faib.DataDictionary``
.. py:function:: load_non_psp_dictionary() :module: nemora.ingest.faib
Convenience wrapper for the non-PSP data dictionary.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.faib.DataDictionary``
.. py:function:: load_psp_dictionary() :module: nemora.ingest.faib
Convenience wrapper for the PSP data dictionary.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.faib.DataDictionary``
FIA (nemora.ingest.fia)
.. py:module:: nemora.ingest.fia
Helpers for working with USDA FIA datasets.
.. py:class:: FIATables(tree, condition, plot) :module: nemora.ingest.fia
Bases: :py:class:object
Container for FIA plot/condition/tree tables.
.. py:attribute:: FIATables.condition :module: nemora.ingest.fia :type: ~pandas.DataFrame
.. py:attribute:: FIATables.plot :module: nemora.ingest.fia :type: ~pandas.DataFrame
.. py:attribute:: FIATables.tree :module: nemora.ingest.fia :type: ~pandas.DataFrame
.. py:function:: aggregate_plot_stand_table(tables, *, plot_cn=None, plot_number=None, live_status_codes=(1,), condition_status_codes=(1,), dbh_bin_cm=1.0) :module: nemora.ingest.fia
Aggregate FIA tree records into a stand table summarised by DBH bins.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~pandas.DataFrame``
.. py:function:: build_fia_dataset_source(state, *, destination, tables=(‘TREE’, ‘PLOT’, ‘COND’), overwrite=False) :module: nemora.ingest.fia
Create a :class:DatasetSource for downloading FIA CSV extracts.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.DatasetSource``
.. py:function:: build_stand_table_from_csvs(root, *, plot_cn=None, plot_number=None, tree_file=’TREE.csv’, plot_file=’PLOT.csv’, cond_file=’COND.csv’, live_status_codes=(1,), condition_status_codes=(1,), dbh_bin_cm=1.0) :module: nemora.ingest.fia
Convenience wrapper that loads FIA tables and aggregates a stand table.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~pandas.DataFrame``
.. py:function:: download_fia_tables(destination, state, tables=(‘TREE’, ‘PLOT’, ‘COND’), *, overwrite=False) :module: nemora.ingest.fia
Download FIA CSV extracts for a given state.
:type destination: :sphinx_autodoc_typehints_type:\:py\:class\:\str` | :py:class:`~pathlib.Path` :param destination: Directory where the CSV files will be written. :type state: :sphinx_autodoc_typehints_type::py:class:`str` :param state: Two-letter state or territory code (e.g., ``HI``, ``OR``). :type tables: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Sequence`\ \[:py:class:`str`] :param tables: Iterable of table names to download (default: TREE, PLOT, COND). :type overwrite: :sphinx_autodoc_typehints_type::py:class:`bool``
:param overwrite: When True, re-download files even if they already exist locally.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\list`\ \[:py:class:`~pathlib.Path`]`
.. py:function:: load_fia_tables(root, *, tree_file=’TREE.csv’, plot_file=’PLOT.csv’, cond_file=’COND.csv’, columns_tree=None, columns_plot=None, columns_cond=None) :module: nemora.ingest.fia
Load FIA CSV extracts from root and return trimmed dataframes.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.fia.FIATables``
HPS (nemora.ingest.hps)
.. py:module:: nemora.ingest.hps
Pipelines for preparing HPS tallies from FAIB PSP compilations.
.. py:class:: HPSPipelineResult(tallies, manifest, tallies_frame) :module: nemora.ingest.hps
Bases: :py:class:object
Container produced by :func:run_hps_pipeline.
.. py:attribute:: HPSPipelineResult.manifest :module: nemora.ingest.hps :type: ~pandas.DataFrame
.. py:attribute:: HPSPipelineResult.tallies :module: nemora.ingest.hps :type: dict[str, ~pandas.DataFrame]
.. py:attribute:: HPSPipelineResult.tallies_frame :module: nemora.ingest.hps :type: ~pandas.DataFrame
.. py:method:: HPSPipelineResult.to_inventory_release(*, release_id, provenance=None, units=None, metadata=None) :module: nemora.ingest.hps
Return a schema-validated ``nemora.inventory.v1`` release.
:rtype: :sphinx_autodoc_typehints_type:`\:py\:class\:\`\~nemora.ingest.schema.InventoryRelease\``
.. py:class:: SelectionCriteria(first_visit_only=True, allowed_sample_types=None, max_plots=None) :module: nemora.ingest.hps :canonical: nemora.dataprep.hps.SelectionCriteria
Bases: :py:class:object
Options that control which plots are retained from the PSP compilations.
.. py:attribute:: SelectionCriteria.allowed_sample_types :module: nemora.ingest.hps :type: tuple[str, …] | None :value: None
.. py:attribute:: SelectionCriteria.first_visit_only :module: nemora.ingest.hps :type: bool :value: True
.. py:attribute:: SelectionCriteria.max_plots :module: nemora.ingest.hps :type: int | None :value: None
.. py:function:: build_hps_pipeline(tree_detail_source, selections, *, dbh_column=’DBH’, status_column=’LV_D’, live_status=(‘L’,), bin_width=1.0, bin_origin=0.0, chunk_size=200000, encoding=’latin1’) :module: nemora.ingest.hps
Return a pipeline that aggregates HPS tallies for selections.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.TransformPipeline``
.. py:function:: export_hps_inventory(result, output_dir, *, release_id, provenance=None, units=None, metadata=None) :module: nemora.ingest.hps
Write a schema-validated HPS release directory and return it.
This is additive to :func:export_hps_outputs: the legacy per-plot CSVs and
manifest keep their historical bytes, while the canonical release adds
release.json, plots.csv, trees.csv, and species.csv.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.schema.InventoryRelease``
.. py:function:: export_hps_outputs(tallies, manifest, *, output_dir, manifest_path, quiet=False) :module: nemora.ingest.hps
Write per-plot tallies and accompanying manifest to disk.
:rtype: :sphinx_autodoc_typehints_type:\:py\:obj\:\None``
.. py:function:: inventory_release_from_hps(result, *, release_id, provenance=None, units=None, metadata=None) :module: nemora.ingest.hps
Adapt an HPS pipeline result to the shared inventory schema.
HPS tallies are DBH-class aggregates, not individual trees. The canonical
trees table therefore uses a null tree_id, a null species_code,
and the tally count for each DBH class. The plots table carries the HPS
manifest fields.
:type result: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.hps.HPSPipelineResult` :param result: Output of :func:run_hps_pipeline. :type release_id: :sphinx_autodoc_typehints_type::py:class:`str` :param release_id: Stable identifier for the release. :type provenance: :sphinx_autodoc_typehints_type::py:class:`~nemora.ingest.schema.InventoryProvenance` | :py:obj:`None` :param provenance: Release provenance. Defaults to the public FAIB PSP source with a current UTC retrieval timestamp. :type units: :sphinx_autodoc_typehints_type::py:class:`~nemora.ingest.schema.InventoryUnits` | :py:obj:`None` :param units: Canonical units. Defaults to centimetres, metres, counts, and ``m2_per_ha``. :type metadata: :sphinx_autodoc_typehints_type::py:class:`dict`\ \[:py:class:`str`, :py:data:`~typing.Any`] | :py:obj:`None``
:param metadata: Additional JSON-serializable release metadata.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.schema.InventoryRelease``
.. py:function:: load_plot_selections(plot_header_source, sample_byvisit_source, *, baf, criteria=None, encoding=’latin1’) :module: nemora.ingest.hps
Load PSP plot metadata and filter to the subset needed for HPS tallies.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\list`\ \[:py:class:`~nemora.dataprep.hps.PlotSelection`]`
.. py:function:: run_hps_pipeline(tree_detail_source, selections, *, dbh_column=’DBH’, status_column=’LV_D’, live_status=(‘L’,), bin_width=1.0, bin_origin=0.0, chunk_size=200000, encoding=’latin1’) :module: nemora.ingest.hps
Execute the HPS pipeline and return tallies/manifest dataframes.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.hps.HPSPipelineResult``
Inventory schema (nemora.ingest.schema)
.. py:module:: nemora.ingest.schema
Shared tidy inventory schema for Nemora ingest adapters.
The schema describes one release as three tables plus release-level units and provenance:
plots: one row per plot visit;trees: one row per tree or per DBH tally class; andspecies: an optional species-code lookup.
The canonical units are fixed so downstream fitting and sampling do not have to guess. Adapters convert source units before constructing a release.
The schema is intentionally additive: an adapter may keep its historical output
bytes while also emitting a validated nemora.inventory.v1 release.
.. py:class:: InventoryProvenance(source_id, source_version, retrieved_utc, source_uri=None, licence=None, checksum=None, notes=None) :module: nemora.ingest.schema
Bases: :py:class:object
Release-level source and retrieval metadata.
.. py:attribute:: InventoryProvenance.checksum :module: nemora.ingest.schema :type: str | None
.. py:attribute:: InventoryProvenance.licence :module: nemora.ingest.schema :type: str | None
.. py:attribute:: InventoryProvenance.notes :module: nemora.ingest.schema :type: str | None
.. py:attribute:: InventoryProvenance.retrieved_utc :module: nemora.ingest.schema :type: str
.. py:attribute:: InventoryProvenance.source_id :module: nemora.ingest.schema :type: str
.. py:attribute:: InventoryProvenance.source_uri :module: nemora.ingest.schema :type: str | None
.. py:attribute:: InventoryProvenance.source_version :module: nemora.ingest.schema :type: str
.. py:class:: InventoryRelease(release_id, provenance, plots, trees, species=
Bases: :py:class:object
A validated, tidy inventory release.
.. py:attribute:: InventoryRelease.crs :module: nemora.ingest.schema :type: str | None
.. py:attribute:: InventoryRelease.metadata :module: nemora.ingest.schema :type: dict[str, ~typing.Any]
.. py:attribute:: InventoryRelease.plots :module: nemora.ingest.schema :type: ~pandas.DataFrame
.. py:attribute:: InventoryRelease.provenance :module: nemora.ingest.schema :type: ~nemora.ingest.schema.InventoryProvenance
.. py:attribute:: InventoryRelease.release_id :module: nemora.ingest.schema :type: str
.. py:property:: InventoryRelease.schema_version :module: nemora.ingest.schema :type: str
Return the schema version written by this release.
.. py:attribute:: InventoryRelease.species :module: nemora.ingest.schema :type: ~pandas.DataFrame
.. py:method:: InventoryRelease.to_manifest() :module: nemora.ingest.schema
Return the JSON-serializable release manifest.
:rtype: :sphinx_autodoc_typehints_type:`\:py\:class\:\`dict\`\\ \\\[\:py\:class\:\`str\`\, \:py\:data\:\`\~typing.Any\`\]`
.. py:attribute:: InventoryRelease.trees :module: nemora.ingest.schema :type: ~pandas.DataFrame
.. py:attribute:: InventoryRelease.units :module: nemora.ingest.schema :type: ~nemora.ingest.schema.InventoryUnits
.. py:class:: InventoryUnits(dbh=’cm’, height=’m’, tally=’count’, baf=’m2_per_ha’) :module: nemora.ingest.schema
Bases: :py:class:object
Canonical units for the inventory tables.
.. py:attribute:: InventoryUnits.baf :module: nemora.ingest.schema :type: str
.. py:attribute:: InventoryUnits.dbh :module: nemora.ingest.schema :type: str
.. py:attribute:: InventoryUnits.height :module: nemora.ingest.schema :type: str
.. py:attribute:: InventoryUnits.tally :module: nemora.ingest.schema :type: str
.. py:exception:: InventoryValidationError(errors) :module: nemora.ingest.schema
Bases: :py:class:ValueError
Raised when an inventory release does not match the shared schema.
.. py:function:: empty_plots_table() :module: nemora.ingest.schema
Return an empty plots table with the canonical columns.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~pandas.DataFrame``
.. py:function:: empty_species_table() :module: nemora.ingest.schema
Return an empty species lookup with the canonical columns.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~pandas.DataFrame``
.. py:function:: empty_trees_table() :module: nemora.ingest.schema
Return an empty trees table with the canonical columns.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~pandas.DataFrame``
.. py:function:: inventory_schema() :module: nemora.ingest.schema
Return the portable schema document for external consumers.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\dict`\ \[:py:class:`str`, :py:data:`~typing.Any`]`
.. py:function:: read_inventory_release(directory) :module: nemora.ingest.schema
Read and validate a canonical release directory.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.schema.InventoryRelease``
.. py:function:: validate_inventory_release(release) :module: nemora.ingest.schema
Validate a release against nemora.inventory.v1.
:raises InventoryValidationError: If the release id, schema tables, units, or provenance are invalid.
:rtype: :sphinx_autodoc_typehints_type:\:py\:obj\:\None``
.. py:function:: write_inventory_release(release, directory) :module: nemora.ingest.schema
Validate and write a canonical release directory.
The directory contains release.json, plots.csv, trees.csv, and
species.csv. CSV columns are always the full schema, so a minimal input
table still produces a stable artifact.
:rtype: :sphinx_autodoc_typehints_type:\:py\:obj\:\None``
Plot selection (nemora.ingest.selection)
ForestPlotTarget and select_forest_plots apply target/BEC/age/radius filters
to public FAIB tables. assign_holdout_split adds a deterministic split, and
selection_manifest records a location-light summary.
.. py:module:: nemora.ingest.selection
Select public FAIB plots for a research-forest validation target.
The selection is deliberately explicit: a target has a coordinate, radius, BEC codes, projected-age floor, and optional sample-type/first-visit filters. The result keeps plot identity, provenance-relevant metadata, and distance so a selection manifest can be regenerated from public inputs.
.. py:class:: ForestPlotTarget(name, latitude, longitude, radius_km, bec_codes, min_projected_age=60.0, max_projected_age=None, sample_type=’F’, first_visit_only=True, min_distance_km=0.0) :module: nemora.ingest.selection
Bases: :py:class:object
Describe a forest validation target and its plot filters.
:type name: :sphinx_autodoc_typehints_type:\:py\:class\:\str` :param name: Human-readable target name. :type latitude: :sphinx_autodoc_typehints_type::py:class:`float` :param latitude: Target centre in WGS84 decimal degrees. :type longitude: :sphinx_autodoc_typehints_type::py:class:`float` :param longitude: Target centre in WGS84 decimal degrees. :type radius_km: :sphinx_autodoc_typehints_type::py:class:`float` :param radius_km: Great-circle search radius. :type bec_codes: :sphinx_autodoc_typehints_type::py:class:`tuple`\ \[:py:class:`str`, :py:data:`…<Ellipsis>`] :param bec_codes: Accepted BEC codes, such as ``("SBSdw1", "ICHmk3")``. :type min_projected_age: :sphinx_autodoc_typehints_type::py:class:`float` :param min_projected_age: Minimum ``PROJ_AGE_1`` in years. :type sample_type: :sphinx_autodoc_typehints_type::py:class:`str` | :py:obj:`None` :param sample_type: Optional FAIB ``SAMP_TYP`` value to keep. :type first_visit_only: :sphinx_autodoc_typehints_type::py:class:`bool``
:param first_visit_only: Keep only visits marked FIRST_MSMT == "Y".
.. py:attribute:: ForestPlotTarget.bec_codes :module: nemora.ingest.selection :type: tuple[str, …]
.. py:attribute:: ForestPlotTarget.first_visit_only :module: nemora.ingest.selection :type: bool
.. py:attribute:: ForestPlotTarget.latitude :module: nemora.ingest.selection :type: float
.. py:attribute:: ForestPlotTarget.longitude :module: nemora.ingest.selection :type: float
.. py:attribute:: ForestPlotTarget.max_projected_age :module: nemora.ingest.selection :type: float | None
.. py:attribute:: ForestPlotTarget.min_distance_km :module: nemora.ingest.selection :type: float
.. py:attribute:: ForestPlotTarget.min_projected_age :module: nemora.ingest.selection :type: float
.. py:attribute:: ForestPlotTarget.name :module: nemora.ingest.selection :type: str
.. py:attribute:: ForestPlotTarget.radius_km :module: nemora.ingest.selection :type: float
.. py:attribute:: ForestPlotTarget.sample_type :module: nemora.ingest.selection :type: str | None
.. py:function:: assign_holdout_split(plots, *, holdout_fraction=0.25, seed=20261030) :module: nemora.ingest.selection
Assign a deterministic train/hold-out split.
The split uses a SHA-256 key over seed and plot_id. It is stable
across machines and Python versions and does not depend on row order.
:type plots: :sphinx_autodoc_typehints_type:\:py\:class\:\~pandas.DataFrame` :param plots: Selected plot frame with a ``plot_id`` column. :type holdout_fraction: :sphinx_autodoc_typehints_type::py:class:`float` :param holdout_fraction: Fraction of plots assigned to the hold-out set. :type seed: :sphinx_autodoc_typehints_type::py:class:`int``
:param seed: Integer split seed.
:returns: A copy with a split column containing train or holdout.
:rtype: pandas.DataFrame
:raises ValueError: If there are fewer than two plots or the fraction is outside (0, 1).
.. py:function:: resolve_tree_detail(tree_detail_source, selected_plots, *, live_only=True, chunk_size=200000) :module: nemora.ingest.selection
Resolve FAIB tree detail for selected plots into canonical tree rows.
:type tree_detail_source: :sphinx_autodoc_typehints_type:\:py\:class\:\str` | :py:class:`~pathlib.Path` | :py:class:`~pandas.DataFrame` :param tree_detail_source: Path to ``faib_tree_detail.csv`` or an equivalent DataFrame. :type selected_plots: :sphinx_autodoc_typehints_type::py:class:`~pandas.DataFrame` :param selected_plots: Output of :func:select_forest_plots (or a frame with the canonical ``plot_id``, ``cluster_id``, ``visit_number``, and ``plot_number`` columns). :type live_only: :sphinx_autodoc_typehints_type::py:class:`bool` :param live_only: Keep only ``LV_D == "L"`` records. When false, dead records are kept with ``status="dead"``. :type chunk_size: :sphinx_autodoc_typehints_type::py:class:`int``
:param chunk_size: CSV chunk size for streaming reads.
:returns: Canonical nemora.inventory.v1 tree columns only.
:rtype: pandas.DataFrame
:raises KeyError: If required selection or tree columns are missing.
:raises ValueError: If chunk_size is not positive.
.. py:function:: select_forest_plots(site_header, plot_header, sample_by_visit, *, target) :module: nemora.ingest.selection
Select FAIB plots for one target.
:type site_header: :sphinx_autodoc_typehints_type:\:py\:class\:\~pandas.DataFrame` :param site_header: FAIB site-level table with coordinates and BEC fields. :type plot_header: :sphinx_autodoc_typehints_type::py:class:`~pandas.DataFrame` :param plot_header: FAIB plot-level table with cluster, visit, and plot numbers. :type sample_by_visit: :sphinx_autodoc_typehints_type::py:class:`~pandas.DataFrame` :param sample_by_visit: FAIB visit-level table with first-measurement, date, age, and sample type. :type target: :sphinx_autodoc_typehints_type::py:class:`~nemora.ingest.selection.ForestPlotTarget``
:param target: Target and filter definition.
:returns: One row per selected plot with canonical identity, distance, BEC, age, and provenance-relevant metadata. The frame is sorted by distance then plot id. :rtype: pandas.DataFrame
:raises KeyError: If a required source column is missing. :raises ValueError: If no plots match.
.. py:function:: selection_manifest(selected, *, target, holdout_fraction, split_seed) :module: nemora.ingest.selection
Build a location-light manifest for a selected plot set.
:rtype: :sphinx_autodoc_typehints_type:\:py\:class\:\dict`\ \[:py:class:`str`, :py:data:`~typing.Any`]`
.. py:function:: write_selection_manifest(manifest, path) :module: nemora.ingest.selection
Write a selection manifest as stable, indented JSON.
:rtype: :sphinx_autodoc_typehints_type:\:py\:obj\:\None``
Plot resolution (nemora.ingest.plots)
PlotInventorySpec, build_inventory_release, and
materialize_tally_trees map source columns into a validated release and
optionally change the tree grain from DBH tally classes to individual trees.
.. py:module:: nemora.ingest.plots
Resolve mapped inventory tables into canonical Nemora releases.
This module is the generic part of the ingest boundary. Source-specific adapters
still own downloading, unit conversion, and source validation; this layer owns
the common mapping into :mod:nemora.ingest.schema, optional tree-count
derivation, and opt-in materialization of tally classes into individual trees.
.. py:class:: PlotInventorySpec(release_id, provenance, plot_columns=
Bases: :py:class:object
Describe how source tables map to the canonical inventory schema.
Column mappings use canonical names as keys and source-column names as
values. A canonical name that already exists in the source frame is used
without a mapping entry. Missing optional columns are filled from the
corresponding defaults or with NA.
:type release_id: :sphinx_autodoc_typehints_type:\:py\:class\:\str` :param release_id: Stable identifier written to the release manifest. :type provenance: :sphinx_autodoc_typehints_type::py:class:`~nemora.ingest.schema.InventoryProvenance` :param provenance: Source and retrieval metadata required by ``nemora.inventory.v1``. :type plot_columns: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Mapping`\ \[:py:class:`str`, :py:class:`str`] :param plot_columns: Mapping from canonical plot columns to source column names. :type tree_columns: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Mapping`\ \[:py:class:`str`, :py:class:`str`] :param tree_columns: Mapping from canonical tree columns to source column names. :type species_columns: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Mapping`\ \[:py:class:`str`, :py:class:`str`] :param species_columns: Mapping from canonical species columns to source column names. :type plot_defaults: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Mapping`\ \[:py:class:`str`, :py:class:`object`] :param plot_defaults: Values used when a canonical plot column is absent. :type tree_defaults: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Mapping`\ \[:py:class:`str`, :py:class:`object`] :param tree_defaults: Values used when a canonical tree column is absent. :type species_defaults: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Mapping`\ \[:py:class:`str`, :py:class:`object`] :param species_defaults: Values used when a canonical species column is absent. :type units: :sphinx_autodoc_typehints_type::py:class:`~nemora.ingest.schema.InventoryUnits` :param units: Canonical release units. :type crs: :sphinx_autodoc_typehints_type::py:class:`str` | :py:obj:`None` :param crs: CRS for plot coordinates, when present. :type metadata: :sphinx_autodoc_typehints_type::py:class:`~collections.abc.Mapping`\ \[:py:class:`str`, :py:class:`object`] :param metadata: Additional JSON-serializable release metadata. :type derive_tree_count: :sphinx_autodoc_typehints_type::py:class:`bool``
:param derive_tree_count: Fill a missing plots.tree_count with the number of tree rows per
plot. This is a row count, not a sum of expansion tallies.
.. py:attribute:: PlotInventorySpec.crs :module: nemora.ingest.plots :type: str | None
.. py:attribute:: PlotInventorySpec.derive_tree_count :module: nemora.ingest.plots :type: bool
.. py:attribute:: PlotInventorySpec.metadata :module: nemora.ingest.plots :type: ~collections.abc.Mapping[str, object]
.. py:attribute:: PlotInventorySpec.plot_columns :module: nemora.ingest.plots :type: ~collections.abc.Mapping[str, str]
.. py:attribute:: PlotInventorySpec.plot_defaults :module: nemora.ingest.plots :type: ~collections.abc.Mapping[str, object]
.. py:attribute:: PlotInventorySpec.provenance :module: nemora.ingest.plots :type: ~nemora.ingest.schema.InventoryProvenance
.. py:attribute:: PlotInventorySpec.release_id :module: nemora.ingest.plots :type: str
.. py:attribute:: PlotInventorySpec.species_columns :module: nemora.ingest.plots :type: ~collections.abc.Mapping[str, str]
.. py:attribute:: PlotInventorySpec.species_defaults :module: nemora.ingest.plots :type: ~collections.abc.Mapping[str, object]
.. py:attribute:: PlotInventorySpec.tree_columns :module: nemora.ingest.plots :type: ~collections.abc.Mapping[str, str]
.. py:attribute:: PlotInventorySpec.tree_defaults :module: nemora.ingest.plots :type: ~collections.abc.Mapping[str, object]
.. py:attribute:: PlotInventorySpec.units :module: nemora.ingest.plots :type: ~nemora.ingest.schema.InventoryUnits
.. py:function:: build_inventory_release(plots, trees, *, spec, species=None) :module: nemora.ingest.plots
Resolve source tables into a validated nemora.inventory.v1 release.
:type plots: :sphinx_autodoc_typehints_type:\:py\:class\:\~pandas.DataFrame` :param plots: Source plot table. It must provide, or map to, ``plot_id``. :type trees: :sphinx_autodoc_typehints_type::py:class:`~pandas.DataFrame` :param trees: Source tree or tally-class table. It must provide, or map to, ``plot_id``, ``dbh_cm``, and ``tally``. :type spec: :sphinx_autodoc_typehints_type::py:class:`~nemora.ingest.plots.PlotInventorySpec` :param spec: Release metadata, column mappings, and defaults. :type species: :sphinx_autodoc_typehints_type::py:class:`~pandas.DataFrame` | :py:obj:`None``
:param species: Optional species lookup table.
:returns: A validated canonical release. :rtype: InventoryRelease
:raises InventoryValidationError: If a mapping names an unknown source column or the resolved release is invalid.
.. py:function:: materialize_tally_trees(release, *, tree_id_prefix=’T’) :module: nemora.ingest.plots
Expand tally classes into individual tree rows with stable IDs.
Rows that already have a tree_id are preserved. Rows without one are
expanded into tally copies with tally=1 and deterministic
<prefix><plot_id>-<sequence> identifiers. Fractional tallies are
rejected because materializing them requires an explicit expansion rule.
:type release: :sphinx_autodoc_typehints_type:\:py\:class\:\~nemora.ingest.schema.InventoryRelease` :param release: A validated ``nemora.inventory.v1`` release. :type tree_id_prefix: :sphinx_autodoc_typehints_type::py:class:`str``
:param tree_id_prefix: Prefix for generated tree identifiers.
:returns: A validated release at the individual-tree grain. :rtype: InventoryRelease
:raises InventoryValidationError: If the input release is invalid or a tally is not a non-negative integer.