# `GeoGenius.Provider`
[🔗](https://github.com/agoodway/geo_genius/blob/main/lib/geo_genius/provider.ex#L108)

Behaviour a format adapter implements to bring one kind of source into the catalog.

A provider is called for its parsing, never for a write: `stage/5` receives
a file path and an `emit` function and reports only whether it succeeded;
`normalize/2` receives one staged row and returns a described area. Every
write is the pipeline's, made through `GeoGenius.Catalog`. That is the
contract this behaviour expresses -- callers should not hand a provider a
`Repo` or a `GeoGenius.Context`, and a provider that follows this
behaviour never needs one. Elixir has no module-level enforcement of that,
so it is a discipline the pipeline and the tests hold providers to, not a
guarantee this module itself can make.

`emit`, passed to `stage/5`, is called with a list of rows rather than a
single row so a provider can batch; the pipeline supplies a function that
writes through `GeoGenius.Staging.insert/4` and heartbeats the run. A
provider stages a large artifact in chunks rather than one `emit` call for
the whole file, both to bound the size of a single insert and because each
`emit` call is also where the run's lease gets renewed.

`stage/5` also takes an `opts` keyword list, which carries adapter modules
resolved by the pipeline -- `[command: ..., work_dir: ...]` for a provider
that shells out to convert its source first. The `:command` a provider is
handed is `GeoGenius.Pipeline.CommandAllowlist`, which accepts `ogr2ogr` and
refuses everything else; it reads the adapter it wraps from
`:command_target`, also in `opts`, and a provider passing `opts` through to
`command.run/3` unchanged (as `GeoGenius.Providers.Shapefile` does) carries
that along without needing to know it is there. Resolving an adapter inside a
provider would make it impossible to override per call and would give a
provider a reason to read application environment, which nothing else in
this layer does. A provider that needs neither entry in `opts` ignores it.
`normalize/2` takes no such list: it is pure, so a caller derives an area
from a staged row with no environment and no adapter to substitute.

`artifacts/1`, `relations/1`, and `asserted_relations/2` have one body
most providers share, so those bodies live here as `all_artifacts/1`,
`always_rebuild/1`, and `no_asserted_relations/2`; a provider implements
the callback with a one-line `defdelegate`, keeping its own `@impl` and
`@doc`. A provider whose relations need none rebuilt, or whose rows carry
a hierarchy in their columns, implements the callback directly instead of
delegating -- `GeoGenius.Providers.SimpleMaps` does both.

## Option vocabulary across providers

Each shipped provider names its manifest `options` keys for the shape its
format actually carries -- a GeoJSON feature has properties, a CSV row has
columns -- rather than a shared, format-neutral name that would read
wrong on the page for one format or the other. The concepts underneath are
the same one; this table is the one place to learn the correspondence:

| Concept                    | `GeoGenius.Providers.GeoJSON` | `GeoGenius.Providers.CSV` | Default when omitted |
|-----------------------------|-------------------------------|----------------------------|------------------------|
| the area's code              | `"code_property"`             | `"code_column"`            | required, no default   |
| the area's area type          | `"area_type"` (shared key)     | `"area_type"` (shared key)  | required, no default   |
| the area's official name     | `"name_property"`             | `"name_column"`            | `"name"`                |
| alias names                  | `"alias_properties"`          | `"alias_columns"`          | `[]`                    |
| free-form attributes         | `"attribute_properties"`      | `"attribute_columns"`      | `[]`                    |
| external codes               | `"code_properties"`           | `"code_columns"`           | `[]`                    |
| authority                    | `"authority"` (shared key)     | `"authority"` (shared key)  | the manifest's own single authority |
| delimiter                    | n/a                            | `"delimiter"`               | `","` (CSV only)        |
| parent areas the row implies | `"implied_areas"` (shared key) | `"implied_areas"` (shared key) | `[]`                  |

`"implied_areas"` is a list of entries, and each entry names its own code
key in the calling provider's suffix: `"code_property"` under
`GeoGenius.Providers.GeoJSON`, `"code_column"` under
`GeoGenius.Providers.CSV`. An entry also names an `"area_type"`, a `"names"`
map from code to display name, an optional `"authority"`, and an optional
`"relation"` defaulting to `"contains"`. A row whose code column carries
`"1"` under

    %{"area_type" => "cluster", "code_property" => "CLUSTER",
      "names" => %{"1" => "Outer Cluster", "2" => "Inner Cluster"}}

implies the area `cluster:1` and asserts `contains` from it to the row's own
area. Both generic providers validate these entries at manifest load through
`c:validate_options/1`, so an entry written in the other's vocabulary is
rejected before anything is downloaded. See
`GeoGenius.Providers.ImpliedAreas`.

`GeoGenius.Providers.Shapefile` reads the GeoJSON vocabulary, since it
converts its archive before parsing it. `GeoGenius.Providers.SimpleMaps` is
outside this table: it parses two fixed files and reads their columns by
name. It requires no options, but accepts `"non_census_state_area_type"` so
the six USPS-only state codes can use a separately declared non-geometric
type while ordinary Census states keep `state`.

Every string value this vocabulary reads out of a payload -- a code, a
name, an attribute, an external code -- is trimmed before use, and a
whitespace-only value is treated the same as an absent one: see
`GeoGenius.Providers.Fields.presence/1`. A fixed-width source that pads a
FIPS or ZIP code with spaces would otherwise carry that padding straight
into `area_key`, a public, stable identifier, and a later re-import of the
same source without the padding would create a second area rather than
updating the first.

# `reason`

```elixir
@type reason() :: String.t()
```

The failure a provider reports for a file it cannot parse or a manifest option it cannot resolve.

# `stage_opts`

```elixir
@type stage_opts() :: [
  command: module(),
  command_target: module(),
  work_dir: Path.t()
]
```

The adapters `stage/5` may use, resolved by the pipeline rather than read from application environment.

# `artifacts`

```elixir
@callback artifacts(manifest :: GeoGenius.Manifest.t()) :: [
  GeoGenius.Manifest.Artifact.t()
]
```

The artifacts, across a manifest's sources, this provider will stage.

The pipeline asks this once per source, with the manifest narrowed to that
one source, so a provider delegating to `all_artifacts/1` answers with its
own source's artifacts. A release may name a different provider per source,
and each logical name is staged by the provider of the source that declared
it.

# `asserted_relations`

```elixir
@callback asserted_relations(
  manifest :: GeoGenius.Manifest.t(),
  row :: GeoGenius.Staging.Row.t()
) :: [
  {parent_area_key :: String.t(), child_area_key :: String.t(),
   relation_type :: String.t()}
]
```

Relation edges this row asserts, which geometry cannot measure.

A source that carries its hierarchy in columns -- a county FIPS on every
city row, an admin parent code on every place -- knows edges no overlap
test can derive, and a source with no geometry at all has no other way to
express one. Each edge is `{parent_area_key, child_area_key,
relation_type}`, where `relation_type` is `"contains"`,
`"mostly_contains"`, or `"overlaps"`.

Both keys are the strings `GeoGenius.Provider.Area.key/1` composes. Build
the `%GeoGenius.Provider.Area{}` an edge names and take its key from there,
rather than joining the three parts by hand: an edge composed a second way
can name a key `normalize/2` never produced, and `GeoGenius.Catalog.put_relation/4`
refuses an area the release does not carry.

Composes with `relations/1` rather than replacing it: a provider whose
areas nest spatially may both rebuild measured relations and assert edges
the geometry does not carry. Edges are written after every area exists, so
an edge may reference an area any row emitted, not only its own.

Asserting an edge for a pair `relations/1` also rebuilds from geometry
overwrites that measurement: `GeoGenius.Catalog.put_relation/4` upserts on
the same `(parent_area_id, child_area_id)` pair, nulling
`intersection_area_m2`, `parent_coverage`, and `child_coverage` and
replacing the geometry-classified `relation_type` with the asserted one,
with no warning that a measurement was discarded. A provider returning
`:rebuild` should assert only pairs geometry cannot derive, keeping the two
sets of edges disjoint.

# `normalize`

```elixir
@callback normalize(manifest :: GeoGenius.Manifest.t(), row :: GeoGenius.Staging.Row.t()) ::
  {:ok, GeoGenius.Provider.Area.t()}
  | {:ok, [GeoGenius.Provider.Area.t()]}
  | :skip
  | {:error, reason()}
```

Describes one staged row as an area, or as several.

Returns `:skip` for a row that carries no area -- a header, a summary
record, a feature with no usable code -- rather than an error, since that
is an expected shape for some rows rather than a failure.

A list is for a source that denormalises a hierarchy into every row: a
city row that also carries its county and state describes all three, and
returning them together is cheaper and truer than staging the same row
once per level. Areas repeated across rows converge on `area_key`, so a
provider emits an implied parent from every row that implies it rather
than tracking which it has already emitted.

# `relations`

```elixir
@callback relations(manifest :: GeoGenius.Manifest.t()) :: :rebuild | :none
```

Whether the areas this provider stages need `parent_area_id` relations
rebuilt after normalization.

`:rebuild` for a provider whose areas nest spatially or by code; `:none`
for a provider whose collection carries no hierarchy.

# `required_options`

```elixir
@callback required_options() :: [String.t()]
```

The manifest `options` keys this provider requires.

Manifest validation checks that every key named here is present in a
manifest's `options` map before the manifest is accepted.

# `stage`

```elixir
@callback stage(
  manifest :: GeoGenius.Manifest.t(),
  artifact :: GeoGenius.Manifest.Artifact.t(),
  path :: Path.t(),
  emit :: ([GeoGenius.Staging.Row.t()] -> :ok),
  opts :: stage_opts()
) :: :ok | {:error, reason()}
```

Parses `path` -- the downloaded or operator-supplied copy of `artifact` --
and hands each resulting batch of rows to `emit`.

`opts` carries the adapters described above. Returns `{:error, reason}`
for a file this provider cannot parse; never raises for a malformed file,
only for a defect in the provider itself.

# `validate_manifest`
*optional* 

```elixir
@callback validate_manifest(manifest :: GeoGenius.Manifest.t()) ::
  :ok | {:error, reason()}
```

Validates relationships between fields of a fully decoded manifest.

Optional. `GeoGenius.Manifest` calls this after its own structural checks and
`c:validate_options/1`, so a provider can enforce a policy spanning options,
area types, authorities, or sources without putting source-specific rules in
the manifest parser. Every provider named by the release or one of its
sources receives the same complete manifest.

# `validate_options`
*optional* 

```elixir
@callback validate_options(options :: map()) :: :ok | {:error, reason()}
```

Validates the manifest `options` map as a whole, beyond the presence of the
keys `required_options/0` names.

Optional. A provider that exports it is called at manifest load, once every
required key is known to be present, and an error it returns becomes a
`GeoGenius.ManifestError` naming the field. That is what makes a malformed
option fail before a release row exists, rather than partway through
normalizing an artifact already downloaded and staged.

This is where a provider checks the options only it understands. The option
vocabulary is per-provider, so the same document can be valid under one
provider and invalid under another: `GeoGenius.Providers.GeoJSON` reads an
`implied_areas` entry's code key as `"code_property"` and
`GeoGenius.Providers.CSV` reads it as `"code_column"`, and neither accepts
the other's.

A provider still validates what it reads at the point it reads it. A
`%GeoGenius.Manifest{}` built in Elixir never passes through
`GeoGenius.Manifest.from_map/2`, so this is a gate on loaded documents, not a
guarantee `normalize/2` may rely on.

# `all_artifacts`

```elixir
@spec all_artifacts(GeoGenius.Manifest.t()) :: [GeoGenius.Manifest.Artifact.t()]
```

Every artifact declared across a manifest's sources.

# `always_rebuild`

```elixir
@spec always_rebuild(GeoGenius.Manifest.t()) :: :rebuild
```

Always rebuilds relations; the collection carries no hierarchy of its own to preserve.

# `no_asserted_relations`

```elixir
@spec no_asserted_relations(GeoGenius.Manifest.t(), GeoGenius.Staging.Row.t()) :: []
```

Asserts no relations; the source carries no hierarchy in its columns.

---

*Consult [api-reference.md](api-reference.md) for complete listing*