GeoGenius.Providers.Delimited (geo_genius v0.2.0)

Copy Markdown View Source

Streams a delimited artifact in chunks, reading its first line as the header row and handing each data line to the provider's own row builder.

Sits directly above GeoGenius.Providers.Batch, which turns one chunk's data lines into staged rows: this is the header-tracking accumulator around it -- :pending until the header line is read, {:headers, headers} from then on -- plus the chunking and the emit call. GeoGenius.Providers.CSV and GeoGenius.Providers.SimpleMaps both read that shape of file and differ only in the parser they choose and in what each builds from a line, so both supply those two things and share everything else. A third delimited provider supplies them too rather than copying either.

Whatever File.stream!/1 and the parser raise is left to propagate: each provider rescues those around its own call and reports them in the vocabulary of the format it documents.

Summary

Types

row_builder()

Builds one staged row from the file's header names and one data line's values.

Functions

stage(path, parser, chunk_size, emit, row_builder)

Parses path with parser, emitting the staged rows of each chunk_size lines in one emit call.

Types

row_builder()

@type row_builder() :: ([String.t()], [String.t()] ->
                    {:ok, GeoGenius.Staging.Row.t()} | {:error, term()})

Builds one staged row from the file's header names and one data line's values.

Functions

stage(path, parser, chunk_size, emit, row_builder)

@spec stage(
  Path.t(),
  module(),
  pos_integer(),
  ([GeoGenius.Staging.Row.t()] -> :ok),
  row_builder()
) ::
  :ok | {:error, term()}

Parses path with parser, emitting the staged rows of each chunk_size lines in one emit call.

Returns the first {:error, reason} row_builder produces, staging none of the lines after it, and :ok for a file every line of which built a row -- including one carrying nothing but a header.