Behaviour for backends that start a claimed import run.
A runner is the thing that decides where GeoGenius.Pipeline.execute/3
actually runs -- in the calling process, in a supervised Task, or handed
to a durable job framework such as PgFlow. It owns none of the run's state:
enqueue/3 starts the work and returns, and the run's progress, status, and
error all live in PostgreSQL -- readable through the catalog regardless of
which backend started it.
No status or await callback. Putting one here would mean one query
implemented once per backend, and would make a run started under one
backend unreadable after the configuration changed -- the exact failure
import_run.runner_backend exists to prevent by recording which backend a
run was claimed under.
A runner must not call GeoGenius.Catalog.fail_import/4. The pipeline
owns the failure path end to end, including the case where its own cleanup
or confirmation read fails after a phase already recorded an outcome.
A runner calling it after the pipeline already finished raises 55000 rather
than recording anything, because fail_import/4 refuses a completed run --
so the call is not a way to report a late failure, only a way to crash the
backend. A backend that definitely refuses the work before acceptance
returns {:error, {:not_enqueued, reason}}. Once acceptance may have
happened, it returns {:error, {:outcome_unknown, reason}} instead. The
import lifecycle caller terminalizes only a definite rejection of a newly
registered or retried attempt.
Summary
Types
Job arguments, JSON-serializable so a durable backend can round-trip them
through storage. Carries :publish and :stale_after_seconds; a backend
reads each with its own reader (publish?/1, stale_after_seconds/1)
rather than Map.get(args, :publish, false), since a backend that decodes
its own JSON hands back %{"publish" => true, "stale_after_seconds" => 300},
not atom keys.
The certainty with which a failed enqueue reports durable acceptance.
The reason a backend could not start a run, or could not be resolved.
Callbacks
Reports whether this backend can accept work right now.
Starts a claimed run.
The name this backend is recorded under in import_run.runner_backend.
Functions
Resolves the runner backend for one call.
Resolves a shipped backend's name/0 back to its module.
Reads :publish out of args, defaulting to false.
Reads :stale_after_seconds out of args, defaulting to nil.
Types
@type args() :: %{ optional(:publish) => boolean(), optional(:stale_after_seconds) => pos_integer() }
Job arguments, JSON-serializable so a durable backend can round-trip them
through storage. Carries :publish and :stale_after_seconds; a backend
reads each with its own reader (publish?/1, stale_after_seconds/1)
rather than Map.get(args, :publish, false), since a backend that decodes
its own JSON hands back %{"publish" => true, "stale_after_seconds" => 300},
not atom keys.
The certainty with which a failed enqueue reports durable acceptance.
@type reason() :: term()
The reason a backend could not start a run, or could not be resolved.
Callbacks
@callback available?() :: boolean()
Reports whether this backend can accept work right now.
@callback enqueue(GeoGenius.Context.t(), run_id :: Ecto.UUID.t(), args()) :: :ok | {:error, enqueue_error()}
Starts a claimed run.
Returns :ok when the run's outcome is, or will be, recorded in the
catalog -- including a run that has already failed and recorded that
failure. A definite pre-acceptance refusal returns
{:error, {:not_enqueued, reason}}; an error after durable acceptance may
have happened returns {:error, {:outcome_unknown, reason}}. Never return
an error for a failure the pipeline itself recorded -- Pipeline.execute/3
already wrote it, and the catalog is the only place a caller should look for
it.
For compatibility, GeoGenius.import/1 still interprets the callback's
historical plain {:error, reason} shape as :not_enqueued. That contract
documented a plain error as a definite refusal; new runners should always
return an explicit certainty tuple.
@callback name() :: String.t()
The name this backend is recorded under in import_run.runner_backend.
Functions
Resolves the runner backend for one call.
Resolution order: opts[:runner], then config :geo_genius, :runner, then
the first of Runners.Task and Runners.Inline whose available?/0 is
true. PgFlow is intentionally explicit: its finite job deadline is a host
policy for bounded work, not a safe inference merely because a PgFlow
engine happens to be running for unrelated workflows. Pin
GeoGenius.Runners.PgFlow per call or in application configuration after
choosing a deadline suitable for the imported data.
Runners.Task is selected
whenever config :geo_genius, :task_supervisor names anything at all,
even a name that is not currently a live process. Setting that key is a
host committing to Runners.Task, and enqueue/3 already turns a dead
supervisor into a named {:error, {:not_enqueued, reason}} rather than an exit (see
Runners.Task's moduledoc), so selecting it regardless of the configured
name's current liveness surfaces a typo'd or not-yet-started supervisor as
that named error instead of silently falling through to Runners.Inline
and blocking the caller for the length of an import -- the worst available
outcome, and what a merely-configured-but-dead :task_supervisor did
before this exception existed. Runners.Inline is always available, so
the chain always terminates -- a host that installed neither PgFlow nor
configured :task_supervisor still gets a working runner, and with
nothing configured at all Runners.Task is still reached because
GeoGenius.Application starts a Task.Supervisor this package owns and
available?/0 finds it genuinely running.
An explicit opts[:runner] or config :geo_genius, :runner is validated
before it is returned: it must load and implement every GeoGenius.Runner
callback, or this raises ArgumentError naming what is missing, rather
than letting a typo surface as an UndefinedFunctionError several phases
into a run. A module reached through the availability chain is one of this
package's own, already proven to implement the behaviour, so it is not
re-checked.
Resolves a shipped backend's name/0 back to its module.
Searches only the shipped backends (Runners.PgFlow, Runners.Task,
Runners.Inline). A host's own runner is not resolvable here -- a caller
that stamps import_run.runner_backend with a host runner's name is
responsible for resolving that name back to a module itself.
A backend that has not loaded -- Runners.PgFlow before its optional
dependency is installed -- is treated as unknown rather than probed, since
calling name/0 on it would raise.
Reads :publish out of args, defaulting to false.
Accepts both %{publish: boolean()} and %{"publish" => boolean()}, so a
backend that round-trips args through its own JSON decoder -- handing
back string keys -- still resolves the flag a caller actually set, rather
than silently treating every job as publish: false.
@spec stale_after_seconds(args() | map()) :: pos_integer() | nil
Reads :stale_after_seconds out of args, defaulting to nil.
Accepts both %{stale_after_seconds: n} and %{"stale_after_seconds" => n},
the same round-trip publish?/1 handles. nil tells the caller no window
was carried -- GeoGenius.Pipeline falls back to its own default rather
than treating nil as a zero-second window.