Revision history for LLM::Data::Inference
0.10.0 2026-08-21T22:44:11+01:00
- LLM::Chat pin: 0.8.0 exact -> 0.10.0+. The exact pin named the one
release whose META was broken on fresh installs (provides-dropped
Response::OpenRouter modules resolving to the fossil
LLM::Chat::Backend 0.2.1 dist); it would have kept resolving to it
even after the fix ships. Raised to the released 0.10.0 rather than
the first fixed 0.8.1 so a fresh install gets the streaming
terminal-state and tool-call assembly fixes too.
- Roaring::Tags and CRoaring pins raised to their released 0.2.3s.
0.9.0 2026-08-09T20:28:57+01:00
- [2026-08-09] The retry/fallback policy this distribution
pioneered now lives in C (LLM::Chat 0.8.0) so a
second executor can share it instead of copying it, and Task runs
on the shared version. Nothing about its behaviour changed: the
bucket table, the backoff curve (C<2 ** (n - 1)> plus 0..0.5 s of
jitter, capped at 30 s), the chunked cancel-aware sleep, the
attempt-record shape (C key ABSENT rather than
undefined when there is no body) and the telemetry payload's
presence-gating are the same code, moved. The public
C method is B and delegates — a caller
holding a Task should not have to reach for another distribution
to ask "what would you do with this error?". Every one of this
distribution's thirteen pre-existing test files passes unedited,
which is the migration's acceptance criterion.
- [2026-08-09] C, C,
C and C are now B of the shared
C types, so BOTH hierarchies match the
same throw: existing C
handlers keep firing byte-for-byte identically, while an app that
runs a Task alongside another executor can write one generic
C arm instead of two.
C still declares C C (inherited
from its shared twin), C and C still
abstain, C is still deliberately NOT an exhaustion in
either hierarchy, and every C<.message> string — including
"LLM::Data::Inference::Task: cancelled by caller after N recorded
attempt(s)" — is unchanged. The C advice
contract's canonical documentation moved to
C along with the types; the section
here now points at it. New C
pins the whole compatibility promise, against constructed
exceptions and against exceptions a real Task throws.
- [2026-07-29] Typed deadline exhaustion. A chain that ran out of
time died as a bare C whose
summary named no lever at all: a review step that hit its 300 s
deadline three times in a row reported only "all 1 backend(s)
exhausted" plus the raw per-attempt error, so nothing told the
operator that the fix was a bigger deadline (or a faster
backend) rather than a better model. Task now throws the new
C when at least one attempt
in the exhausted chain died on a response deadline — its own
C<$.timeout> firing mid-poll, or a backend reporting error
class 'timeout'. Like C it is a SUBCLASS of
C, so DLQ routing, C<:on-exhausted> hooks and every
existing C handler keep working unchanged,
while a caller that wants to distinguish "raise the deadline"
from "this model cannot do the task" can match it first. The
summary gains a lever line quoting the deadline that was in
force ("At least one attempt hit the 300-second response
deadline before completing: raise the task timeout, use a
faster backend, or shrink the response."), and a chain that
truncated AND timed out carries BOTH lever lines — they name
different fixes. Precedence for the thrown TYPE is
C > C > C: truncation is the
more specific diagnosis and the deterministic one. Summaries
of chains that hit neither pathology are byte-identical to
before.
- [2026-07-29] C now declares
C Bool:D) { False }>. A truncation is
deterministic — C is per-backend C state,
so an orchestration layer that blindly re-runs the failed item
re-issues the identical request and gets cut off in exactly the
same place, burning the whole item budget on the way to the same
dead-letter record. The method is duck-typed ADVICE in both
directions: nothing here reads it, consumers probe with C<.?>
and default to True when it is absent, and neither side imports
the other's types — so every other exception (including
C and plain C, which deliberately abstain)
keeps its full attempt budget. See the C section
of the L Pod.
0.7.0 2026-07-27T15:25:50+01:00
- [2026-07-25] Cancellation and deadline aborts now route through
C instead of C. Closing the
Response only quits the local supplier; the upstream stop signal
lives in the backend override, so against KoboldCpp a cancelled
Task now fires C and the server actually
stops generating. Previously every Task-mediated job (reconcile,
narrative-state, downstream-projection, image prompt-gen,
scene-shakeup, relationship-mode in App::Cantina) looked
cancelled to the caller while the backend drained the full
completion. Backends with no upstream abort endpoint degrade to
exactly the previous local close, and a backend whose C
throws falls back to the local close so the poll loop can never
spin on a live response. Both the C branch and the
C<$.timeout> branch are covered.
- [2026-07-27] Typed truncation. A blocking completion cut off by
C comes back as an HTTP 200 SUCCESS with a partial
body and C 'length' — the "advance on
finish-reason quit" rule in the docs was only ever true of the
STREAMING path. Task therefore handed half-finished output to
the parser, watched it fail, and (with C<:parse-retries> set)
re-rolled the identical over-budget request until the budget
was gone. Task and JSONTask now take
C<:truncation-policy('fail' | 'accept' | '')>: 'fail' records
the truncation and advances to the next backend at once WITHOUT
consuming a parse re-roll (C is per-backend
C, so only a different backend can change it);
'accept' is the historical behaviour; the default C<''> derives
'fail' when a parser is installed and 'accept' when one is not,
so JSON tasks fail fast while prose tasks (continuations, style
passes) keep their partial output bit-for-bit as before. The
check runs BEFORE the empty-body rule, so a reasoning model that
burns its whole budget inside C<< >> is diagnosed as
truncation rather than "empty response body". A chain that
truncates throws the new
C — a SUBCLASS of
C, so DLQ routing, C<:on-exhausted> hooks and every
existing C handler keep working unchanged — and
its summary names the levers (raise C, add a
larger-budget fallback, or shrink the request). Failed responses
also keep any text they emitted before quitting, so partial
streamed output is capturable instead of dropped. Requires
LLM::Chat 0.7.1 for the Mock C knob the
tests drive this with.
- [2026-07-27] Add C<:retry-feedback> to Task and JSONTask (default
False). A parse re-roll used to be blind — identical prompt,
identical model, no word about what was wrong with the answer
just rejected. That is survivable for malformed JSON on a
stochastic sampler, but hopeless for a SEMANTIC rejection: a
verbatim-quote validator that says "quote 2 does not appear in
the passage" says exactly the same thing on every re-roll, so
the item burned its whole budget failing identically and then
dead-lettered something the model could have fixed on the second
try. With the flag set, an attempt that follows a parser or
validator failure on the SAME backend carries one extra C
turn: "Your previous response was rejected: . Respond again with the same JSON contract,
corrected." Each re-roll REPLACES the previous feedback instead
of accumulating a transcript of complaints (the message list is
re-derived from the pristine prompt every attempt), and the
pending reason lives per backend, so truncation advances,
backend advances and network retry-sames all send the prompt
they always did — a model is only ever told about output it
actually produced. The rejected text itself is deliberately not
echoed back (it is already the model's own last turn, and this
ecosystem's validator diagnostics are the part it cannot infer);
the canned wording names a JSON contract, which is why prose
Tasks are expected to leave the flag alone. Telemetry is
unchanged — the extra turn shows up only as a larger
C figure (~100 tokens) on the retry row.
0.5.1 2026-07-21T03:15:12+01:00
- [2026-07-19] Add C<:on-exhausted> to Task: an optional hook that
fires exactly once, on the execute thread, immediately before
C is thrown (never on
success, never on the Cancelled path). The payload carries the
same C records and C string the thrown
exception carries, plus C 'exhausted'> and a
C count — letting callers (e.g. a pipeline DLQ) get
the rich attempts/raw-text payload without re-parsing the
caught exception. Shielded exactly like C: a
throwing hook is caught and noted, never suppressing or
altering the Exhausted throw. JSONTask now forwards
C<:on-exhausted> to its inner Task, same as its other hooks.
- [2026-07-04] Cooperative cancellation: Task and JSONTask accept
C<:is-cancelled>, polled before every round-trip, on each
in-flight completion poll (~10 ms cadence — the pending response
is aborted via Response.cancel, same as the timeout path), and
between chunked backoff sleeps. When the hook reports True the
Task throws the new typed X::LLM::Data::Inference::Cancelled,
which carries the same C<.attempts> records as Exhausted but is
deliberately NOT its subclass — a user cancel must never be
swallowed by exhaustion handlers. A cancelled Task no longer
burns the remaining retry chain (previously up to
max-retries x backends round-trips plus full backoff sleeps
after the caller had already walked away).
- [2026-07-03] Add C<:parse-retries> to Task and JSONTask: an
opt-in same-backend re-roll budget consumed only by parser
failures (default 0 keeps the historical advance-immediately
behaviour). Interactive callers on stochastic samplers re-roll
malformed JSON instead of exhausting the chain on one bad shot.
- [2026-07-03] Chain exhaustion now throws the typed
X::LLM::Data::Inference::Exhausted (new
LLM::Data::Inference::Exceptions module) whose C<.attempts>
records every failure — including the RAW model output for each
failed parse — while C<.message> keeps the exact historical
summary string for back-compat.
- [2026-07-03] Harden JSONTask extraction: strip blocks and
markdown code-fence markers, then pick the longest complete
top-level JSON structure via a string-aware balanced-bracket
scan that actually parses (reasoning preambles with stray
braces, plan-out-loud draft objects, and fenced answers all
extract correctly). Hopeless responses fall back to the legacy
slice so established error messages are preserved.
0.3.0 2026-04-29T23:49:54+01:00
- Bump Github Actions to use node 24+
- BREAKING (telemetry payload): the on-call-complete hook payload
key C<:provider-id> is removed and replaced with C<:generation-id>
(clearer name — the value was always OpenRouter's gen-XXXX id,
not a provider name). New keys C<:provider-name> and C<:is-byok>
are added alongside, lifted off
C when present. Reads
now use C<.?> so non-OR Response subclasses (which have no such
accessors) don't blow up — they simply omit the keys.
- Sink contract is otherwise unchanged: every key remains
presence-gated (absent when the provider didn't supply it).
Migrating sinks: rename C<%payload> →
C<%payload>; optionally consume the new
C / C keys for richer routing visibility.
0.2.0 2026-04-23T15:57:28+01:00
- Added auth specifiers to META6.json dependencies
- Task + JSONTask accept an ordered :@backends fallback chain in
addition to the legacy single :$backend. The retry loop
classifies failures into three buckets: abort (HTTP 400 / 401 /
402 / 403 / 404 — config/account errors where retrying any
model in the chain won't help), retry-same (connection errors /
5xx / unclassifiable — likely transient), and advance (timeout /
429 / empty body / parser failure / other 4xx / content-filter-
style finish-reason quits — model-specific pathology). Each
backend gets up to $.max-retries HTTP attempts (initial +
retries-same, exponential backoff with jitter capped at 30 s);
an advance-class error on any attempt short-circuits the budget
and moves on. Abort-class errors re-raise immediately without
trying the rest of the chain. See classify-error and the module
Pod for the full rule table.
- Task.classify-error(:$error-class, :$error-status,
:$parser-failed) exposed as a public method for testability and
for consumers that want to implement the same policy outside
the retry loop.
- Telemetry hook payload adds :backend-index, :model-name,
:error-class, :error-status, so sinks can identify which model
in the chain served each call and what failed. Existing keys
(:attempt, :success, :error, :latency-ms, :prompt-tokens,
:completion-tokens, :total-tokens, :cost, :model-used,
:provider-id, :finish-reason, :stage) are unchanged.
- Single-:backend constructor shape is preserved; a one-element
chain behaves exactly like the pre-fallback Task on the
retry-same path (connection / 5xx still get $.max-retries
same-model attempts with exponential backoff).
BACKWARD COMPATIBILITY — one behavioural break:
Advance-class errors (timeout / 429 / parser failure / empty body
/ content-filter-style finish quits / other 4xx) used to retry
the same model up to $.max-retries times. They now advance to
the next backend in the chain, or die immediately with
"all backend(s) exhausted" if the Task only has one. Retry-same
errors (connection drop / 5xx) are unchanged — still retried on
the same backend up to $.max-retries times before advancing.
Consequence for single-backend callers: a Task that previously
survived a stochastic parser failure via retry now dies on the
first parse error. Three mitigations:
- Preferred: pass :@backends with a fallback model. A chain
of [primary, primary] is also legal and preserves the exact
old "try the same model twice" behaviour on advance-class
failures while keeping retry-same semantics intact.
- Build the retry loop at the application layer if the same
model genuinely recovers for your workload.
- Accept the fail — in practice the old behaviour rarely
recovered on parser failures (malformed JSON tended to
repeat), which is the motivation for the change.
0.1.3 2026-04-07T20:03:42+01:00
- Updated readme examples to use generic routing terms
0.1.2 2026-04-07T20:00:40+01:00
- Removed Windows from CI (upstream Digest::SHA256::Native does not build on Windows)
0.1.1 2026-04-07T19:52:19+01:00
- Added Windows CI support via MSVC
0.1.0 2026-04-07T18:52:19+01:00
- Initial release
- Task: blocking LLM calls with configurable parser and retry
- JSONTask: JSON extraction from LLM responses with key validation
- Router: query-based routing using Roaring::Tags
- PromptBuilder: mustache-style template rendering