Auction lot intelligence — pipeline design
Heavy equipment and Class 8 trucks sell at auction with a free-text condition disclosure attached. That disclosure is priced by the room. This pipeline scores what the disclosed fault probably is against what it probably costs, and then grades itself: the auction house publishes the realized price, so every prediction has an answer key.
Status: design. Nothing here is running. This page documents an architecture, not a deployed system, and no numbers have been produced by it.
Named providers are planned integrations. No commercial agreement exists with any of them, and none should be read as a partner, a sponsor, or a committed dependency.
Pipeline topology
batch · no ingress · one boxnumbers deterministic · prose generative
→ swipe the diagram sideways to pan
Data compilers
planned integrations
- EDA — UCC filings
- RigDig — VIN, DOT
- EquipmentWatch
- Rouse — residuals
- Sandhills reports
none contracted
Realized prices
licensed · not scraped
- sold-price series
- the answer key
adapters/
one module per provider
- → LotRecord protocol
- redistributable: bool
- license_tag per row
- fetch ≠ normalize
raw archive
gzip payload + Parquet
- immutable · replayable
- re-score never refetches
DuckDB
one file · no server
- versioned, never mutated
extraction
deterministic — no model
- ^[PBCU]\d{4}$ codes
- fault-phrase lexicon
- 37-category taxonomy
- field normalization
scoring
rules · mechanical first
- called vs likely cost
- uncertainty_premium
- confidence · abstain
- no LLM in this step
backtest
SQL · realized price
- distribution, not a score
narrative
prose only
- reads frozen table
- no database handle
- computes nothing
LLM seam
OpenAI-compatible /v1
- BYOK · your key
- Ollama = self-host
- reused from prod
Redistribution rights
enforced by a column, not a convention
- publication reads redistributable
- licensed data ranks; never prints
Operations
systemd timers · SOPS + age · borg
- no ingress — nothing listens
- silence pages, not just failure
Provider adapters
| Adapter | Signal it contributes | Status |
|---|---|---|
| Realized sale prices | Sold-price series — the backtest's answer key. Sourced under licence from a compiler, not scraped from auction sites: a measurement product cannot rest on data it has no right to use. | planned |
| EDA | UCC financing filings: purchase events with dates — the strongest available signal for when a fleet next turns over. | planned |
| RigDig | Class 8 VIN history, DOT inspections, liens, fleet rosters. | planned |
| EquipmentWatch · Rouse | Used-equipment values, residuals, rental rates. | planned |
| Sandhills | Listing and realized-price series across the trade publications. | planned |
Design decisions
Why adapters, not scrapers
- Scraping auction sites directly is fragile, rate-limited, and grey under each site's terms — and the failure mode is silent, so the data quietly degrades.
- The compilers already did the normalization work and sell a licensed version of it.
- Coupling to any one source is a design smell regardless of which source it is. One
LotRecordprotocol, one module per provider. - Adding a provider becomes a new file, not a refactor.
Why the numbers are deterministic
- Extraction is regex plus a version-controlled fault-phrase lexicon. If a model does extraction, every downstream number inherits its variance and nothing reproduces.
- Scoring encodes an explicit protocol — mechanical fundamentals, then sensors, then integration, computer last.
abstainis a first-class outcome. A scorer that answers confidently for every lot is a wrapper with a temperature setting.- The abstention rate is reported as a headline number, not hidden.
Where the model is allowed to act
- Exactly one stage: writing the prose a human reads. It never scores, ranks, adjusts, or fills a gap.
- Enforced structurally, not by prompt — the narrative stage receives a frozen result set and holds no database handle. It cannot query, so it cannot invent.
- Bring-your-own-key and model-agnostic:
LLM_BASE_URL,LLM_MODEL,LLM_API_KEY. - That seam is not new code — it is the same one running in the live DiagnosticPro backend. Ollama makes the whole thing self-hosted.
Rights as a column
- Every row carries
redistributableand a license tag, stamped by the adapter that produced it. - The publication layer filters on that flag. Licensed-derived signal can influence which lots surface; it can never appear as a printed number.
- This is "buy for build, publish from public" expressed as a
WHEREclause — a rule enforced by a column cannot be forgotten at 11pm.
Why DuckDB and not Postgres
- The workload is one writer, no concurrency, and repeated full scans over a few hundred thousand rows — a columnar analytical shape.
- Postgres would add a container, a port, a user, a backup policy, and a connection string in exchange for concurrency this will never use.
- DuckDB is one file that the existing backup fabric already captures.
- The escape hatch is real: Parquet is the source of truth, DuckDB is derived. Moving to Postgres later is a load script, not a migration.
Operations
- systemd timers, staggered per adapter. Airflow, Dagster, and Prefect are all services to operate; a timer is a file.
- No ingress. Nothing listens. The output is a document.
- Alerting pages on failure and on silence — a job that quietly stops running looks identical to a healthy one unless you watch for it.
- Re-scoring reads the archive, never the network, so a rule change can never trigger a metered API bill.