Code-First AI Systems: Why the Model Should Not Run the Workflow
The model should classify, extract, summarize, judge, or propose. Code should own routing, state, validation, retries, evidence, review, and evaluation.
Building AI systemsField notes on building AI systems that hold up in real work. Written by Teja Sagiraju, an ML engineer with 13 years in the job, from systems built in public.
22 notes · 4 systems · RSSStart here
Latest
New in a series: PolicyTrace7 notes · 17 Oct
- Why PolicyTrace: the claims handler and the policy pack
- Try PolicyTrace yourself: a step-by-step guide
- The folder my privacy story forgot
Last run on the demo pack: 59 fields · 46 traced to the PDF · live demo ↗
See all 7 in order →Why One Model Should Not Handle Every AI Task
Using one model for every AI task turns model choice into a hidden default. Production workflows need explicit model policies for task type, risk, latency, evidence, fallback, and evaluation.
Token Economics: Why Prompt Bloat Kills AI Margins
Token waste is architecture debt. Oversized prompts, broad context, verbose outputs, retries, review, and eval runs compound into the cost per trusted completed unit.
Semantic Caching Is Harder Than It Looks
Semantic similarity measures intent, not validity. Safe AI caching needs source versions, schemas, model policy, evidence, permissions, review state, and wrong-hit tracking.
How to Trace a Failed LLM Run
When an AI workflow fails, the final answer is not enough. You need the input, route, prompt, model, context, schema errors, evidence gaps, review action, and eval case.
Designing Fallbacks for AI Systems
AI failures need designed paths: retry, narrower scope, simpler model, stronger model, human review, partial answer, safe message, or stop.
Document AI
Getting reliable fields out of real documents, and showing where they came from.
Why PDF Extraction Is Harder Than It Looks
PDF extraction fails in production because business documents are not clean text. They carry layout, tables, repeated fields, scans, conflicts, and evidence requirements.
Why Chunking Breaks Business Documents
Chunking is not just a retrieval setting. In business documents, the wrong chunks can separate values from labels, tables from headers, clauses from definitions, and evidence from the answer.
Two documents, one policy, four disagreements
I ran PolicyTrace on a four-PDF demo pack and got four conflicts. Two were whitespace. One exposed a bug in my own merge code.
What happens to a PDF inside PolicyTrace
One upload followed through the code, file by file: where masking happens, why the evidence corpus is built before it, and where the model's job ends.
Building AI systems
Architecture that keeps the model as one step, not the whole system.
The Death of the 1,000-Line Prompt
Giant prompts feel productive at demo stage. In production, workflow logic needs to move into code, schemas, routing, validation, evidence, review, and evaluation.
Pydantic as an AI Architecture Boundary
LLM output is not application state until it passes a typed boundary. Schemas, validation errors, evidence fields, and versioned contracts make AI workflows easier to inspect and change.
Code-First AI Systems: Why the Model Should Not Run the Workflow
The model should classify, extract, summarize, judge, or propose. Code should own routing, state, validation, retries, evidence, review, and evaluation.
Why One Model Should Not Handle Every AI Task
Using one model for every AI task turns model choice into a hidden default. Production workflows need explicit model policies for task type, risk, latency, evidence, fallback, and evaluation.
Running in production
What happens after launch: failures, fallbacks and cost.
What Happens the Day After an AI System Launches?
Launch is not the finish line for an AI system. It is when review queues, failures, costs, user feedback, model changes, and ownership finally become real.
Designing Fallbacks for AI Systems
AI failures need designed paths: retry, narrower scope, simpler model, stronger model, human review, partial answer, safe message, or stop.
How to Trace a Failed LLM Run
When an AI workflow fails, the final answer is not enough. You need the input, route, prompt, model, context, schema errors, evidence gaps, review action, and eval case.
Semantic Caching Is Harder Than It Looks
Semantic similarity measures intent, not validity. Safe AI caching needs source versions, schemas, model policy, evidence, permissions, review state, and wrong-hit tracking.
Foundations
Evaluation, evidence and deciding what is worth building.
Why Most AI Demos Fail Before Production
A visual guide to the gap between a working AI demo and a production workflow that survives real inputs, users, review, cost, and failure.
From Demo to System: The Practical AI Checklist
A compact 8-check framework for deciding whether an AI idea deserves a production workflow, before you spend weeks building around it.
AI Systems Need Evidence, Not Just Outputs
A practical framework for making AI outputs reviewable, traceable, and trustworthy in real workflows.
Evaluation Is an Engineering Problem
Why AI evaluation is not a report card after launch, but a design constraint from day one.
Systems
Open reference systems. Each one has its own page, code and notes.
● Live demo
PolicyTrace
Reads a UK motor policy pack and returns one checked record, each value traced to its line in the PDF.
Document AI · Insurance In progressUK Company Risk Intelligence
Agentic research for company due diligence, with source-aware evidence and a human decision point.
Agents · ResearchIn design
FNOL Decision Lab
Claim email intake: classify, extract and route, tested on synthetic emails first.
Workflow · EvaluationBuilding
Paperlens
Summaries, translation, Q&A and entities across documents, including scans and forms.
Document AI · MCP