53 articles tagged Llm
3 LLM context techniques: when each fails
Compare fine-tuning, retrieval-augmented generation and long context by failure mode, cost arithmetic and a decision checklist, with code for each.
15 min read · 1 November 2026
Context engineering: the real fix for agent hallucinations
Long-running agents fail because their context fills with stale tool output and contradictions. Here is how to structure, prune and measure that context.
16 min read · 27 October 2026
7 guardrails for safer LLM apps
A practical review of seven guardrail layers for LLM applications: what each catches, where it fails, and which to build first.
18 min read · 25 October 2026
Agentic systems: architecture principles that hold
How to enforce deadlines, token budgets, idempotency keys and bounded fan-out around multi-model agent graphs, with Python code and tests.
15 min read · 23 October 2026
Prompts are code: how to measure their value
Prompts and agent graphs need outcome metrics, not token counts. A practical guide to boundary instrumentation, versioning, and avoiding vanity metrics.
19 min read · 16 October 2026
Detecting silent agent failures
A layered quality gate that catches well-formed but wrong agent output: schema checks, PII scans, citation grounding, and a model judge, plus how to measure each.
15 min read · 13 October 2026
Agent audit trails: event log vs state snapshots
Event sourcing and state snapshots solve different audit problems for AI agents. Compare causal reconstruction, debugging fidelity, cost, and replay limits.
13 min read · 12 October 2026
LLM evals: LangSmith vs Promptfoo in prod
How to choose between hosted trace-based eval platforms and local CLI eval runners, with latency, cost, CI and data-residency tradeoffs.
15 min read · 3 October 2026
Agentic workflows: surviving network partitions
How to keep agentic workflows correct when networks drop: idempotency keys, an outbox with stable action IDs, and reconciliation on reconnect.
18 min read · 1 October 2026
Multi-agent handoffs: the latency tax nobody measures
Multi-agent pipelines pay a latency tax at every handoff. Here is how to measure serialisation, transport and re-prompting cost, and what to fix first.
11 min read · 29 September 2026
APM is blind to your AI agents
Traditional APM monitors transport, not semantics. See how to instrument retrieval quality, token cost, and answer correctness in AI agent pipelines.
12 min read · 25 September 2026
Multi-agent systems: when they quietly degrade
Multi-agent LLM systems drift into slow, costly behaviour without failing loudly. Here is what to instrument, what to budget, and how to catch the slide.
12 min read · 24 September 2026
When your agent goes silent: degrade gracefully
How to keep agent-backed features useful when a model is slow or confidently wrong: deadlines, fallbacks, circuit breakers, validation, and how to measure each.
13 min read · 21 September 2026
Agentic AI cost control: 7 tools, one backfire
How to control agentic AI spend: ground-truth billing data first, attribution second, and why LLM optimizers over billing data backfire.
15 min read · 19 September 2026
Context windows: agents that run for days
Long-running agents fail when context is treated as a log. Keep constraints verbatim, checkpoint durable state, and cap the working set per cycle.
17 min read · 19 September 2026
Most LLM cost dashboards lie…
Per-request token attribution, prompt caching, model routing and retry caps: how to instrument LLM spend so the numbers match the invoice.
13 min read · 19 September 2026
Multi-agent systems: why failures hide
Why multi-agent LLM pipelines fail silently, and how per-agent tracing, schema validation, budget enforcement, and loop detection make those failures loud.
13 min read · 18 September 2026
Feature flags became AI rollout platforms
How flag platforms handle structured model configs, eval-metric gating, and offline fallback — plus a checklist for choosing one for AI rollouts.
18 min read · 18 September 2026
Stop Putting LLM Keys in .env
Why flat .env files fail at scale for LLM API keys, and how a secrets manager plus gateway brokering scoped credentials fixes rotation and leaks.
14 min read · 17 September 2026
LLM agents: why monoliths win
Why splitting an LLM agent into micro-services often costs latency, audit continuity and money, and how to measure before you decompose.
11 min read · 10 September 2026
Why your routing layer will collapse at scale
How to build stateful, region-aware LLM routing: config schema, retry budgets, circuit breakers, audit trails, and the failure modes to plan for.
12 min read · 4 September 2026
Commit ownership in AI pair coding
AI pair coding shifts who is accountable for a commit. A practical contract: prompt hashing, test gates, ownership tags, and review discipline.
14 min read · 31 August 2026
LLM eval drift that users notice first
Three pipeline-level causes of user-visible LLM drift, how to instrument each one, and how to verify the fix with metrics rather than vibes.
16 min read · 28 August 2026
AI errors hide in plain latency
Why token counts and GPU utilization miss user-visible AI degradation, and how to instrument end-to-end p95/p99 latency and error rate instead.
10 min read · 21 August 2026
AI-era promotions hinge on this one thing
AI features fail through cache eviction, schema drift and missing rollback paths. A practical guide to validation gates, fallback caches and metrics.
11 min read · 15 August 2026
Senior engineer title inflation in 2026
How AI-generated code changes senior-engineer expectations: residency-aware LLM gateways, retry budgets, audit logs, and verification you can run.
11 min read · 13 August 2026
Agent frameworks: stop rewriting the same glue code
Agent glue code — retries, credentials, state — is mostly boilerplate. Here are three abstractions, worked examples, and how to measure the savings.
13 min read · 10 August 2026
AI cost attribution without the spreadsheet hell
How to attribute AI inference costs to users, features and models: instrumentation, cardinality limits, sampling error, and a 30-minute first step.
15 min read · 7 August 2026
Why APM Misses GPU-Bound LLM Incidents
Application monitoring traces Python calls, not CUDA kernels or batch queues. Here is how to instrument the layers where LLM latency actually hides.
12 min read · 1 August 2026
Latency trade-offs: Nairobi teams beat global AI providers
Why AI teams serving East African users weigh total cost of interaction, not raw p99, and how to measure the trade-offs yourself.
14 min read · 26 July 2026
Context windows live longer than you think
Long-running agents exhaust context windows through unbounded history growth. Covers pruning, summarisation, retrieval, and retention policies with code.
13 min read · 25 July 2026
Agents keep breaking prod: 3 boundary rules you still need
Agents that mutate external state fail silently when 2xx is mistaken for completion. Three boundary rules: outcome polling, schema drift gates, atomic claims.
14 min read · 17 July 2026
Stabilize multi-model agent routers: six production rules
Six rules for multi-model agent routers: bounded concurrency, idempotency keys, timeouts, observability, and how to measure each one yourself.
15 min read · 15 July 2026
Trim LLM bills: 3 FinOps moves that work
Request shaping, two-layer caching and priority queuing explained with working code, honest failure modes, and a cost measurement method you can run today.
11 min read · 12 July 2026
Observability for LLM Features: Metrics That Drive Decisions
Log one structured event per LLM interaction instead of tracing every token, then alert only on user-facing outcomes that justify keeping a feature.
14 min read · 8 July 2026
LLMOps 2026: evaluation stack that replaced RAG dashboards
Why RAG dashboards miss real failures, and how to build an evaluation stack around task resolution, dataset rotation, prompt versioning, and drift checks.
13 min read · 5 July 2026
LLM drift: the silent quality drop metrics miss
Latency and error rate stay flat while LLM output quality quietly degrades. Here is how to log prompts, measure embedding drift, and catch it early.
11 min read · 4 July 2026
Detect lazy agents without false alarms
Success/fail gates miss agents that keep running while producing harmful output. Measure outcomes instead, with a worked predictor design and decision rules.
14 min read · 4 July 2026
AI made self-service platforms trivial in 2026
AI can draft Terraform and Kubernetes manifests, but correctness still needs policy checks, schema validation, and runtime verification before merge.
10 min read · 3 July 2026
AI agents: circuit breakers are not optional
AI agents fail by hanging and returning plausible garbage, not by crashing. Here is how to add a circuit breaker that catches semantic failures.
13 min read · 3 July 2026
JSON mode's hidden costs revealed
LLM JSON mode guarantees syntax, not usable data. A two-stage extraction-plus-correction pipeline makes parsing reliable and cheap without fine-tuning.
12 min read · 3 July 2026
Debug AI agents: failures aren't just bugs
AI agent postmortems need different questions than service incidents. Compare structured failure narratives with LLM-assisted causal graphs, and when each wins.
11 min read · 1 July 2026
Postmortems for AI agents: drift, cost, replay
How to run postmortems on AI agents: capture prompts and model versions, measure drift and cost per session, and choose between log-based and instrumented pipelines.
12 min read · 29 June 2026
AI revives unloved legacy codebases
A practical stack for maintaining legacy code: static analysis, generated characterization tests, and LLM documentation, with failure modes and a 30-minute first step.
15 min read · 29 June 2026
AI observability: the metrics that matter
AI observability needs drift, confidence, and attribution metrics. Covers instrumentation, cardinality limits, failure modes, and when not to build it.
13 min read · 28 June 2026
AI apps break when you treat them like regular software
AI apps fail differently from CRUD services: state accumulates, caches churn, and model outputs drift. Here is how to design, measure, and bound those failure modes.
15 min read · 27 June 2026
Debug AI agent failures differently
Structured logs catch infrastructure faults; behavior traces catch hallucinations and prompt drift. How to instrument both, measure detection time, and pick a stack.
13 min read · 27 June 2026
LLM output metrics that actually move the needle
A layered approach to measuring LLM output quality in production: structured validation, p95 latency, cost per request, and retention signals.
10 min read · 26 June 2026
AI agent postmortems: the three surprises
How to run postmortems for AI agents that fail semantically, not just technically: structured semantic logging, live shadowing, and a decision checklist.
14 min read · 24 June 2026
LLM evals: skip the hype, track these 4 metrics
A practical guide to production LLM evaluation: four metric classes, code for deterministic validators, and how to detect prompt drift and scorer decay.
11 min read · 22 June 2026
JSON mode’s silent failures exposed
JSON mode enforces grammar, not your contract. Here is how to build a schema-validated envelope pipeline with deterministic repair and quarantine.
11 min read · 21 June 2026
Postmortems for AI agents: human review vs blind replay
Compare human-in-the-loop and deterministic-replay postmortems for AI agents, with a decision checklist and a 30-minute audit you can run today.
16 min read · 20 June 2026
LLM metrics that stop wasting time
A practical guide to LLM evaluation in production: grounding checks, drift detection, cost and latency metrics, and how to measure each one.
13 min read · 14 June 2026