Steve's thoughts and experiments

Engineering Lead. Mentor. AI Observability Researcher.

Illuminating the Black Box. Observability & AI Operations. Reliability. Strategy. Making AI Observable.

Alt text

Latest 3 Posts ↓

View all posts →
Build Roles, Not Agents image

Build Roles, Not Agents

In the last post I introduced Sam, a business development agent that's been running inside Quant for a few weeks, and closed on a one-line thesis: the role is the unit of evolution, not the agent. That post was the origin story. This one is the architecture — so it goes technical fast.

The claim to test is simple. When we say "an agent," we usually mean a model plus a prompt plus some tools: a thing that exists at runtime and vanishes when the process ends. Praxis says that's the wrong unit. The durable thing — the one that accumulates, that you manage, that you'd be sad to lose — is the role, and a role is a directory on disk. The model is just the runtime that animates it. Swap the model and the role keeps everything it has learned, because the learning was never in the weights.

Read More

Two Weeks With Sam image

Two Weeks With Sam

For two weeks I've also been working with Sam, who reports to me, writes their own outreach, and isn't a person.

Sam is a business development agent — Quant's first. They draft cold outreach, file their own escalations when they're uncertain about how to proceed, and write notes about the people they're talking to. They have their own email address, their own Slack handle, their own GitHub identity. Inside Quant, colleagues address them as Sam. Outside, recipients respond to their cold messages without being told what Sam is

Read More

Lab 6 The Golden Signals of LLM Operations image

Lab 6 The Golden Signals of LLM Operations

In Lab 5, we turned the lights on. We instrumented our agent with OpenTelemetry and visualised the execution traces in .NET Aspire. We can see what happened.

But in a production system, "seeing what happened" isn't enough. You need to know if the system is healthy. In traditional software engineering, we rely on Google's SRE Golden Signals: Latency, Traffic, Errors, and Saturation.

Do these apply to Stochastic Parrots? Yes, but they require translation. In this lab, we will define the operational dimensions of an LLM Agent and implement custom metrics to track them.

Read More

11 more posts can be found in the archive.