unfiltered

I wrote this down a few months ago while working on Sunnyday. Recording here for posterity.

There are 4 levels to understand if an agentic trace was successful. It breaks down to the following:

• L0: did the run complete? These are binary outcomes at the infra level, basically if there are errors at the infrastructure level that prevented a run completion. Common examples are whether the LLM provider is at capacity, less obvious ones can be an example of a misconstructed tool call that loops forever.
• L1: did the agent do what it planned? At this level, we ask the AI agent to pre-register what it thinks it’s going to need to do based on the information that is available at the message. Then we grade the completed run against the plan to check for gaps that we can fill in earlier.
• L2: did it do the work well? Independent of the plan, did the agent encounter any obstacles when doing the work. Two separate tracks here. The first is if the run was optimal, where we detect for stumbling, error recovery, path efficiency, looping and flailing. Then there is the counterpart where we check if the tools and infrastructure is providing the right level of access and guidance to the agent.
• L3: did the customer get what they wanted? This is a check on the work quality. The user started the agent to achieve a certain outcome; did the agent produce it? We start at examining whether deliverables are completed e.g., prose or pdf files, and then we can inspect the content of those things, whether they reflect the request, and furthermore we can delve into whether the content is accurate or aesthetic. This can probably be further broken down into multiple layers.

Axle 0.28.0: a release long in the making focused on completing the compaction API and steering (changelog).

The hardest bit was finding the tradeoff sweet spot between complexity and API design. I first started with steer() – an API that allows the user to send a message that both cuts the queue and inserts in the first opportunity – which I backed out off because it introduced multiple queues in the scheduling internals. The compromise was to do less internally and expose a more granular API: stop() for a graceful stop, clear() to empty the queue, and chaining them together with a send() for steering semantics. I’m still finding the right line between opinionated and flexible; I’ll try the APIs out in a few Axle-Orbit experiments and see how they fare.

Other things shipped: make Axle baseline checks run in parallel; minor updates to Sunnyday fixing bugs and bumping up models and dependency.

Evals is on my mind. Even outside of the hype, having tools to slice and dice transcripts and corpus of transcripts feels useful. I’ve been thinking a lot about the how, and so far the most tenable way for me to make progress is to start bottoms up. I build a lot of fixtures to run Axle against LLM APIs as smoke tests. They are pretty rudimentary now, and perhaps iterating on those will lead me down more interesting conceptual paths.

One thing I’m trying to get myself to do now is to learn by doing. I have a tendency to want to think through problems thoroughly before attempting them. That’s one way of working through a problem. But sometimes, the smart and right thing to do is to stop thinking and start chipping at it. I need more of that in my practice.

I’ve been thinking about what kind of work I want to do next. It then occurred to me: instead of thinking about problems, I can just work on them. The cost of making is dramatically lower these days, why not spend the time trying things and getting hands-on. The trick is picking problems small enough that I can finish. I’ll write up and post my findings here, so that I actually do.

Everyone else is talking about evals these days. Unsurprisingly, since we are at the phase where AI adoption is hitting critical mass.

There are two ways to know a system. You can know it from the inside, by holding the rule that generates its behavior: I built this, so I know how it works and what it will do. Or you can know it from the outside: when it does this, it needs to do that; quacks like a duck, is a duck and all that. Roughly speaking, it can be mapped to implementation versus specification.

In traditional software teams, software engineering and product management are the two equivalents. With AI doing most of the software writing these days, the location of the inside knowledge gets displaced. Prior to AI, the software engineer in the code base absorbs the burden of a lot of the complexity for free. AI removes the hiding spot, the thing still gets built, but no accountable human acquires the inside knowledge along the way.

I want to recall Tesler’s law of conservation of complexity. In Tesler’s formulation, there is a minimum amount of complexity that is either handled by the programmer or the user. In this case however, the knowledge of the complexity that is inherent to functioning software needs to be captured by implementation (inside knowledge) or verification (outside knowledge).

All the chatter around evals and dark factory patterns is about this. At the end of the day, someone needs to know that the system is doing what it’s supposed to do, and while the work is easier, I don’t know if the knowing can ever be.

This morning, I was multitasking. I spent a few minutes making Claude Code go, and while it’s revving I’m writing in my paper notebook. Something catches the corner of my eye, I looked up and went to tend to the AI’s request. It whirls again, so it’s back to the notebook. Back and forth. As you can imagine, neither turned out particularly well, although if I were to be fair, the robot did a much better job than my writing. After all, I was only the one multitasking.

We have long known that multitasking reduces our cognitive abilities. There are numerous credible studies on this topic. Yet everyone is dual or triple wielding coding agents these days. Everyone feels more productive, and they are. They also say the work is not better ¹.

My observation is that the current crop of AI coding agents and models sit in the anti-flow zone of productivity: there is a little too much down time in between prompts, yet not enough to be able to effectively do something else. Once you hit enter on a prompt, a typical coding agent can take anywhere from minutes to tens of minutes, and while it’s doing its thing there is nothing much the human observer can do. We can try to follow along, but the terminal outputs are not really comprehension friendly. Of course we get bored and we multitask.

As a thought experiment, I wondered what would happen if coding agents are a magnitude faster². If we can get instant gratification it might solve the desire to let the attention wander. Yet at the same time, it raises an interesting question: if the robots can generate thousands of lines of code in a matter of seconds, then how are we able to really understand what’s going on? The temptation will be to do more and understand less since it’s the path of least resistance. Consequently, we will drown in systems that we do not understand.

When I ask people about this, everyone says that taste is going to be the thing that differentiates them. Make sense, since our perspective and judgement is what we really bring to the table and affect the world around us. That said, to be able to render accurate judgement requires us to understand the thing that we are judging. If our comprehension is being overwhelmed, then our judgement is what’s being overwhelmed.

I don’t know if there’s a neat little solution to this puzzle. My sense is that the status quo is not sustainable and things will need to change: either we develop tools to help us hold and understand more complexity, or we will have to delegate. Either the craft matters, or it becomes utilitarian.