unfiltered

Axle 0.28.0: a release long in the making focused on completing the compaction API and steering (changelog).

The hardest bit was finding the tradeoff sweet spot between complexity and API design. I first started with steer() – an API that allows the user to send a message that both cuts the queue and inserts in the first opportunity – which I backed out off because it introduced multiple queues in the scheduling internals. The compromise was to do less internally and expose a more granular API: stop() for a graceful stop, clear() to empty the queue, and chaining them together with a send() for steering semantics. I’m still finding the right line between opinionated and flexible; I’ll try the APIs out in a few Axle-Orbit experiments and see how they fare.

Other things shipped: make Axle baseline checks run in parallel; minor updates to Sunnyday fixing bugs and bumping up models and dependency.

Evals is on my mind. Even outside of the hype, having tools to slice and dice transcripts and corpus of transcripts feels useful. I’ve been thinking a lot about the how, and so far the most tenable way for me to make progress is to start bottoms up. I build a lot of fixtures to run Axle against LLM APIs as smoke tests. They are pretty rudimentary now, and perhaps iterating on those will lead me down more interesting conceptual paths.

One thing I’m trying to get myself to do now is to learn by doing. I have a tendency to want to think through problems thoroughly before attempting them. That’s one way of working through a problem. But sometimes, the smart and right thing to do is to stop thinking and start chipping at it. I need more of that in my practice.

I’ve been thinking about what kind of work I want to do next. It then occurred to me: instead of thinking about problems, I can just work on them. The cost of making is dramatically lower these days, why not spend the time trying things and getting hands-on. The trick is picking problems small enough that I can finish. I’ll write up and post my findings here, so that I actually do.

Everyone else is talking about evals these days. Unsurprisingly, since we are at the phase where AI adoption is hitting critical mass.

There are two ways to know a system. You can know it from the inside, by holding the rule that generates its behavior: I built this, so I know how it works and what it will do. Or you can know it from the outside: when it does this, it needs to do that; quacks like a duck, is a duck and all that. Roughly speaking, it can be mapped to implementation versus specification.

In traditional software teams, software engineering and product management are the two equivalents. With AI doing most of the software writing these days, the location of the inside knowledge gets displaced. Prior to AI, the software engineer in the code base absorbs the burden of a lot of the complexity for free. AI removes the hiding spot, the thing still gets built, but no accountable human acquires the inside knowledge along the way.

I want to recall Tesler’s law of conservation of complexity. In Tesler’s formulation, there is a minimum amount of complexity that is either handled by the programmer or the user. In this case however, the knowledge of the complexity that is inherent to functioning software needs to be captured by implementation (inside knowledge) or verification (outside knowledge).

All the chatter around evals and dark factory patterns is about this. At the end of the day, someone needs to know that the system is doing what it’s supposed to do, and while the work is easier, I don’t know if the knowing can ever be.

This morning, I was multitasking. I spent a few minutes making Claude Code go, and while it’s revving I’m writing in my paper notebook. Something catches the corner of my eye, I looked up and went to tend to the AI’s request. It whirls again, so it’s back to the notebook. Back and forth. As you can imagine, neither turned out particularly well, although if I were to be fair, the robot did a much better job than my writing. After all, I was only the one multitasking.

We have long known that multitasking reduces our cognitive abilities. There are numerous credible studies on this topic. Yet everyone is dual or triple wielding coding agents these days. Everyone feels more productive, and they are. They also say the work is not better ¹.

My observation is that the current crop of AI coding agents and models sit in the anti-flow zone of productivity: there is a little too much down time in between prompts, yet not enough to be able to effectively do something else. Once you hit enter on a prompt, a typical coding agent can take anywhere from minutes to tens of minutes, and while it’s doing its thing there is nothing much the human observer can do. We can try to follow along, but the terminal outputs are not really comprehension friendly. Of course we get bored and we multitask.

As a thought experiment, I wondered what would happen if coding agents are a magnitude faster². If we can get instant gratification it might solve the desire to let the attention wander. Yet at the same time, it raises an interesting question: if the robots can generate thousands of lines of code in a matter of seconds, then how are we able to really understand what’s going on? The temptation will be to do more and understand less since it’s the path of least resistance. Consequently, we will drown in systems that we do not understand.

When I ask people about this, everyone says that taste is going to be the thing that differentiates them. Make sense, since our perspective and judgement is what we really bring to the table and affect the world around us. That said, to be able to render accurate judgement requires us to understand the thing that we are judging. If our comprehension is being overwhelmed, then our judgement is what’s being overwhelmed.

I don’t know if there’s a neat little solution to this puzzle. My sense is that the status quo is not sustainable and things will need to change: either we develop tools to help us hold and understand more complexity, or we will have to delegate. Either the craft matters, or it becomes utilitarian.

• I love designing and building UIs. Like love, love. It is work that combines product thinking, UX, systems, and visuals all at the same time and it tickles every part of my brain. There are easily five iterations to get to a satisfying outcome: original idea, wireframes, implementation, iterating through the trough of disillusionment, and then the transcendence.
• I’ve gotten to a good state with the Sunnyday Agent configuration page. Good for now at least; and I don’t use the word good lightly. There are still rough edges, and aspects of the page which are candidates for a teardown and redo in the near future as my understanding evolves. But at least everything feels and works well within the constraints it’s supposed to right now.
• That’s one of the great things about working with coding AI agents. They give me higher degrees of freedom in working and can express in higher fidelity. If I were to put it in an analogy, it’s akin to the transition from 2d to 3d. We gain a new dimension that brings it closer to the real thing.
• After working on agents for the last few months, I have a lot of thoughts about performance. All the benchmarks tend to flatten them down to a single dimension, and to be fair that’s what benchmarks are supposed to do. However, agent performance is a much more nuanced, especially because there is the what is not said gap between the lines of the instruction prompt. I will have more to say about that as I collect my thoughts.

A lot of the discourse these days conflates what’s possible with what’s good. That’s understandable. Possibilities are exciting; they let us do things we couldn’t before, start new projects, and feel squarely in the driver’s seat.

But because it is possible doesn’t mean that it is good. Too often, we just let things happen. Our judgement gets fatigued when it takes more effort to understand and decide if they are worth our time and attention. And then we let them slip. The word slop encodes that energy.

I’m not here to gatekeep possibilities. I love possibilities, but I also care that things are good. Good can be achieved when we—the people who are involved in and affected by new possibilities—spend time poring over them and iterating on them. Extending our care and exercising taste.

Now that we know that things are possible, let’s spend time making them good.

I write this as a reminder while I reflect on the old year and look forward to the new. I have heard all the proverbs in this vein, and as I grow older they take on depth. Even as I yearn for the alternative.

It’s easy to be impatient. I secretly wish that building the next feature or dieting for 3 months will get me everything that I want. First, there is no magical destination that will solve the unease I expect it to cure. Second, if it is something I want, I should work toward it even when it’s slower than I’d like. More importantly, if the work is meaningful, I have to keep doing it.

In that perspective, a year is a mere segment in the marathon. That is the inch that I toil in.

I’m a firm believer that we, humans, are primarily preoccupied with what other people are up to. The social instinct is something that is hard-wired. Politics, alliances, rivalries, and gossip – all human drama –are part of that.

AI may become the best information retrieval and even information using machine in the world. But it remains a tool. Not even a peer. There is no real reason why humans should be concerned about the opinions of AI much less the edict from one.

If there comes a day where we would rather defer to AI, then we would have indeed done our worst harm.

I thought I would write more when I started “unfiltered”. But that wasn’t the case. The format was one challenge, but it wasn’t the main obstacle.

I realized I was missing two things.

First, the instinct to turn ideas into prose. A good writer acts while ideas are still alive. They fade for two reasons. Sometimes we take too much of an inside view and we underrate how compelling they are. Other times, we overwork them so much that it feels impossible to start.

Second, the habit itself. Turning thoughts into writing is a skill honed through repetition and practice. When I don’t write regularly, it starts to feel difficult. The longer I wait, the more daunting it becomes.

I’ve always want to write more. I would love to write more. Writing is the best way to wrestle with ideas and communicating them to others. Maybe it just comes down to making it a priority.

A few observations about complexity while working on my agent library.

1. One of the foundational decisions that we have to make when designing software is where to place complexity. In a way, designing is about discovering complexity and shifting it between the user and system. As designers, we are making assumptions and decisions about what to solicit from the user and what to bake into the design.
2. Complexity here refers to decisions or steps taken as part of accomplishing a task. An example of complexity is something as mundane as naming a file. They are often trivial as individual units, but when accumulated they can make a task feel complex and frustrating unless the user is well trained.
3. Software is excellent at encoding decisions programmatically and is thus great at removing these from the user. There was a movement around sensible defaults a decade ago or so that aimed to decrease the amount of decisions that needed to be made out of the box. The end result is perceived simplicity.
4. There is no free lunch however. By its nature, simplicity is diametrically opposed to control. What is made not explicit must be implicit, either in terms of effort required to discover them or the ability to access them in the first place. In the latter case, it will sometimes make a design awkward–that is, users will have to contort themselves to take advantage of it–or, a non-starter.
5. It is extremely difficult to hide complexity completely. If a choice can be completely removed without consequences, then it is either redundant or obsolete. The reason options exist and matter is because users are not a single homogenous bloc. People want to make use of technology to achieve certain means and the diversity in needs and preferences are as kaleidoscopic as people themselves.
6. Sometimes new technology can emerge that changes the landscape of complexity. Language models are a prime example of that. The ability to use language–one that is native to us–as the medium and interface opens up much more possibilities compared to the crude resolution of commands and GUIs. This introduces new possibilities and thus complexity into the system. Current AI apps punt them onto the users but as the field develops, we will start seeing the complexity shift again.