AI generates code an order of magnitude faster than any human can. Hand-writing code is no longer a tenable proposition, and so the effort has shifted to doing whatever we can to hold AI code to our own quality bar. Some hand wave it away, comparing it to the evolution of C over Assembly, arguing that over time AI is going to be so good that we never have to think about the lower level code anymore. Others reach for the dark factory, where we validate the inputs and outputs and never care about what happens in between. The most developed version of this is a methodology now, promising to turn vibe coding into agentic engineering.
I’ve written about the conservation of complexity. The desired output is functionality, and the burden of knowing the details and verifying that it performs to the standard is something that still needs to be performed. In the agentic engineering case, the work shifts from software design and planning to a combination of bringing in the relevant contexts, using the appropriate set of prompt incantations, asking all the right questions, and rigorous testing.
I can see the allure. With the right process and guardrails, we can coerce AI to produce quality software. That said, the fragility of quality is often due to established processes being upended by unknowns, and it is impossible for us or anyone to know everything ahead of time. If the price of producing software approaches zero, we can take inspiration from practices from a recent past: prototyping.
In a formal Design process, we use prototypes extensively to validate desired behaviors and outcomes, which then gets handed over to the engineering process. Why not extend the idea: use AI to create prototypes, learn, and then generate specs, validation, and tests from them. More importantly, we discard the prototypes and rebuild the code from ground up using these produced artifacts. If anything changes with the artifacts, we discard the code and generate it from ground up again.
This separation between spec generation and code generation is important. A lot of bugs in software happen because of workarounds accumulated over time. Instructing AI to “make small changes” sounds good until it becomes a volley of patches layered on top of each other that becomes impossible to reason about.
This might sound a tad radical in a “we’ll rewrite it later” never world. However, consider the strengths of an AI agent: they are able to endlessly and tirelessly transform a set of words into another and conform them to an infinite suite of tests. Perhaps if production code is only produced with clean and locked down specs, we can maintain the rigor required of an industrial process. And notice the trick here, a fully deterministic set of requirements that produces a verifiable set of outputs sounds like a compiler. In the course of the rise of generative AI, we’ve been endlessly, tirelessly trying to coax probabilistic AI into deterministic outcomes, and maybe one day we will succeed.