The Guilded Age of PoCs
Last week, I was talking with a colleague about how "it’s been quite a while since I’ve seen my work reflected in the real world". I don’t want to be misinterpreted: I understand (and I hope my employers understand this as well, for my sake and that of my professional future) that I do my part in the development of the team and the people I manage, and that I contribute to the evolution of the unit I work for, but I believe that a significant change has occurred—or is occurring—in what an engineer is expected to achieve through their work. We used to build things just to build them; now we build things to prove that they can be built—and, as I’ll discuss later, the zeitgeist isn’t helping at all.
Automating our workflow has opened many doors for us to explore at once and introduced us to many different types of people whom we need to convince of the suitability of the results we’ve achieved. As if that weren’t enough, the number of touchpoints they require varies significantly and is quite diverse.
From Prototype to the Hypothesis Factory
I’ve worked in pre-sales, in the early stages of product ideation, and in providing technical advice to determine the feasibility of a feature proposed by a functional leader, so the concept of prototyping is not foreign to me. I can’t deny the value it adds to development: identifying whether a premise is valid as early as possible can save headaches and money. A smoke test, an end-to-end test supported by scripts that call APIs and serialize their results to ensure all the boxes are checked, or a relatively interactive design mockup that’s worth more than a thousand functional documents—these have always been valuable companions during those uncertain moments when the scope of software development is still being defined.
But what we're experiencing now isn't that. What we've been experiencing these past few months is an unbridled cult of PoC.
We have tools that exponentially increase the speed with which we can respond—visually and interactively—to questions or scenarios that weren’t on the table yesterday. There’s a positive side to this, of course, because if we interact with peers who are agile in this medium, we’ll see a substantial improvement in productivity; dead ends are quickly ruled out without having to spend time exploring them, and, if the situation is favorable, we’ll be able to focus on the core elements of our project.
Or maybe not.
Because agility, speed, and apparent immediacy can easily unlock that bootstrap—which, depending on its scale or the number of workdays it might entail, used to be restricted by high-level decision gates. Furthermore, these decision gates did not usually have clear ownership from the outset, and it was easy for them to get lost along the way. Now this path appears clear, since the creation of PoCs is virtually immediate, but it catalyzes a proliferation of scenarios to consider: a continuous storm of ideas, of what-ifs, of potential that never materializes. It is simply a greater involvement of the engineering team in the ideation of the product to be built—which is not a bad thing in and of itself—but without a clear separation of responsibilities and a definition of the expected outcome, it can drag on indefinitely and branch out in countless directions.
Enter Heraclitus
The river of generative AI-based automation in which we are swimming today is not the same river we were swimming in a quarter ago, and it certainly won’t be the same river we’ll be swimming in another quarter from now.
The technical approach to a project might start with an MCP implementation, move on to orchestration using A2A, and then return to MCP because we want to use a new harness—a harness that seems to eliminate the need for a number of the guardrails we assumed we’d have to build ourselves. However, the models have advanced so much since the last meeting that context management is no longer as critical, but, be careful—we must start treating models as commodities and make our code agnostic as to which model we call… which means we’ll have to reverse our decision to use the harness we wanted; we’re going to have to build it ourselves… Does anyone still have the notes we took two months ago on RAG for grounding? They’ve become relevant again…
It’s a race to build something that, while further than almost anything else from a production-ready development, doesn’t seem to have a solid foundation anywhere. Proof-of-concepts (PoCs) are simple and allow us to offer relatively well-founded opinions on the feasibility of solutions (although others would have to determine how suitable those solutions would be).
We engineers build PoCs just as Puyi’s entourage runs after him, fulfilling his every demand, in Bertolucci’s The Last Emperor.
What Should We Use Generative AI For?
I don’t think this apparent flip-flopping—these constant shifts in direction that I’m beginning to notice in my line of work—are unrelated to a larger problem: we still don’t know where the value of automation through generative AI lies. We can view it as mere automation—perhaps on a scale an order of magnitude greater than what we’ve experienced before—but one that can be managed in the same way by dedicating the necessary time and analysis to it. This would imply that the value of generative AI as a tool for automating processes would improve productivity within the way we’re accustomed to working. In my specific case as a software engineer, this means improving every aspect of the SDLC that can be enhanced through these means, thereby achieving greater productivity and higher-quality results. However, it seems that doing what we were already doing, but better is falling short. Investment as a concept—which is what keeps companies alive—does not seem to view this as disruptive enough to warrant its attention. Improving what you were already doing well isn’t enough, because while you’re simply working, someone else can overtake you by demonstrating that those outdated assembly lines you are polishing are no longer needed.
It feels as though no one can any longer become the gatekeeper of anything or build impenetrable moats by exploiting an idea that will make them rich. It took the opening up of access to mechanisms that greatly enhance the development of our ideas for us to realize that we don’t know what to do with them. The atmosphere of uncertainty in the shadow of a rapidly changing, overfunded technology—which occasionally pops up in the news with warnings that someone is going to be able to create a sentient doomsday machine based on predictive text models—is full of noise, haste, and PoCs.
Nothing substantial enough to go into production, but enough to join the fray.