Introduction
The advent of coding agents has improved many teams’ ability to deliver software, but it comes with new challenges that the industry is only now starting to grapple with. In this short essay I will argue that AI gives superficial confidence, especially in legacy codebases, impairing a new developer’s ability to build an in-depth theory and mental model of the codebase and business requirements. Developers used to wrangle with the system and enquire with team members to better understand the nature of a system and clarify requirements. AI has changed that.
What is a theory of a program
This is not my idea but is from a widely read essay by a Danish computer scientist named Peter Naur. His argument was quite simple: he states that programming is not just the production of code, it is the building of a human mental model. He argues that a programmer who possesses a theory of the program has many advantages such as:
- Real world mapping: They understand how the code maps to real world business use cases and know the limitations and what was left out.
- They know the “why”: They’re able to justify their decisions and explain why alternative approaches were rejected, and be able to answer complex queries regarding unexpected system behaviour.
- Can confidently introduce changes: They can seamlessly make modifications to the program to meet new requirements because they intuitively understand how their change can impact other interconnected parts of the conceptual framework.
You can find this essay here.
The risks of poor theory building
A failure for a developer to correctly build a good mental model of the codebase can have many severe consequences. We all agree that a program has to undergo change over time because business requirements change and change is inevitable.
When original developers leave, they take the mental model they had built over time with them, and even if they captured parts of it in documentation, that will never be enough. Some pieces of documentation could also be outdated and the devs might have developed debugging techniques or mental shortcuts that aren’t easy to capture on paper. So with the original developers gone, new developers are left to make sense of the system themselves, and they do this by experimenting and debugging in lower environments.
As time moves on and new changes are introduced, bugs begin to surface as the theory is lost and changes are now being made blindly and with very little confidence. This ultimately leads to system decay and ruin.
Hence, a failure to correctly build a good mental model of the system leads to poor outcomes and an increased chance of introducing defects. Sure, given enough time the new developers are able to catch up with the program’s theory, but it’s a tragedy.
How AI makes theory building worse
My argument is that the introduction of AI impairs theory building and gives false confidence to new developers. Say you’re working on a codebase as a junior, and you get stuck on an issue, and you whip up Claude Code or your favourite agentic coding tool. You ask the AI to introduce the new feature outlined in your ticket. The AI introduces the change and voila! It works. Meets all your acceptance criteria.
What’s wrong with this? You delivered the feature right? You did, but you missed one important thing about programming: “theory building”. The agent was able to do most of the heavy lifting for you but as a result you missed out on the struggling and debugging you would have had to go through had you introduced this yourself. I believe that this process leads to an in-depth understanding of the codebase, because it leads to serendipity, and you stumble on parts of the system you may have not understood.
Sure, when used correctly, AI can be an effective sidekick that can help you learn, but we are cognitively lazy and will let it do 100% of the work and call it a day.
It’s not hard to see how this can slow down or impair a developer’s ability to build a good mental model of the system. It leads to overconfidence and as a result increases the potential for defects.
Conclusion
So we see that programming is not just spitting out code, but another intricate process runs parallel as developers code. This parallel process is theory building and is the part that is hardest to even convey in documentation; it requires developers to actively work with the system to build intuition over time. When we receive requirements, they aren’t often complete and through clarification we are able to improve our mental model of the system.
AI is here to stay, and it does give us new superpowers, but with great power comes great responsibility. We can’t prevent people from using AI and even if we could, we would be depriving them of a resource that can improve their productivity. So we need to find a balance; perhaps we should encourage juniors and newcomers to pair with existing devs to initially build a baseline understanding of the codebase before they jump onto agentic coding tools.
A poor mental model of a program can lead to an increase in defects and adversely impact end users and even your bottom line. So do we really want to take the risk of cultivating developers who never develop a strong mental model of the system and are overzealous because their new friend with very little context can spit out the code?