One argument keeps surfacing in discussions about AI and software development, repeated often enough that it has started to feel like common sense: AI will take over the routine work, but judgment will remain human. Developers will shift from writing code to evaluating it. From production to oversight.
That sounds reasonable and in fact, it is already happening from yesterday. But it contains a hidden assumption that rarely gets examined. Judgment is outcome of learning.
The ability to recognize a fragile architecture, to notice that generated code does something subtly different from what it claims, to see that a test suite covers the happy path but nothing else — that ability does not appear from nowhere. It is built through years of work in which you made bad decisions yourself, debugged other people’s mistakes, and slowly developed a model of how systems fail.
Good judgment about code presupposes deep experience with writing it. And that experience is exactly what AI is absorbing first.
The oversight paradox
Consider a doctor who has never conducted an examination directly, only interpreted the outputs of automated tools. Or a pilot who flies exclusively on autopilot and intervenes manually only in exceptional situations. Both arrangements can work well — until something falls outside the range of what the system was trained on. At that point, what matters is whether the person in the room has genuine competence, or just a habit of clicking confirm.
Software is no different. AI produces plausible code. But plausible is not the same as correct, robust, or secure. The gap between those two things is precisely what an experienced developer sees — and what someone who skipped years of routine work may not.
What that gap actually feels like
Let me be specific, because this is easy to dismiss as abstract.
Early in my career I inherited a database for a system used during Special Olympics competitions. The schema was not in third normal form (3NF). I knew it was wrong — I had learned 3NF at university and could see the problems clearly. But the system was somehow functioning, and the implicit message from the codebase was: this is how it has always been done. So I left it.
Two years later I paid for that decision in a way that I have not forgotten since. The kind of pain that does not come from reading about a mistake, but from living inside it for long enough that you understand exactly how you got there and what it cost.
That experience changed how I read database schemas. Not because I learned new theory — I already knew the theory. But because I now had a specific memory attached to the principle. A cost I had personally carried.
The same pattern repeated with memory management in C++, starting with my diploma thesis. Until you have chased a memory leak through a system that runs for weeks, or debugged one on an embedded platform where the tools to catch it were either expensive or unavailable, you do not really know what “write code that doesn’t leak” means. You know the words. You do not know the weight behind them.
An AI tool would have hidden both of those consequences from me — at least for much longer. It might have helped me avoid the immediate mistake. But it would also hide the lesson from me. And the lesson was not the mistake itself. It was the understanding of why it mattered, built from having to fix it with my own hands.
A contrast from hardware
Chip design offers an instructive comparison. The path to designing a processor is long by necessity: digital logic first, then subsystems like ALU and memory controllers, then verification on FPGAs before anything goes near a fabrication plant. Juniors do not get handed a full block to design until they have demonstrated they understand what happens underneath.
This is not culture. It is economics. A bug that ships in silicon takes months and millions to fix. That cost creates a natural brake on skipping the fundamentals — and a strong institutional instinct to preserve the junior-to-senior pipeline even as tooling becomes more automated. Experienced chip designers are notably conservative about letting AI tools substitute for foundational understanding, precisely because they know what happens when someone does not understand what those tools are doing beneath the surface.
Software has no equivalent brake. The cost of a mistake is lower, the feedback loop is faster, and the pressure to ship is higher. Those are usually considered advantages. But they also mean there is no natural forcing function that says: you need to understand this before you are allowed to move on.
What disappears without an announcement
This is where the real risk sits. It does not look like a crisis. It looks like efficiency.
Companies will hire fewer juniors because AI can do what juniors used to do. The seniors who remain will have built their judgment in an era when things were written by hand. In ten years, they will retire. And who comes after them?
Someone who has never seen what a poorly designed database layer looks like from the inside. Who has never spent a week tracking down a race condition. Who knows how to talk about architecture — not because they have read about it and the AI confirms it — but who has never carried the consequences of a bad call in production.
Human oversight as a strategy only works if the human has something to contribute. Otherwise it is just a signature on an output no one fully understands.
The loom replaced weavers. Everyone could see that. AI may not replace developers — but it may slowly drain away the expertise the whole field is built on. That is a much quieter problem, and much harder to catch in time.
