The Great AI Illusion: Training on Trash, Paying Twice, and the $80 Billion Hangover


It seems the tech industry has the memory of a goldfish.

Just a short time ago, the absolute peak of corporate strategy was the "Metaverse." Mark Zuckerberg bet roughly $80 billion on the idea that we all wanted to attend budget-meeting standups as legless VR avatars. Now, after quietly burying that financial crater, Meta is cutting more jobs and aggressively doubling down on the new savior: Generative AI.

The pattern is impossible to ignore. We are witnessing a massive, industry-wide FOMO (Fear Of Missing Out) cycle. But as companies eagerly fire engineers to free up capital for Nvidia GPU clusters, they are ignoring the fundamental mechanics, economics, and architectural realities of what they are actually buying.

AI investment is not a straight line to infinite productivity. It is a highly expensive, deeply flawed cycle. And for software developers, it is the ultimate job security guarantee. Here is why companies are about to pay twice.

The Meter is Always Running

The illusion of AI is that it is a one-time infrastructure purchase that yields infinite free labor. The reality is that the meter is always running.

Compute is not cheap. Integrating AI into traditional, legacy systems is a monumentally expensive and complex task. You cannot just duct-tape an LLM to a 20-year-old monolithic architecture and expect it to magically optimize your database queries. Rebuilding or adapting these legacy systems to play nicely with AI agents comes with exorbitant costs, massive operational risks, and absolutely no guarantee of success.

Furthermore, the daily operational costs are skyrocketing. API rate limits are getting tighter, compute is bottlenecked, and yesterday’s premium features are rapidly becoming today’s baseline.

Prompts are literal money. Every time a Product Owner or a middle manager sits at a terminal and blindly iterates through 40 different prompt variations trying to get the AI to spit out a functioning microservice, they are burning cash. To know what to prompt, you need to understand the underlying architecture. To know if the output is actually secure and performant, you need a developer.

The "World Literature" Fallacy

To understand why AI will fail to replace senior engineering, you have to look at the fundamental nature of a Large Language Model.

No matter how much marketing you pour over it, an LLM is, at its core, a stochastic parrot. It is a highly sophisticated next-token predictor. It doesn't "think." It guesses the statistically most probable next word based on its training data.

When it comes to human language, LLMs are undeniably brilliant. Why? Because they have read the world literature. They have consumed Shakespeare, Tolstoy, encyclopedias, and centuries of meticulously edited, published human thought. That is why an LLM can write a beautiful poem or draft a flawless corporate PR apology. Its training data for human language is the best of the best.

But code is a different story entirely.

The AI did not learn to code by reading the "world literature" of software engineering. The most robust, battle-tested, highly optimized enterprise code in the world is proprietary. It is locked behind corporate firewalls, VPNs, and NDAs.

So, what did the AI learn from? GitHub.

It learned from a gigantic, public mountain of shitty pet projects. It learned from abandoned computer science homework, half-finished bootcamp clones of Netflix, unoptimized React to-do lists, and StackOverflow copy-paste answers from 2014. The AI's concept of software architecture is fundamentally skewed because its training corpus is dominated by amateur code, not highly optimized, bare-metal enterprise logic.

The COBOL Reality Check

If you want proof of this training data bias, look no further than the recent attempts to use AI to "modernize" mainframes.

IBM and other major players recently had to quietly confess that using Generative AI to automatically translate and modernize legacy COBOL into Java or C# doesn't actually work very well.

Why? Because the LLMs simply don't have the dataset. Real, complex, perfectly tailored COBOL—the code that actually processes billions of global credit card transactions every night—lives exclusively on isolated IBM mainframes in banking basements. It is not sitting in a public repo.

The only COBOL the AI has ever seen are basic "Hello World" examples from university archives or hobbyist projects. When you ask it to refactor a massive, multi-threaded DB2 batch processing job with strict memory constraints, the AI hallucinates, because it has never read the "world literature" of mainframe engineering.

The Boomerang Effect: Paying Twice

We are currently in the first phase of the cycle: The Purge. Companies are laying off developers to afford the AI hype, believing that a Scrum Master armed with an enterprise ChatGPT license can replace a senior backend engineer.

But we know exactly how this ends. It is the Cobra Effect all over again.

Without developers to architect the system, non-technical staff will prompt the AI to generate vast amounts of code. Because the AI is trained on fragmented pet projects, it will generate fragmented, unmaintainable, insecure spaghetti code. It will lack proper error handling, it will misuse memory, and it will fail catastrophically under real-world enterprise loads.

And then comes the second phase: The Panic.

When the systems grind to a halt, when the data leaks occur, and when the AWS bill spikes by 400% because the AI generated an infinite loop of inefficient database calls, the companies will have to fix it.

And who are they going to call? Us.

The developers who actually understand the bare metal. The engineers who know how memory works, who understand bitwise operations, and who know how to write a raw SQL cursor without an ORM.

The companies will have paid the high stakes for the AI illusion, and then they will have to pay a massive premium to re-hire the developers as consultants to untangle the AI's mess.

So, to my fellow engineers: don't panic. Keep writing C++ (or like me Cobol as well), keep learning how the machine actually works, and get comfortable. Job security has never looked better.