It’s a Feature Not a Bug: Why Code Won’t Disappear

First off, Chris’s recent article “Will AI Replace Software?” is great. It’s exactly the kind of forward-thinking perspective that helps us collectively imagine possible futures in tech. Chris always had a knack for identifying paradigm shifts before they become obvious, and his exploration of AI bypassing human-readable code entirely is fascinating.

But (you knew there was a “but” coming)… I think there’s a perspective worth exploring from the other side. While I share Chris’s excitement about AI’s transformative potential, I believe the continued existence of human-readable code isn’t just a transitional phase but a fundamental necessity.

The Incomplete Language Analogy

Chris draws a compelling parallel between programming languages and human languages, noting how both use abstraction to express increasingly complex concepts simply. But I think this analogy actually points to why code won’t disappear.

Human languages are inherently ambiguous, contextual, and evolving. We embrace these characteristics in everyday communication, but they would be catastrophic in a system controlling critical infrastructure, financial transactions, or medical devices. Programming languages evolved specifically to provide what natural language cannot: precision, unambiguity, and consistency.

This isn’t a limitation we’re waiting for AI to overcome – it’s a deliberate feature that serves crucial purposes.

Code as Communication, Not Just Computation

When we write code, we’re not just telling computers what to do. We’re communicating with other developers, with our future selves, with auditors, with users. Code repositories aren’t merely functional artifacts; they’re organizational memory and collaborative canvases.

Imagine if Shakespeare had written his plays directly as emotional states to be induced in audiences, bypassing the “intermediary” of written language. We’d have lost not just the ability to perform those plays, but the cultural dialogue they’ve enabled for centuries.

Similarly, if AI began generating machine code directly, we wouldn’t just lose the ability to understand how our systems work – we’d lose a shared medium through which humans collaborate on computational problems.

The Deliberate Constraints of Programming Languages

Programming languages aren’t just simplified ways to express machine instructions; they’re carefully designed systems that impose beneficial constraints. When we choose a strongly-typed language, we’re deliberately limiting what can be expressed to prevent entire categories of errors. When we adopt functional programming paradigms, we’re consciously embracing constraints that make certain bugs impossible.

These constraints aren’t just training wheels we’ll outgrow. They’re more like the rules of chess – restrictions that paradoxically enable deeper strategy and creativity. By limiting what can be expressed, programming languages actually expand what can be reliably built.

Embracing the Best of Both Worlds

Rather than AI replacing software entirely, I suspect we’re heading toward a more symbiotic future. AI will likely transform how we write and interact with code, but the code itself – as a readable, verifiable, shareable medium – will endure.

We might see AI helping us navigate increasingly higher levels of abstraction, automating the translation between human intent and formal specifications. But those specifications will still need to exist in a form humans can reason about, debate, and improve.

In this future, code becomes more like legal contracts or architectural blueprints – documents with formal properties that serve both practical and communicative purposes. AI might help us draft them, interpret them, and implement them, but their existence as human-readable artifacts remains essential.

The Long Game

To be fair, Chris acknowledges that his vision might seem “expensive, unreliable and mildly stupid to attempt today.” I’d argue it will remain so tomorrow and beyond – not because of technical limitations, but because of the enduring value of having computational systems we can collectively understand and reason about.

The history of computing isn’t just about increasing automation; it’s about expanding what humans can comprehend and control. From punch cards to assembly to high-level languages to visual programming, each evolution has made computing more accessible to human understanding, not less.

I believe AI’s greatest contribution won’t be eliminating the need for human-readable code, but dramatically expanding who can participate in writing it, reading it, and using it to solve problems.

Conclusion

So while I love the boldness of Chris’s vision and the important questions it raises, I’m betting on a future where code endures – not as a relic of pre-AI computing, but as an essential interface between human and machine intelligence. The abstraction layers and human readability of code aren’t bugs that AI will fix; they’re features that will help AI and humans work together more effectively.

What do you think? Is code destined for obsolescence, or will it evolve into something new but still recognizably code-like?

This post was written as a friendly counterpoint to Chris Loy’s excellent article “Will AI Replace Software?” which I highly recommend reading.