Most AI breakthroughs are thought of as isolated advances, but Karpathy argues they signal a broader shift: code is being replaced by neural networks trained on data.
“This reframe changed how I think about software. Writing code is increasingly the wrong level of abstraction — you train the behavior instead.”
4 comments
The section on the 'new paradigm' advantages is what I keep coming back to — in particular the bit about optimizing over a program space that's fundamentally different from hand-written code.
Two years later and this is even more true. The speed at which 'write code' is being replaced by 'describe what you want' is staggering. Software 2.0 is becoming Software 3.0.
What Karpathy didn't predict is how fast this would move to non-technical users. My marketing team is building data pipelines with AI tools now. No code, no neural nets - just natural language.
The compliance implications here are fascinating. Every CTO should read this.