The present and future of programming with AI (Keynote) | Peter Norvig, AI By the Bay 2025
[Music] Okay. Uh, thank you, Alexi. Great to be here. Great to have this opportunity to talk about the the present and the future. Remember what happened in 2017 and how fast we're moving. So I want to start off with this quote from one of our most distinguished computer scientists Edgar Dystra who says in this discrete world of computing there's no meaningful metric in which small changes and small effects go hand in hand and there never will be. So what he's saying is you know you have a program you change one bit you get a completely different program. That's very different than sort of the continuous real world where if you make a small change to the input, you get a small change to the output
Now, if he's right, we're in big trouble because then gradient descent's not going to work if uh small changes don't have small effects. So, you should be worried if you trust Edgar Dystra. But let's go to another expert, Arthur C. Clark, who says when a distinguished elderly scientist states something is possible, he's probably right. But when he says it's impossible, he could be wrong. So who do we go to to adjudicate that? Uh our third expert, Ken Thompson, who says when in doubt, just use brute force. And then we go into the data center and they say, "Yeah, we got this." So it is true that you can change one bit in a program or change one character in the source code of a program and now it's a completely different program. But it's also true that that doesn't happen that much
And so in fact you can treat them with this neural network approach. So now ancient history all the way back into the 2000s up to uh 2017. So there was this uh uh history in the 2000s of trying to generate programs by search over program trees. A lot of it done with genetic algorithms but other approaches as well. And you could do simple things, right? So if you wanted a program to uh add up numbers and return their mean, you could generate a program to do that and you could do things a little bit more complicated than that. But we kind of got stuck. Now I was working uh with a team of colleagues in 2017 and we said maybe we can use this neural net approach and do better than the traditional approaches. And so we looked at what was available at the time which was these recursive neural nets and we said we can't quite use them because we want these dynamic trees
We don't just want a static token after token. So, we built this system to do dynamic computation graphs. It was pretty cool. It worked and did what it did. But I got to say, I missed the boat. I was close to the right thing, but I missed the boat because just a few months later and and just a few uh doors down the hall, uh Ashish and Nome and their team were doing this attention is all you need approach. And that was really what happened, right? So I thought yeah for natural language you just want to take in tokens and process them because we don't know the grammar of English but for programming languages we do know the grammar so we should be taking advantage of the trees and I was wrong because what was most important was just having a technology that could flow the most data through and taking advantage of the fact that we actually knew the grammar didn't really matter because if you have enough tokens you know the grammar close enough anyways. Right? So, this is really this bitter lesson that Rich Sutton has says is don't try to be clever
Don't try to do this complex things with trees. Just shove more data through it. So, I was close, but I missed by a little bit. And they those guys won the prize. Okay. So, now uh a little bit less ancient history. 2023, DeepMind comes out with Alpha Code, which does really well on these uh programming contest type problems. And this was the one problem they decide to really highlight in their in their blog post
This thing uh called backspace. We won't go into details exactly what the puzzle is, but here's the program they wrote. Uh so the system wrote the uh program in the middle the stuff on the side that was uh human annotations and it gets the right answer right so that's great and it uh for most of these programming contest type problems it got most of them right but there was a lot of things wrong it was inefficient there wasn't very good analysis there were style issues uh there was no code review so I decided to do a code review and I kind of marked up almost every line there Right. And there's a lot of stuff going on. A lot of it is just style issues. Some of it is inefficiencies. And then there's one thing uh kind of right in the middle. I don't have a pointer, but if you see they uh they're dealing with these stacks A and B and manipulating them
And then they say, uh, well, I'm going to introduce a third stack C. And in the while loop, they say, well, whenever I pop something off of stack B, I better store it over in stack C just in case I might need it later. And then C is never used anywhere else in the program. And so I know programmers that do stuff like that and I don't want them on my team, right? But this was state-of-the-art then. And I call this the Lake Wiggon effect, right? So half the programmers are below average and that's what we trained on. So it maybe it's not surprising uh that the code is kind of average level, right? And at the time some people thought well maybe this is a ceiling. Maybe this is the best we can do. But that again that was ancient history
Now onto modern history. Uh so back in February this year uh Andre Karpathy introduced this term vibe coding and says how well it works for him. I thought he was being a little bit uh disingenuous. Right. So it's not that anybody can do this. Andre is super successful at this because when he gives a prompt, he not only understands exactly what he wants, but he understands exactly how to do it and he's just not doing it himself. So, his prompts are going to be better than than anybody else's. But we have these platforms uh that do an amazing job now
So, I asked uh and I went through sort of all the platforms and compared them and just picked out Claude here say solve alpha codes backspace problem. It goes and finds the problem. It gives a very good summary better than was available in the original problem. Uh gives a really nice uh program didn't have any of those difficulties that I mentioned from just two years earlier. Uh it's efficient, has good style, good documentation, passes the code review. So we made amazing progress in just two years and it's not just this one example. It kind of goes across the board. Here's another example that I thought was a little bit more challenging
Right? So, uh, this was back from 2024. There's a lot of talk of do LLMs have a theory of mind that is not only just what do they know, but can they reason about uh what I know that you don't know and what you know that I don't know and go back and forth. And there are a lot of logic puzzles that deal with that. There's a famous one called Cheryl's birthday. So, there's some initial facts and then Albert says, "Well, then I don't know where Cheryl's birthday is." And then Bernard says, "Well, I didn't used to know, but after you said that, now I know." And Albert says, "Okay, then I know, too." Right? So, it's definitely you can only solve this if you have a theory of mine. And so, in 2024, I asked a human programmer, me, I could solve it. But nine out of nine LLMs could not solve this problem because they did. They got confused between what they know and what different people know
Uh but back in May uh the majority of them could do it. Still not quite all. Maybe if I ran it again now, maybe the rest of them would be getting it right. So fast progress. So I want to talk about these AIdriven changes to the software development process. The who, the what, the when, and the how. So when I think the answer is now, right? So if you did it in 2023, it was probably more trouble than it was worth. If you did it in 2024, you're an early adopter and good for for you
2025 is really the time when this is going to help. It is going to make you better. It can't do everything, but the technology is here. Let you do it faster just in time. One thing I'm really interested in is doing it in real time, right? So it's still you make a prompt, you wait 10 seconds maybe if it's a big prompt 10 minutes or more. Uh and that's okay but it's very different than being in the flow right and past tools we have like autocomplete which tells you which methods uh you can call that has to come up in 100 milliseconds to keep you in the flow. So I think it'll be very interesting if we can get these uh LLMs faster and see what can we do immediately versus what do we do in batch mode. So stay tuned on that
So who's it for? It allows you to have smaller teams. It makes experts better. It allows non-programmers to actually produce stuff. And it seems like for prototyping it makes you 10 times faster. For the whole software life cycle it's not quite there yet. And we've seen differences of 20 to 50% improvement on that. We've also seen some uh negative results. Right? So, one survey said uh people became less productive
Uh I kind of thought of that as an investment. They thought the tools were so cool that they played with the tools rather than achieving the work. So, productivity went down, but maybe it'll go up in the future. Uh and then uh just yesterday uh Lus Toval says pipe coding is okay as long as it's not used for anything that matters. So basically he's saying he's not going to let it into his kernel. Uh but you can put it on the other stuff off to the side. And probably that's a wise choice for something as crucial as a Linux kernel. Now, there's all these roles in a software development team, and one approach is to say, I have a separate agent for each one of these roles, and they're all going to communicate with each other
And we're starting to see IDE that can do that. It's not quite integrated yet. So, maybe I should have said 2026 is going to be the year in which we can have all this happen. So, so maybe you have, you know, an AI software developer, an AI front end, a backend, and so on, AI tester. Uh, or maybe it's each feature, right? So, maybe you it's as you assign one feature to one programmer, maybe you'd have an AI that has that and and it keeps all the context and and does a better job of really focusing on one thing. I think we don't yet know yet how to aggregate this ensemble of teams and make them into the most productive way and there'll be experimentation with that going on. Now I took the standard graph of the software development life cycle plan analysis design implementation test maintenance and then go around again. Now one thing that's really awesome about these neural net approach is that you can do back propagation through the your implementation
and you've got this neural net, you put in some input, you get the wrong output, you can feed that back and now it gets better. I want to do that for the whole life cycle, not just for this tiny little neural net that's a small part of that, right? I want everything to feed back, right? So, if there's an issue with the design document, that should be connected to the code and we should be able to uh get feedback something's not right and and close that loop. Uh, I got to say I've been playing with the vibe coding of various kind for a long time, but I got to say this is probably the first sort of real app that I've built. It's based on uh my previous book paradigms of AI programming which the the copyright reverted to me. So I'm able to do whatever I want with it. And uh there's a site called li little piper.ai AI in which I have a Socratic dialogue to interact with the book, write code, get criticism on it and so on. And basically, you know, I said, I don't want to write another static book. I want to only write interactive experiences and this is the first attempt to do that
I didn't know npm and node, but we were able to build this with my colleague Peter Dannenburgg through the help of VIP coding. All right. how define programming as coming up with step-by-step instructions to do the job. But if you define problem solving, it's a process of achieving goals by overcoming obstacles. And it really looks like that's what's more important, right? It really wasn't important what the steps by steps were. It's important can we get the end result? Can we overcome the obstacles? So maybe we want a language that talks more about that process of overcoming obstacles and less about you know incrementing registers and so on. Uh here's a p picture of a programmer. You might have seen this before
This is Margaret Hamilton who was credited with inventing the term software engineering and was in charge of the Apollo software project. Here she is standing next to the stack of assembly language code from the Apollo project. And fun fact, Margaret was my boss in my first job out of college. How cool is that? But I got to say, this picture keeps on coming up and I remember those old times and it's it's great to have those memories. But this picture also reminds me of this picture, right? And it feels like sometimes, you know, you really feel like you're strangling that code and it's not doing the right thing and you're abusing the code to try to get it back in shape. And Darth was a micromanager. Nobody likes that. He never won boss of the year award
So maybe we want something that gets away from that a little bit. So if it's not not a micromanager, what is it? Maybe you want to think of it more as a teacher or a partner or a community that works together to try to get your code to work. And you know, we have this book, the design patterns. It was very influential. But the gang of four got that name, design patterns, from this book, which is about architecture and designing things in the real world, designing buildings and cities and towns and you know what makes livable communities. So maybe our design pattern should be more like that. How do we make a livable community and not step-by-step instructions? This book by Hansen and Susman, software design for flexibility, says maybe our systems should be more like biological systems. Biological systems use contextual signals that are informative rather than imperative
There's no na master commander saying what each part must do, right? So I have a brain but it's not like my brain is saying okay at this step each part of my body is going to do this thing and then at the next step it does the next thing. Rather all my parts are kind of communicating with each other back and forth. There is no central control. So maybe our software should be more like that. So what well we started off with assembly language because the expensive thing was a CPU and we wanted that to be efficient. Then we made high level languages because the expensive thing became the programmer. We wanted something that worked for them. And LLMs can certainly learn these languages, but they were designed just for the human and maybe they're not the best language for human AI partnership
Maybe we should have more specialized notations, right? So we have notations in math and chemistry and music and all these other fields. uh maybe we should incorporate whatever notation is appropriate for the job and be able to learn with that rather than say you have to write in Python or in JavaScript. So I want something uh that's a conversational language for this human AI partnership keeps track of everything we have to know. Right? So it's not that we got the code here which really matters and then off to the side there's some documentation. I want that all to be part of the conversation. I want you to be able to choose when to be precise and when to be vague. And I want to be give advice rather than dictate commands because advice is more flexible. advice can be compiled down into commands for now, but later on when the world changes, the same advice can be compiled down to different actions
Okay, so we have these AI idees. Andre again uh said the hottest new programming language is English. Uh Google anti-gravity came out yesterday. I've been playing with all of these. uh they make things easier and they just generate better descriptions right so you know we have all these companies that say to the developers make sure you document and so on and make sure you write things down and they say ah I don't want to do that these IDEs do a much better job of that uh so that's the programming language then finally what do we want for our operating system so we've gone through a couple errors right so I'm old enough that I came in at the very tail end of this mainframe error and then we had PCs and then we had mobile OSS and we're in this funny situation where there's all these buttons on my phone and when I tap one of them I've given up complete control over my phone and all my private information and everything else to the company that that installed that icon and somehow I trust them. Right? And I never read these screens. Do you read these screens? No. Right
We just trust that it's gonna work and if it seems shady, we delete them later. And maybe we can do better than that, right? Maybe we can have instead of having 50 buttons on my home screen, I got one button and say now I only have to trust one thing, which is my personal uh LLM operating system assistant. And so I can trust it. And if I believe it's on my side, now I'm all set. And I no longer have to say I'm going to give up everything to Uber. Rather, I can ask this one button. Well, go out and negotiate with Uber and Lyft and whatever taxi services there are and find the best one for me and yeah, you got to share my location and my credit card if I want to take a ride, but keep all my other stuff private to the extent that you can and operate on my behalf rather than on Uber's behalf. And I don't know if we're going to end up with something exactly like that, but it's certainly a possibility that I think we should think about
All right. So, let me stop there. We got a couple minutes left uh for questions and discussion. [Applause] >> Thanks for the nice presentation. Uh so, so in one of the slides you mentioned software development has multiple roles and each one can be a different agent or something like that, right? But there are certain aspects of software engineering. People used to uh like it they wish it was done that way but but there was human elements which were made making it impossible or harder like for example formal methods and uh and all right. So do uh uh like do you see any uh any changes in that instead of just automating what humans were doing to be faster than things which humans couldn't do? >> Yes, that's a great question. Right
So we've ended up we've we've tried lots of ways of making the software process work and we've ended up with something uh that seems to work pretty well. But as you say, we did that in large part because of the limitations of humans, right? They only have so much uh attention and memory and speed and so on. So we broke up jobs with that limitation in mind. If the limitations of machines are different, then maybe we should have a different organization. uh I think in the short run we probably will want to uh duplicate what we have because all these systems are built on training data and we have training data from the way things were. So for the present trying to duplicate and get to uh where we are make everybody faster and better and better communication that's probably the right role but I think you're right that in the future we can say uh now that we have different underlying technology maybe we should take a different approach and and you know we can all together try to discover what that better approach might Uh hello Peter. Uh great talk. I'm John
I uh find what you had to say uh is extremely resonant because I think as we think about that AI or that computer human interface, we need to think in terms of uh concepts that we understand as humans like people and assigning agents roles in a dev team in the SDLC. I get it. That's how we're thinking about uh how to increase automation. One of the things and I also find that your discussion of community is extremely important because if you think about how highly effective people work together, it's in community. It's, you know, a pizza sized group of team members. So, there's a lot of things we've learned about how humans work well together. And I wonder to what extent you've looked you talked about notation and specialized notation but have you thought about uh how we communicate effectively as humans and there's this great quote that's attributed to George Bernard Shaw and I don't know that it's actually his but the single the the quote is if you've heard it is the singest the single biggest problem in communication is the illusion it has taken place at all. >> Yeah
>> Right. And I got that from Alan Alda's book on communication. >> And if I look this way, does it look like I understand if I had this look? So just to what extent are you have you uh explored the space of communication and community as a way to shape the problem of problem source identification and problem solving and observable challenges that we can overcome because I think that's where the money is. >> Yeah, I think that's a great question. I also like the the Alen Alda book. Uh you know I was impressed by uh some of these newer idees and the way they can write documentation and so on. It just seems like a big improvement just over the last year. Uh so I think that's important
Uh you also talked about this idea of understanding. uh and I certainly see a generational switch in how we deal with understanding versus overload. Right? So several times I've been in this position where I'm working with a younger colleague and we say, "Oh, here's this new software package. Looks like it's going to do what we need to do." So I start reading the documentation, trying to understand how it works. And they come back a couple minutes later and say, "Okay, I'm done. Let's move on to the next thing. And I say, 'What do you mean?' And they said, 'Well, you know, we call this method, it gets this result, you reformat it this way, now we got the answer. And I say, but how does this package do X, Y, and Z? And they say, well, no idea, but I got the right answer
And I feel like sometimes understanding how things really work deeply is important, or else it's going to come back to bite you later. And sometimes it's just a conceit on my part to want to understand that and we should move ahead because it's a big complicated world and I can't expect to understand everything in the world. I should just get it to work. And I feel like there's a tradeoff between those two. And I feel like both of us didn't make a rational trade-off. We just went by our history. Right? So my history was I was used to understanding things deeply. Their history was, I'm used to lots of things coming at me and and you just get it to work
And neither of us probably did the optimal tradeoff. So maybe getting help to do that better might be important. All right. [Applause] [Music] [Applause] [Music]