Transcript: DevReal: Charles Frye Interview

hello everybody I'm Alexi ker the founder and organizer of Bay Area AI which is the longest running deepest technical biggest a up in the world running in Bay Area since 2015 and tonight we're on location at WS J Loft and we have an amazing event full stack open source AI from chips to apps and with us we have Charles fry who is uh the guy model yep and I must say that full step comes in no small part from Full step D learning which Charles founded and ran successfully also uh we did that as a part of uh scale by the bay conference which run here yearly and uh I must just jump in because you spoke at um uh scale by last year and you came up with this notion of lmos yeah which seemed extremely prophetic then and like people are coming around with this idea now so can you tell us you know how do you see lmos evolve and yeah um yeah I think yeah the original idea I think the terminology is maybe Andre Kathy's originally but the like core idea is that the operating system is like uh wraps the hardware and makes it more approachable for humans uh and like like Hardware is like completely asynchronous and you know like programming drivers is so much more difficult even than like low-level operating system other like lowlevel systems programming and so the operating system just like takes all that and makes it like makes it approachable to humans and and encapsulates it and I see like there's like a similar opportunity with like more intelligent machines machines that have more semantic understanding to put like another rapper around uh now now around the like machine language that we invented that was like our original bridge between humans and machines um and I think like there's a couple of like cluster related ideas like natural user interfaces which has been around for a long time but got picked up again by people like Sam Whitmore and Jason Yen um and I think like to to an extent part of that the excitement around that was with this big release of super capable Foundation models and like a bunch of like kind of en visioning what the future of computing looks like I think you know that it's the devil's in the details of making something like that work has turned out to be you know a little bit harder than just like combining a couple of Lang chain programs together um and so like while I still have that as like my big long-term vision of where we're going as a field of making more human uh machines it's uh it's sort of like receded into the Horizon is like here's where we're going um here's where we hope to be in 10 years yeah interesting and so you know we talked about lmos and it having memory right it having State and so Eric Meer started this Lang universales and he talks about AI native programming languages where LM is the runtime of these languages right and so recently I came across uh start up uh I think they autogen people and they talk about agent OS right so what do you think are going to be the elements of this like for instance are agents the elements of this like what do you see is like units uh uh and and obviously like where are the gpus like are they so deep in the bottom I don't even see them yeah I think I think a lot of that is going to have to recede uh I think there's kind of two parts one is that some stuff is going to be super it's like less time sensitive it's maybe more beneficial to have a lot more int it's like adding more intelligence makes it better uh like without bound and those kinds of workloads will move will like remain in cloud data centers which is where the majority of like neural network executions currently live um but despite being at a cloud provider and like uh and and really like thinking about and focusing on that kind of deployment like there's another side that's actually probably where a lot of this like humanizing uh machine interfaces is going to happen which will just be like purely local and I think like apple is already Dem already like accidentally stumbled upon the right formula like high bandwidth memory um and and like a GPU with unified memory between this like with the uh with the main processor to get like very like buttery smooth experience Nvidia seems to be going in a similar Direction with digits and their like other sort of like Jetson platform uh and like Orin uh platform stuff uh and so I would see like a lot of that actually ending up ending up local like it's it's just a thing that computers do and like we keep getting the that the like power of those things keeps getting greater the our ability to squeeze intelligence into smaller and smaller modules so neural networks with fewer parameters or with faster inference uh gets better and that's how we get like more intelligence at at the edge um or like at the at the users compute so I mean that sounds great and like it already kind of gives us a lot of details you know from you know sub nuts like you talk about gpus and local compute right and then we go all the way to OS so I'm really curious about your personal uh path in this because you know with like you you did the PD berlay and you did the Deep learning which was a high level concept and now basically and you talk about this ground vision of LMS and then you end up with model which is really close to the metal yeah right and and so but then you obstruct uh the metal for the developer so can you tell me like about your personal Journey like why is this a sweet spot for you what makes it fun yeah yeah there's a there's like a lot of things that make it fun I think the you mentioned full stack deep learning a couple of times like kind of what happened is I was teaching that class that was like mlops and like deploying an operationalizing machine learning um after you know working on that at weights and biases for a couple years uh and teaching people that class and like I had like a pretty sad story to tell them when it came to deployment it was like well with computer vision models that do classification you can squeeze them down small enough to fit to like run fast on a CPU so then just put them in a Lambda an aw like on AWS like run them serously on a CPU like unfortunately there are no options with serverless gpus and that's what that was like 2022 and I was like man this is a startup shaped problem there has to be somebody working on this and so I found banana replicate modal um tried out any scale lightning AI a bunch of these sort of like newer like Cloud providers or or like Cloud abstractions um with a focus on GPU and gpus and data intensive workloads and modal was the one that I like enjoyed using the most and that I kept coming back to um and that most importantly like extended my like vision of what was possible with computers or sort of like extended my Powers um it was like oh wow I can turn this I don't just have to like run this into jupyter notebook or run this locally like it's actually so easy to turn this into a service I can just do it right away like oh I should be doing this all the time like I should be making tiny little apps to solve all kinds of problems not even just one that have big neural networks in them but just like yeah I have like a little yoga tracking app that I made in about 10 minutes using Claude and then deployed on motal and now I like track whether I'm keeping up with my like personal yoga goals and that was like you know that the like Cloud magic of like easy deployment plus the like magic of foundational models has like is what drew me into like using modal over and over again telling people they should use it and then eventually joining um and then it's been an opportunity to just like really dive deep work with some of the like you know best um you know some of the like frankly best engineers in the world on really hard like you know really hard problems and on you know learn more about like the nature of computers and then get to share that with people that's exciting and I mean you've been out there like really teaching people a lot having a great following people flew in SFO specifically for full step dear from around the world which was amazing to me right like it was a local workshop with mostly people from around the world so uh so I'm curious what do you think of the role of devels in the revolution how do you see kind of us collectively teaching the developers at large all these things what works what doesn't where should the developers focus when learning about this yeah it's a lot of lot of great questions I think like internally from like the you know developer relations as like an industry or as a as a community of people solving similar problems it's like there's evangelism and advocacy uh and in between them is like relations so like an evangelist is going out there and like sharing how to use the technology with people and like teaching people this like um BTO malsky is like a hasal evangelist I think is his title um and he's like out there like getting excited about functional got me excited about functional programming um also Shar a of great Parisian food oh yeah that's uh yeah he's great Twitter um great social media presence in general um and then there's also the like yeah on the other side is the advocate who is like representing the needs of developers internally and I think yeah Kelsey high tower is maybe the best example of this um it's like your primary goal is actually to like hold the organization to account a bit it's like you are the developers representative inside the company that's the kind of that's like that's the title that I chose like and that's the direction I would definitely want my career to go because I think it's like that is what delivers the right balance I think for me between like value for other people as in like expressing their needs and giving them like you know helping them have a seat at the table um and then also D driving value for the organization which like they actually do want those opinions they do want that information and they struggle to get it through different means through support or through Engineers or through like you know other mechanisms of like Word of Mouth um so like in this particular AI world I don't know that there's like a specific thing I think maybe one of the most important things is just like cutting through the honestly because there's a lot of like linked influencing there's a lot of like um yeah Naro well crypto refugees entering into the latest hot new thing and people who have like a prominent following and who are sufficiently technical to be developer Advocates evangelists or relations people uh like have the ability to like you know cut like silence those voices or amplify other voices and like be TR more trusted Source more High signal um so I think like that's particularly important in this in this moment in this field where there's so very much noise I totally agree and thank you by the way for helping us put this together because I think personally open source and it's like self-evident right we hold this Tru self-evident and we show people code and in this case we actually put together a stack right and we show people how to do this basically as a as a stacks and I think to me this like Integrations is the most we create network effects MH and you guys are running a lot of this stuff so thank you for that and looking forward to your talk and thank you for being such a great part of the community yeah yeah thanks Lexi always a pleasure thanks Charles for