Bay.Area.AI: Interview with Michael Ryan, Stanford
Recording: Bay.Area.AI: Interview with Michael Ryan, Stanford
hello everybody my name is Alexi kov I'm the founder and organizer of B AI which is the most established longest running deep technical metop on AI in the Bay Area and the world run for 10 years we've been NLP KD machine learning now ai and we do it from location GitHub headquarters we have like top uh AI developers coming to learn about interesting projects and I call it DP you can tell me what you call it is been top of mind uh I uh found found discovered Tomar kab who was who created thisp student of M Zakaria we we launched the very first Spark me up when I found spark in 2012 we launched mate at you know at SCA as spark so then I've seen Omar and he spoke at our conference Kill by the way and then now Omar told me he wants more folks working for thisp to talk about it and here come Michael as one of the top developers and Michael is a master student in Stanford that's correct so Michael tell us how did you get to do disp and like what's what made you you know like what's fun about it and how come like you made one of the key components with yeah for sure so I got involved with so I call it DSP I think any pronunciation kind kind of worked but um I got involved with DSP when I got started at Stanford um so about a year ago I was taking a class on meta learning and one of the projects that we decided we wanted to work on was like prompt optimization and I had just seen a talk from Omar at the NLP group about dspi and how he was working on especially F shot bootstrapping optimization for language model programs in dsy so I was working with somebody named Christa on this project uh we were just both students in this class and I reached out to Omar and said like we want to work on prompt optimization and you said this is maybe like a next step for DSP so maybe we can collaborate on something here and immediately we got to work on brainstorming and that's how we started making something called the co-pro optimizer uh which has now been kind of deprecated in favor of the MEO Optimizer and that's what the talk will be on today is our Meo Optimizer which is like the state-of-the-art optimizer for language model programs in in DSP and so some of the stuff I'm really excited about with DSP I think it's such an amazing framework for just considering what you should uh sort of conceptualize for a language model program so the idea that you can break it into your your uh structure of your program into inputs um a metric and the program itself and then optimize the entire system end to end I think having that abstraction removes all of the confusion about what should I optimize is it the weights is it the prompts um and what uh different data do I need to collect and how do I actually optimize this do I need a gradient signal DSP actually can kind of take all of that confusion and allow you to just write a program that solves the task that you want to solve and optimize it for you in the best way possible so you have our prompt optimizers you have weight optimizers and we're working on like RL based optimizers too so I think that just the abstraction itself is really cool and it's empowering people to do really amazing things with language models that you can't do with just a single language model like you have to build a system a compound system awesome you know it's interesting right like I was spending this mic and Industry world like I did a few moves back and forth you know did my PhD in computer science at un of Pennsylvania kind of you know went back and forth to Industry and like finally went back you know to Silicon Valley so like I kind of understand how it works right academics live in their own world and they really like isolate themselves with very smart people and they like Ste in their own kind of environment and so you're kind of doing this but you bring this open source to the world right the people start doing stuff with it and they not at Stanford right and like some of them don't know basics of this so I wonder when you like do you uh interact with the usual developers or the regular folk and like does it inform how you build uh gpy yeah definitely so one thing I'll say that's amazing about dspi is the community that's formed around it and I have to attribute that to Omar uh he does an amazing job building the community uh actively involved in the Discord obviously on X he's very um well presented late interaction yes yes at late interaction um so I think I get a lot of experience interacting with people outside of Academia because of that and actually I've been very fortunate this summer I'm interning at snowflake so I'm working with Folks at snowflake who are also using dspi so I'm getting that perspective um a bunch of startups reach out and uh ask U we're implementing this in dsy how can we do that and I think it actually gives me a great perspective to uh stepping out of Academia a bit and seeing how people are actually building language model systems in industry and how how can we optimize those and one thing that we're working on right now is actually a sort of Benchmark of language model programs and how people are actually using them in Industry because we want to show that our optimizers work in real world settings so I'd love for more people to reach out if you're using dspi let me know how you're using it because we might add your task to our Benchmark and show like which Optimizer which language model is best for this particular task fantastic it makes me very happy because you know first of all no bunch of folks at snowflake a lot of my friends went to work there right it's a big company like a lot of developers in all areas so now you know uh I'm a j which is the category defining graph database and I don't know anything yet about apply disput to graph databases but I wonder if you think that you know we can add that to your stable of tasks right because you see relationally to data Lakes uh do you think uh the graph databases make a good use case for disp I think that would be a great use case actually I think um dpy started off with um you know Omar has this background in retrieval augmented generation and retrieval as a whole um and so a lot of the initial implementations in dspi were for optimizing rag systems end to end um and I think plugging in a a graph database instead of a relational database or instead of a CO bear index or you know I think that is a really cool way to see um potentially now we typically optimize the language model part parts of the the rag pipeline so like rewriting your query rewriting the generation step I think plugging that in with uh with graphs uh graph databases is like a really interesting use case yeah so the interesting thing I'm going to you know show a little bit in in my talk on graph rag today that we you know in addition to to textual prompt obviously we have we build a knowledge graph on right on on the fly or like from document collection and so we can actually add uh Cipher query so Cipher is a graph query language which new for uses so very similar to SQL right so can you optimize SQL query in addition to the texal prom yes yeah we can optimize that fantastic so then then we probably can figure out how to plug in CER of EX which is now also a standard called gql right almost replace one letter you have a different query language fantastic so and uh one more question about D and then I just ask you a fun question uh so uh like we really have a lot of developers here and so we really need to figure out right how do we kind of help them learn all of this like this overwhelming mod information Academia is a machine optimized for knowledge consumption you drink from the high fire hose you read papers you have people who do this together like this you have special skills superpowers you know how to kind of very quickly triage information right a lot of developers do not do this it may be difficult to them like what advice do you have for developers who want to come into DP do they need to read papers can they just take code and play with this like and they take your kind of examples what's a good entry point how they can start learning and incrementally learn more and feel happy yeah that's a really great question and we're working on even more documentation for dspi I think the best way to get started right now is go to the repository and we have an examples folder um where we have uh collab notebooks about here's how you use the MEO Optimizer here's how you use assertions um and you know here's how you do fine-tuning in DSP and just go through those notebooks they're already set up to work out of the box in Google uh so that's a great way and I think watching talks like the one I'm going to give today will give the insights into how Meo Works behind the scenes and what dsy is actually doing to optimize your language model program so I think just watching this video doing the collabs that's the best way to get started fantastic well and last question tell us some fun fact about yourself what do you like to do for fun yeah yeah so a fun fact about me is I have I guess I would say three different licenses so I have my driver's license that's that's obvious um but I also have a boating license in Florida because I wanted to drive a jet ski so I took the course uh at the age I wanted to I had to take the course and I also have my Scuba diving license which is something I like to do for fun um although the water's very cold out here in the bay so I don't get to do it too often well fantastic so you're like a very licensed guy and so you know you're doing your master so you're going in the industry so we wish you you know uh luck and uh really uh looking forward to learn from you looking forward to your J thank you so much thank you Michael cheers