Bay.Area.AI: DSPy: Prompt Optimization for LM Programs, Michael Ryan
Recording: Bay.Area.AI: DSPy: Prompt Optimization for LM Programs, Michael Ryan
hi everyone I'm Michael uh I am working on dspi I am a master student at Stanford and this summer I'm a research intern at snowflake uh so I got involved uh with dspi working on the prompt optimization part of dsy along with Christa opsal along and Josh pel uh so we make up the prompt optimization team within dspi um and the title so because I work on prompt optimization the title of the talk here is dspi prompt optimization for LM programs but I hope by the end of the talk I convince you that the power of dspi is actually in the abstraction behind it and how it makes you think about your problems uh and then the optimization itself can be abstracted away to the framework so right now we do prompt optimization and fine-tuning but if something comes along tomorrow a brand new way to adapt language models dspi can adapt to that as well um and so you're not stuck in one optimization framework so it's never been easier to build AI powered applications uh so for example chat gbt is the one everybody knows um and if I ask where is GitHub headquarters it accurately tells me where we are right now however turning mon lithic language models into reliable AI systems remains challenging so if I ask chat EBT when was Stanford University founded it'll tell me uh actually the incorrect date that's the date that it opened not the date that it was founded and if you ask who it was named after it'll tell you leand Stanford when in fact it was named after Leland Stanford Jr so uh llms are prone to hallucinations uh we know that that's an issue and this has real world consequences so this is a story of that is well known now that Air Canada had a chat bot that was doing customer service and it told a customer that they could get a discount uh and then Air Canada didn't want to honor that discount because the chatbot made it up and it got taken to court and they actually did have to honor the discount that the chatbot made up so it's very important that chatbots are reliable and do what you want them to so every AI system is going to make mistakes at least the way that llms are set up currently um and the monolithic nature of language models makes them hard to control debug and improve but we already know how to build controllable systems that and also how to improve them modularly and that is called programming so that's already something that we do so this is where dspi enters um dspi is the framework for programming not prompting your language models and what do I mean by that I'm going to go into detail here um so with dspi uh you're actually using language models to build and optimize modular programs instead of optimizing the model like I'm going to fine-tune the latest iteration of llama I'm going to build a compound system that calls llama three times in these three ways and I can optimize each of those calls uh and these LM programs are more accurate transparent and efficient than language models in isolation so instead of tweaking brittle prompts um here's a prompt that was human engineered uh for a retrieval augmented generation uh problem a multihop retrieval problem so you would ask a question that needs you to reason over multiple documents and so a human prompt engineer wrote this uh really nice prompt uh for GPT 3.5 uh you can take a moment and look at it but it's it's very complicated and it covers a lot of details about how the language model should be thinking about this problem uh and if you run this on hot poot QA uh you'll get 33% accuracy um on gbd 3.5 so uh it's it's a it's a good prompt but it takes a lot of work to build something like this um instead you can write a short language model program in Python and automatically optimize your prompts um and essentially the program will learn how to use the language model effectively so one thing that this affords you is composition via control flow um so let's take a look this is an actual uh dspi program uh so I'll go through this step by step and explain how this works but essentially uh you can see we have a forward pass so this is in inspired by pytorch um and the input is a question uh so in this multihop retrieval setting we might have a question that requires you to reason over several documents and that's the input to your program uh and we're building up a context so we actually have just normal python syntax and we have this for Loop where we say we're going to Loop over this two times we'll generate a search query we'll use some sort of backend retrieval engine so this could be a SQL database this could be a Col bear index whatever retriever whatever vector or graph retriever you have set up U and we'll retrieve documents based on that search query we'll search again based on whatever documents you got maybe it didn't fully answer the question maybe you need more detail so we'll write a second search query we'll search again and finally using all the documents we retrieved we produce a final answer so this is our our retrieval augmented generation program in dspi um and these are our two language model calls here the generate query and generate answer uh so we have really abstracted all of the language model calls into these two lines um and you also have declarative layers so what we can do is Define what these language model calls should look like uh and so in DSP we call this a module it's all very inspired by pytorch so you can Define self. generate query to be a Chain of Thought call that takes in a context a question and produces a query uh and generate answer should be a Chain of Thought uh call that takes in a context and a question and produces an answer so given this what we've actually defined here is just what the inputs and outputs to our language model should be but we haven't put any work into prompt engineering how the language model actually gets called and now given this we can automatically optimize all of our calls to the language model so really briefly I want to show a uh diagram of what how how will you like imagine this program every time you see this uh this kind of stylized llama thing think about the DSP program behind the scenes but we'll be using this stylized one going forward um but we have our two llama calls and our one retrieval in the middle um so let me talk a bit about how we optimize the the prompts uh in this program so if you look here this is one of our declarative calls to the language model um where we say we want a Chain of Thought prompt that takes in a context question and produces a query so how does this work well signatures which are what we call the uh context question to query part are compiled into prompts using a template to start um and we'll optimize from there but they they start with a template um and so the signatures Define our inputs and outputs uh and then we put them through this template uh that is a predefined template uh just to compile into a basic prompt and it'll end up looking something like this so uh given the fields context and question respond with the field query and we just say follow the following format um and give it the specific things that we want and of course because this is a Chain of Thought we'll also o ask think step by step um so that's that's how dsy will automatically write your prompts for you but that's not an optimized prompt that's a default template prompt um so I'm going to talk more about how we can further optimize this prompt um jointly along with all of the other prompts in your program so here's our starter prompt uh given the fields context and question respond with the field query um and this might have a score of say 37 so the way we compute this score is we have a bunch of input questions and we know the correct answers to them um and we compute an exact match accuracy when you run your program on this uh validation set um and and the inputs and Metric can be varied to whatever task you're working on this doesn't have to be a retrieval augmented generation setting um so using an Optimizer like Meo V2 which uh is the optimizer that Christa and I created you can compile your program and it will produce a prompt something more like this uh you will be provided with a question along with a set of documents that we've already found to help answer the question and um it gives some examples so we for the interest of space I've cut this off but it will it'll produce few shot demonstrations and plug those into the prompt as well and something like this might get a score of say 55% on your on your validation Set uh so let's look closer at an example prompt um so here we have this self. generate query prompt um and this will have uh it we'll parameterize it into a few components so first of all we want a plain text instruction that describes what you're trying to do um and so in this case we have carefully read the provided context in question your task is to formulate a concise and relevant query um so on uh and then we put in few shot demonstrations that we will bootstrap ourselves so I will talk about how we find these later um but you can see here is the context um remember the signature defined that we wanted a context and a question and write that into a query so we say here's our context uh about Twilight um the mystery series and here's our question in which year was the first vampire themed fantasy romance novel for the Twilight cycle published um and then we have reasoning so let's determine you know when this was first published so on so forth and we have our output which is the search query um when was the first uh vampire theme fantasy romance novel published which we would use to get more context and so given this and uh input this is a full demonstration of what your language model should do in this situation it should take in the context question it should produce reasoning because we said we wanted to do Chain of Thought and it should produce a search query as its output and we can just find another fuse shot demonstration and put that in the prompt as well so this is the sort of thing that dsy will produce entirely for you um and all you have to provide is that Top Line so it will find all of this in between so doing this we can actually get 55% accur accuracy on the same Benchmark with GPT 3.5 uh as opposed to 33% with the human engineered prompt um we can get 50% with a much smaller model llama 213 billion um and with an even much more incredibly smaller model uh T5 770 million parameters we can get 39% accuracy um with an additional fine-tuning step where essentially we do this bootstrapping uh process to find example outputs and we fine-tune on those um but entirely automatically dspi will optimize this for you and you can beat GPT 3.5 performance with a T5 model so this is possible thanks to new optimizers uh which tune the prompts and language model weights in arbitrary programs um they first bootstrap demonstrations uh they update the program with those demonstrations um and this is done through either automatic fuse shop prompting such as with dspi do bootstrap few shot with random search or uh finding revised instructions to plug into the program which is what dspi doco proo and dp. mepro V2 will do and so I've highlighted Meo V2 because I'm going to talk at length about how that one works um and you can also do multi-stage fine-tuning which is what we did in the T5 example um with dspi bootstrap fine-tune and that will actually update the weights of your model and you can also compose these optimizers into comp complex meta pipeline so you could imagine um and there's actually a paper that came out recently also from the DSP team that is looking at fine-tuning and then prompt optimization and then fine-tuning again or you start with prompt optimization and then you fine-tune so all of these things can be composed together to get the most impressive gains possible and so this works incredibly well in practice um dozens of tasks in production and in research uh have shown that this excels in quality or cost um so I'm going to give some two examples here um in May of 2024 the University of Toronto researchers have built and optimized some dspi programs that won the medic QA competition by nearly a 20o margin um so here's the leaderboard for that task the idea here is looking at Medical documents and finding mistakes in the documents um so they built a dpy program um and optimized I believe GPT 3.5 and GPT 4 with it and and all other systems that were designed by really good researchers working on this task were beat by almost 20% um and then University of Maryland researchers in June of 2024 ran a direct case study so they had an expert prompt engineer work on a prompt for uh I believe it was like mental health um tasks about identifying mental health issues on Reddit um and so they had a human prompt engineer work on prompts for doing this identification of mental health challenges and they then used DSP to automatically optimize the promps um and they showed that um even though the humans spent 20 hours of work optimizing their prompt um dspi was able to automatically beat them in a fraction of the time so now I'm going to dig deeper into how the latest dspi prompt Optimizer Works uh me proov V2 which I've been teasing for a little while now this is my research um and Christa's research um and it stands for multi- instruction proposer Optimizer V2 and we've done this with several collaborators uh both at Stanford um kth UC Berkeley uh and basis so I'm going to define the problem setting of a prompt Optimizer for us all and I've sort of been talking about this the whole time but it would be nice to formalize it so in our uh prompt optimization setting we have uh three things we have our inputs um which are basically training and validation data so in the case where I gave that rag pipeline that's several questions that you might want answers to um you have your language model program which you as the programmer have written yourself um and so here's the call back to earlier you saw the DSP program here I've made it this cute cartoony figure but you can imagine that's your DSP program and you have a metric um and so in the rag case we typically use exact match accuracy but this could be any metric this could be F1 score for a classification test this could be llm as a judge if you have something that's more subjective or hard to measure mathematically um it could be a loss function um so anything that you can optimize um and it gives a numeric output um and so what we want to do is take these prompts um these calls to llama and find Optimal prompts for them um and so as I gave an example earlier this is what an optimized prompt might look like and this is what we're trying to find um so we want to find optimized instructions and optimized F shot examples or demonstrations that will go into this prompt and one thing that we assume in dspi and uh is active you know research uh if you don't want to make this assumption there's more things that you can do but because a lot of the um Cutting Edge models are closed behind apis and we want to make this a tool that works for everything we actually assume no access to log probabilities or intermediate labels um for your program so we're in this kind of difficult setting where you have to treat the language models as a black box so some of the key challenges that come with this optimization problem um first of all prompt proposal because here we are rewriting the instructions um not just finding few shot demonstrations so searching over all possible strings is intractable especially when we have multiple calls to language models so we had the one that writes a search query one that generates an answer but DSP programs can uh you know be there's no limit to how long they can be I've written DSP programs with 10 different language model calls that need to be optimized I've seen people write uh 13 calls that need to be optimized there's no reason you couldn't do a 100 so searching over all possible plain text strings is completely intractable we can't do that so we need to propose a high set a set of high quality options that we can search over instead um and then we need to do a credit assignment problem because we don't know that swapping The Prompt and the first stage of the pipeline necessarily led to all of our gains if we are swapping like all 10 prompts at a time how do we determine which prompt actually led to different gains in our optimization and we can't either just swap one at a time and keep all the rest constant because that's going to be an intractably expensive optimization as well so we have to be able to swap multiple and still figure out which ones are important so Meo V2 Works in three steps first prompt proposal it bootstraps task demonstrations uh it proposes instruction candidates with a language model program itself um and then it does credit assignment uh by doing a joint optimization with beian learning U and I'm going to go in detail about what each of these steps entails so first of all bootstrapping our task demonstrations um so given our language model program our inputs and our metric we need to find optimiz or we need to find examples of our task working to use as few shot examples to plug into the prompt the way we do that is by simply running your program so you can see I take a training input I run it through your program and so I've generated one search query I append it to the context and I generate a second search query and then using those that context I generate an answer so I've gotten three outputs um here in our program the two search queries and the one answer and I run it through our metric that will check exact match accuracy on our answer um and it maybe returns that this was bad so given we know that this didn't work we'll discard these outputs and we'll try again so we'll take another training input or even the same one and try again with just um resampling from the language model and we'll run it we'll get a search query we'll get a second search query and we'll produce an answer and maybe this time it worked um so we can say that this is an example few shot candidate we make an assumption here that all of these intermediate outputs are at least partially correct because it was able to get us to the correct Final Answer um and practice because we try a lot of things and search over combinations it doesn't matter if maybe one of these search queries was bad because once we do the fuse shot optimization we'll discard some of the cases that don't work um but yes we make the assumption that all of the outputs in your intermediate steps of the program are correct that got you to a correct answer in the end and so now we have a f shot demonstration um and that's exactly what we plug in here is this um this full demonstration that shows context question question rationale and answer and we'll plug several of those into your prompts and find which ones work the best so that's what we're trying to to build here um and I think one thing I'd like to highlight is the rationale part um because I think when people think of few shot demonstrations they don't often think about bootstrapping the rationale as well um but in my mind that's actually a key contribution of dspi and a key thing that sets it apart and makes it better than other few shot demonstrations so typically if you have a data set you could produce all of the rest of these parts you could say I have I know what the gold documents are I know what my question is I know what my answer is so I'll plug in a bunch of few shot examples but in that case you don't have a rationale that works to serve as a way of telling your language model how to think about the problem and so I think it's pretty cool that dspi will find good examples of how to think about the problem and put those in your prompt as well um basically teaching your language model how to do the Chain of Thought itself the next thing we do is propose instruction candidates so the key idea here is grounding which is understanding your task um so we do a couple of things to understand your task and write better instructions remember we're finding both F shot demonstrations and the plain text instructions that parameterize this prompt so we have program aware optimizers which essentially will take your DSP code and read it and have a language model summarize it so we feed your code directly to a language model and because I think because the dspi synta TX is so similar to pytorch it understands it quite well um and so I guess it's intuitive hopefully um and so it produces this summary of your of your program um so in this example this rag example it will say that this program is designed particularly in answering questions from a given context it uses language models to generate search query so it it generates this summary of your program that it can use to understand it and potentially write better prompts uh we also make the uh optimized data aware so we'll take examples of your training data the inputs and outputs whatever you provide to dspi we we'll show that to a language model uh several rows from your training examples and ask it to write a summary so here's what it writes for Hotpot QA it says the data set contains trivia style questions from a wide range of topics like music film history and literature questions are well structured so it goes into detail about what it observes about your data uh and I think this is actually an active interesting area um that you could potentially add in even more complex analysis so we really just feed it directly to a language model right now but I think you could build a more sophisticated pipeline as well that says maybe makes hypothesis about your data and tries to write rules like this is what should work this is what shouldn't work and we can talk more later about that as a as a Next Step but um right now we are looking at it and just summarizing broadly what your data looks like um we can also we also make it demonstration aware so recall that we just bootstrapped those demonstrations um and we can plug those into the context of the language model as well well and say now given that we are we have this path that worked we generated these searches and It produced this answer you know uh what worked what didn't work and and keep that in mind when you're writing a prompt as well and finally we give it prompting tips so the things that everyone's familiar with um the the secret sauce that you try with your language models uh things like don't be afraid to be creative include a high-stake scenario provide a Persona all of these things that as a human prompt engineer you might try and just see what works um so we take all of this and we feed it to a language model we have our program summary our data summary your bootstrap demonstrations prompting tips and we say rewrite these prompts we we give it the prompts the the templated prompts that I showed earlier and say rewrite these into better prompts um and so here are some examples that the MEO V2 uh proposal generator comes up with uh so use a language model to generate a detailed and accurate answer to a given question by providing the fields context and producing the field answer um you know consider specific details of the question extract the most relevant information so it's just different ways of saying it um that are contextualized by the task itself and what you're trying to accomplish and the final step now that we have proposed instructions and F shot demonstrations we want to optimize the combination of both of these um and the key idea here is that Meo uses a beian surrogate model to do this credit assignment um so I'm not going to get into too much of the nitty G gritty details of the beijan math here but um the highle thing is that we we use something called the structured tree Parson estimator uh which has a belief over all of the different candidate variables that you're optimizing over uh and so it has a belief that a given variable is positive towards getting a good score or a belief that it is negatively contributing towards getting a good score on your metric function and so what we'll do is we'll actually just randomly sample um we we'll use a library called optuna behind the scenes which is a hyperparameter optimization library and you can really think of this as a hyperparameter search what is the right setting of all of these variables so we have a demo set of all of these different um demonstrations that we bootstrapped and all of these instructions that we found using our proposals and we'll say let's try um in let's try uh instruction one demo set three or something like that and we will say okay uh that worked well um and so we're going to up our belief that these are good um and then we'll say I'll try uh demo set three and instruction two and those got us a low score on our metric when I ran it on my validation set so now I'm going to lower my belief that those are good um and like as I said behind the scenes there's actually two beliefs I've just shown it as one bar here but you have a belief that it's good and a belief that it's bad um and that's just the way that the uh structured tpe algorithm Works um but yeah you keep sampling these and keep updating your beliefs and every few mini batches we will we'll take the ones that are performing the best and run it on the full training set and say uh how high of a score does that get um because what we're what we're doing here is running this on on many batches so if you have like 200 training examples we'll try it on just 25 just to get a certain belief that it's good quickly efficiently um and for low cost um but when we find a candidate that's especially promising we'll try it on more data to get higher confidence and ultimately uh we'll select the combination which scores the highest on your validation set and that's essentially how we're finding all of the different fuse shot demonstrations and instructions that parameterize your program so how well does an Optimizer like Meo V2 work actually um we have a benchmark called length probe uh which is essentially meant to measure this so um this is something that's an ongoing work as well um this is what we have in the current paper are these six tasks um but if anyone here is using dsy or if you will use dsy after the talk I'd love for you to talk to me about what you're building with dsy because this is something we're actively extending to more tasks um and what we really want to capture is how people use dspi either in production or in research in actual applications because we are building further benchmarks to test our new optimizers and also to show people what language models work the best with DSP um for example do I pick llama do I pick gbd4 um do I pick mixol all of these questions are going to be answered by benchmarks like this um but essentially this is what our Benchmark looks like uh right now we have several different tasks so hot pot QA Iris heart disease scone hover um and we have different programs that we've written to solve these tasks in dspi and they have SE different numbers of modules to optimize and different numbers of language model calls and they all have different metrics and we want to we want to expand this to like 30 tasks um but let's look at what these tests actually look like in case you aren't familiar with any specific ones I'll just show three quick examples so hot poot QA um this is the one I've been going through uh this talk uh talking about so uh we have a question for example when was the Houston Rockets point guard recruited by Frank Sullivan born and this is a tricky question because it doesn't say when was this person born um it kind of does a wraparound way of asking when it's like they were recruited by this um who they were recruited by Frank Sullivan and when were they born so first you can do a search and find who did you know Frank Sullivan recruit and you find out that he recruited Jeremy Lynn and now you can search uh Jeremy Lynn uh and you find that he was born August 23rd 1988 so you need two different Wikipedia articles to answer this question um and so we'll say August 23rd 1988 um and so that's one of the tasks we try to optimize and we use this multihop retrieval program another test that we try to optimize is the hover task um which is claim verif ification so this is a particularly tricky claim to verify the MVP of a game red fery umpired was elected to the Baseball Hall of Fame so you have to look over um the different MVPs of the games that red flarey umpired and figure out if those players were elected to the Baseball Hall of Fame and so if you search enough Wikipedia documents you'll find that he umpired the World Series in 1965 the MVP of that game was Sandy kufo and kufo was later elected to the Baseball Hall of Fame so this is like a really complicated problem for a language model to solve that involves a lot of different searches um and ultimately it should verify that the claim is true uh and the final task that I'll I'll demonstrate that we optimize for is this Iris task um which uh many of you will be familiar from some intro ml um classes a lot of times we use this data set I think it comes bundled in in scipi um but essentially what we're looking at is given these uh variables the SE length with pedal length and petal width what species of uh Iris are we we identifying and so the way that we actually solve this with a language model is a single call instead of making a multi-stage program all you do is give it all of these variables so you can imagine giving it a row from the table just saying SE length 5.1 SE width 3.5 and just say what species of iris is this um and we want to optimize that prompt as well and see what we can do with that um so here are our results um these are our different optimizers that we test um and this table I'll just keep extending it um but this setting here is the zero shot setting so this is where we do not put any few shot demonstrations in the program we just optimized your prompt uh and the finding here is that zero shot Meo so ignoring that bootstrap demonstration step um is performing better than not optimizing your your prompts at all so uh essentially optimizing instructions can deliver gains over the Baseline templated signatures and in particular this grounding process of looking at your program looking at your data it seems to help it make observations and actually optimize your your prompts there um the next setting we explore is demonstrations only so this is just bootstrapping the F shot examples here we are not rewriting your prompts at all um the plain text instructions stay in that templated format that dspi automatically comes up with um and instead we just look at boot running your tasks several times uh collecting those outputs and plugging those into the prompt uh and we optimize in two ways we do a random search that's what that bootstrap RS means um and we do a bean bootstrap which is essentially using that structured tree pars and estimator that we use for Meo um and what we find here is that the uh Bean is outperforming random search uh on the three tasks that we ran that one on and also that the gains from bootstrapping your demonstrations far and a Far and Away uh bypass what you're getting from just optimizing the instructions it's uh so much better to if you have to choose one just optimize your F shot demonstrations it's so significantly better um you can see those gains are much larger from optimizing just your F shot demonstrations um and an interesting thing um about this is that the demonstrations that you actually plug into the program are very important so I think the the the idea behind showing this plot here is to show the variance um what we're really doing here is we have a bunch of different fuse shot demonstrations that we can plug into your program um we bootstrapped say like 50 different ones and we can try all the combinations of these and so when what you see with these blue dots is the validation score of running with that set of f shot demonstrations and we're just randomly sampling each time here and you see this massive variance in how high quality is and it's just a few really good cases that stand out and actually do really well in your program so this is something that uh understanding more about is useful for future research and I've I've already seen some papers um starting to come out looking at like what how do you select the optimal fuse shot demonstrations um but it's really interesting to me that there isn't really a clear pattern um and some f shot examples that you could put into the context of the language model will make it perform awfully and some will make it perform incredibly well um so actually it's very important that you do this search um and the final case here is Meo um which is the the optimizer that we released to the community um and you can see that Meo is optimizing both the F shot demonstrations and the instructions and across all but one of our tasks it beats all of the baselines um and in some cases by a large margin uh for example the iris task that I showed uh is where we see some of the largest gains um but scone and Hot Pot also do very well um this heart disease task is interesting and I I can talk more about why I think it didn't do quite as well but you can even see uh that it was only five points below the random search Baseline um so the optimization with Meo was still quite good in that case um so essentially running Meo V2 is often the most effective thing you can do for your language model program and the impact of optimizing instructions rather than demonstrations is more visible in tasks that have many isolated conditional rules so that's these last few tasks what we mean by that is for example the iris task um there's a trivial solution that you can come up with that I forget exactly which variable you should look at but I think it's something like if you look at the pedal length and it's above 5 cm you know that it's definitely this type of Iris um so if you could figure out what these rules are you could write them all into the prompt and then just have the language model follow those rules um and so something that I think would be an interesting solution going forward and I mentioned this when when we were talking about the data aware optimization is actually having your language model propose hypotheses like I wonder if um a petal length of greater than 5 Centimeters means it has to be this type of flower let me write a program that that actually tests this and if the rule is true it could add that to your prompt um so things like that um that's what we mean by these conditional rules um hot pot QA conditional we add interesting conditions such as like you have to capitalize your answer or you have to put punctuation and these are things that you could put into your prompt but they're not super obvious from just few shot demonstrations about like which things should be capitalized because we'll say if it's a person it should be capitalized if it's a place don't put punctuation stuff like that that can really be described by your instruction Alone um and then for the heart disease task it's also similarly a classification task where you're trying to predict based on people's cholesterol age gender all of these um properties whether or not they have heart disease um so on these conditional tests the instruction optimization is often more significant um than it is on more simple tasks um or more well well-known tasks like Rag and nli things like that um so there are a couple key lessons um about the about what I've discussed here today and then I'm happy to open the floor up to questions as well um but some of our key lessons here so natural language programming um programs can often be more accurate controllable transparent and even more effici efficient than just single large language models um so instead of just having something like uh chat TBT that answers directly based on you give it a question and it produces an answer um instead it might be useful to hook it up to a retrieval uh engine or to make a call to another API or maybe some sort of entity extraction um and you can write a whole program to do that instead of just making it a single end to end language model um the second takeaway is that you need declarative programs not implementation details so high level optimizers can bootstrap your prompts or weights or whatever the next Paradigm deals with um and what we're trying to say here is instead of thinking about um what data do I need to collect to fine-tune llama or what hyperparameters and what learning rate do I need to figure out and set um and all of these uh High Lev details that you could be or all of these low-level details rather that you could be trying to figure out how to optimize um instead you should take your time as the human expert to think about how the problem itself should be approached and how the problem should be solved and leave the optimization to the framework because we can optimize your prompts we can optimize your weights we're working on RL optimization um and whatever the next thing that comes along we can add that to DSP too um so really the the value of the human is as the developer as the architect thinking about how to solve the problem um not necessarily as the person uh writing the code defined to in the language model um so like the rest of computer science machine learning is finally becoming truly modular you can write algorithms that directly learn to process your domains highlevel objects such as questions um uh and not these machine learning Paradigm specific parameters um so finally I'll talk a little bit about the impact of dspi um so this uh blog post from Haze Labs uh was showing that for red teaming language models um you can write a program in dspi um and you can find uh prompts that will break your language models uh 44% of the time on this you know uh safety Benchmark um whereas if you just run the uh run the prompts default uh it would respond 10% of the time so you get a 34% um direct gain or if you look at it as a proportion it's a 70% increase in success rate in red teaming language models when you optimize them with DSP um so this is a a company that is essentially doing this um sort of jailbreaking process and you can automatically do it with with dspi um we have some articles written prompt engineering is a task best left to AI models um and demand is skyrocketing for prompt Engineers one of the hottest roles in Ai and some say it won't last so essentially we can automate your your prompt engineering and um I think that is a powerful thing that you should consider doing so I'll stop on this slide for for a moment uh if you guys want to scan any of these but here are the three things that I would recommend doing if you want to learn more about dsy optimizers uh first we have a tutorial for getting started with dspi um and so that's just how do you actually write your program how do you format your data how do you write your metric to run everything with dspi and optimize with dspi if you are curious about Meo V2 our state-of-the-art best prompt and uh demonstration Optimizer uh currently in dspi we have a thread a Twitter thread and a notebook on how to get started with Meo V2 and if you are really interested in the technical details beyond what I've gone over in this talk we have a paper on archive as well um that really goes into all of the tasks um and all of the things we tried and all of the lessons that we learned um and you know all all written up in about 30 pages um for you to read there um so those are the three calls to action I have um so I guess the final question is this the path to AGI well Omar katab our our leader for dspi certainly thinks it's the path to API artificial programmable intelligence so um and I I've left this slide in as as sort of a joke for a lot of the talks but actually if you've been following the AI news today um open AI released 01 which is their new flagship model and in my opinion the key takeaway from how 01 actually works is that they are and and they don't announce everything but I think one of the key insights is the test time compute that they do where they produce a ton of different outputs find the ones that work well and then I believe that that's what they use as their RL signal so they have some sort of metric of whether this answer was correct or not they try a bunch of times probably with GPD 40 which is already you know one of the best models in the world or the best model in the world and they find reasoning traces that work well and they use RL to encourage the model to follow those traces better so a lot of these ideas that I've been talking about in dspi of bootstrapping your reasoning and then fine-tuning on that or optimizing your prompts on that I think that's what open AI is working on to build the best models in the world so I mean I put this as a joke is this the path to AGI but I mean maybe maybe that's what we're we're talking about so um yeah uh that's all I have prepared so I'm opening the floor to questions now and thank you all for your time uh you want um I have two questions um first question I have is is there a way to enforce certain structure for optimized prompt you mentioned prompt tips but for example if I want to have specific XML tags in specific order uh if that's possible and the second question I have is whether you can use llm as a judge for metrics right to if it provides numeric values an output can you use llm yeah great questions so the two questions uh the structured outputs and using an llm is a judge so for the structured outputs uh you can enforce this in dspi there's actually two things we have typed predictors in dspi uh which can at least enforce the type of output and ensure that um if you want a number you can get a number if you want a string you can get a string but for something like XML there's other ways you can enforce that in dspi so dsy has something called assertions um where you essentially write a bit of code that looks at an output um and it could be an llm based assertion but um for something like XML you could write an X you could put an an XML parser and say given this output run it through the XML parser if it compiles that's good if it doesn't then that's bad and what the assertion will actually do is um this isn't necessarily the way that it changes the prompts but it's actually the way that it changes the control flow of your dspi program so if the assertion fails then then it will explain in whatever way like you basically put a plain text uh string of what it means for your assertion to have failed and so you could say like the XML didn't compile or something like that as what happens if this assertion fails and it'll plug that back into the prompt and it'll say assertion failed on this output um try again and so um you we have two things in DSP actually assertions and suggestions um a and a suggestion is just basically a lighter assertion where it'll try a certain number of times and if it fails it'll just continue um but if your assertion fails a certain number of times it'll crash the program and so you can enforce structure through things like that um and then dsy also works with libraries such as SG Lang which can also enforce structured output Generation Um so if you use a library like that uh alongside dspi uh you can make even stronger guarantees about that and I believe your second question was about llms as a judge and that is something that we've also done so I didn't show it uh in this talk but we have other tasks where we've written llm as a judge programs um and some of the work that I'm doing this summer I use llm as a judge and optimize that as a metric so um not only is that something you can do but interestingly if you write it as a DSP program your llm judge you can optimize your llm judge to correlate with human feedback for example um so that's something that we do as well and that works um so LM as a judge is a perfectly good metric to use awesome uh question at the very end thank you for the wonderful talk I had quick question so like the thing that is being optimized is like hey the task like hotbot QA or whatever and the llm as well so you presumably would have different uh proms that would be generated for different LMS how how varied like have you seen them to be like let's say from llama Lama to 7B to 3.5 gbt 3.5 to gbd4 40 yeah yeah that's a that's a great question so there's actually a lot of variables at play here because you can change the uh you can change the the language model that's writing your prompts you can also change the language model that's running your task um and the prompt that works the best for each language model that's running the task uh might be different the prompts that it writes based on the prompting model Also may be different um and so one thing is I notice that the uh one one thing that I think is a is an indicator to me that the instruction optimization is fading away from importance but the fuse shot demonstration optimization is here to stay is that the larger the model gets the less it seems like the exact phrasing of the prompt seems to matter um the more the few shot demonstrations matter um and when we were optimizing some small models really really weird prompts would work where like we're doing hot pot QA and you plug in the as as the plain text instruction say like what is the score of this basketball player uh in this uh game or whatever um in the like 1965 Championship what what did this player score and that prompt actually works better across your whole data set because these small models like they they have weird quirks and weird behaviors um but uh what I what I would say is like I have noticed like a decent amount of variation but one thing we tested is if you optimize prompts on one model and run it on another model does that work that does work so you can theoretically optimize your prompts on a smaller model maybe not a tiny model because you'll end up with super weird prompts but you can optimize them on a smaller model and then use them on a more expensive model um and the other thing I've noticed is that uh gp4 writes decent prompts I've actually been personally impressed and I haven't benchmarked this but this is what the idea of having Lang probe and expanding it would be um but llama 70 billion llama 3.17 billion has written some really really nice prompts for me um I I I think that's actually a great model to use for prompt engineering um does that answer the question awesome fantastic job thank you thank you for the presentation um quick question as a team maybe uh what's your primary focus on is it observability or access to DSP what's the next uh V3 taking us to yeah yeah it's a great question so dsy actually has a road map on the on the GitHub repository um the cool thing about dspi is as an open source Library U that's expanding quite quickly there's a lot of people working on a lot of different things in DSP right now so I think one push is like how do we make Pi um something that like people can see as production ready you know and we're building Docker images and um different ways of productionizing your dspi programs um so that's what one pushes another pushes on this RL front um so using RL to optimize your programs um there's work um some of the stuff that I'm doing is how do we make a human in the loop in this optimization so right now everything is automatically optimized um but what I often end up doing is I look at the F shot demonstrations that it's come up with and I will rewrite some of them myself and and then rerun it to make sure I didn't make things worse but I think it would be cool to have a human in the loop as it's doing the optimization to say oh I like I like what you've done with this instruction or I should edit this and and so we we're basically building a DSP co-pilot um that will allow you to optimize alongside the language model um kind of a prompt engineering assistant um so so that's one Avenue and I think improving the documentation um and doing some stuff uh where uh the last I think cool push that we have going on is um maybe different inputs should have different prompts um that that work best so how can we build like a routing function that um behind the scenes will will figure out which prompts to actually use for this particular task and and how do we actually optimize your program itself um I think that would be cool too so lots of different initiatives with dsy and we're always looking for more contributors so feel free to to reach out and and also just submit a PR there'll be question from the audience and if you want to ask more questions you know hopefully you can con with Michael offline and we'll figure out how to relate thank you um maybe it was card I don't know I was a little late to this talk U but I'm curious about is there any way or technique to get some sort of um confidence levels um in llm response to certain questions yeah I think that's no yeah so I didn't I didn't cover that um I think that there's a lot of interesting uh work in that space it's not really something that we have built into dspi right now but I think you could design it into your your metric potentially where um I I guess one way of producing confidence could be um and something that fits nicely in the DSP framework is like generating a bunch of outputs to a given query and seeing how many of them are consistent with one another um or um I've actually seen some interesting work recently on looking at some interpretability methods for comp confidence prediction where what you would do is um write the answer uh where you do put the Chain of Thought reasoning and then have the language model produce the answer without the Chain of Thought reasoning and use uh causal attribution to determine if it actually used the reasoning to produce the answer or not um so this isn't something that that particular thing is not within the dspi framework I think there's interesting ways that we can use this notion of bootstrapping to produce confidence um and the E most simple way is to run it a bunch of times and actually make confidence intervals or something um but no I think um I think that's really cool I think that would be also an interesting thing to consider incorporating with dspi but that's um that's not necessarily covered right now good question [Applause]