DevReal: Introduction to LlamaIndex Workflows, Zeyu Alex Yang
Recording: DevReal: Introduction to LlamaIndex Workflows, Zeyu Alex Yang
uh hi everyone glad to be here and uh I'm Alex Young and today I'm going to talk about the Linus workflow so let me introduce myself first uh I'm Alex Young and uh I'm uh right now I'm working for Lama index and uh I'm also a genz and uh I uh in my in my University um I became the new GS member and after that I spend few years on front end and Main on react and uh uh react system like draai which is State Management and waku which is a uh react framework right now I'm working on Lama index and uh be a for stack and product engineer and uh building uh our RX system so so let's start with a simple uh agent you can see here we uh human like inut some message and we call a LM and uh and the LM decide whether they need to call a tour or not if if so we we call the tour then give the get the uh to Output then then start this loop again again until it doesn't need more uh to call so what we uh so so imagine if we don't uh imagine we just uh start start from scratch and we don't use any any framework we we might use uh we might first Define a function we might call Rom and uh give a uh user message and uh our initial step just have a chat history and we attend our uh user message and give a simple system prompt after that uh we can just call a LM then if there's no to call then we return the message if there uh what about if there there's a tour then we have to start over how should we do that so we we might uh we might write a weal here and uh uh start and uh and push B the tool every time so you can see from this example like uh it's very pretty hard to manage the state right uh you put you have to put all the your state into the top level and keep in mind I I have this I have something the global State or the local state and also it's it's hard to parallel to a function call that right so for here it's just a single stress and also it's it's pretty hard to manage the error handling right and uh and it's it's super hard to debuging and uh also it's hard to SC for the scalability if you want to really deploy this into the right into a HTTP server so how do we uh what we going to do uh with using the Lama index workflow uh first we just uh need to declare some event so in the example we just need like core event and some uh some progress event then uh for for the each workflow we we just need uh Define the step in each step we just uh declare what what he going to do for the setups for for here the setup step we just uh initialize the context and we we call the return the call event then uh in in the agent step we we we try to call the LM and check if there's the to call we return the to call event and uh in the to call event then we can uh in the Lama index we can just set the non worker to any number that you can this can run parallel and uh and also uh we can for example we can embed the a new 4G Vector store in our our uh in our tool call and you can see here we just uh import uh from our L index and we we also have many other uh Vector stce sport then then you can just uh uh WEA to have a uh to whether to Define it as a python type animation and you can see here it's basically generates the uh workflow and you can let's check by in our Pyon uh yeah yeah you can see here uh this uh this is a uh Tri not which is the for code example I set and uh here's the uh here's the uh generated uh the node you can [Music] see uh uh yeah it it basically generates all the all the possible no basically uh it yeah and uh and uh if you want to like if you have already have a workflow if you want to deploy it we have a Lama index deployed and this uh uh sport simulus deployment and sport is uh scalability and we we have a ability to have a state management and also it's aing first then you basically you just need to write a yo file and uh it will uh you in your terminal and you can easily to deploy your workflow into the production and this uh the diagram and also in JavaScript basically I'm mainly maintain the our L is Javascript and you can uh the the benefit of the jav of the JavaScript is that you can and just write some UI into your JavaScript and enter and return to the front end this uh JavaScript and for here I just need for example here I use a react server component to uh to the our with the workflow and you can just use async Itor here and and each step you can update UI and uh and also the the B the the the one feature of the workflow is that the is for the contest cation that you can you can uh you can allow to stop and resume the workflow in each step for example you have a you have some un need more user input in the middle of the workflow and in it's not it's might not possible in the like HTP server then but we for the St of the workflow you can and then you can for example you can hear return a user input event then in the code you can just uh sterilize the event cize the workflow and return to your front end after that you uh resume that workflow and continue yeah and uh yeah you can resume the workflow here and also you can do this in our python it's kind of low level API but it's we do support that in yeah also if you uh we also have a quama k Lama C if you if you have a you can start uh our uh code just from templates for example here uh we have a example that basically user give you can upload some PDF or CSV then you can chat with AI agent then it will give some uh streaming UI and and some yeah today and we talk about the workflow and the JavaScript side and we talk about the Llama deploy and if you if you just don't want to from scratch you can have a cre mpm CR Lama we support both Python and the JavaScript template and for some lower API we have a contact station that you can custom your uh workflow how to run your workflow yeah all the code you can you can see on our R this report Tre yeah thank you question bring to you Michael I have a question so in the in the work FL you def different methods how do you decide or how does the workflow decide which method to call does it look at the event type that you Emit and then decides on the event type which which of the methods that you define is called or was there any dependency for I think for the each each step we uh let me show for example this step uh basically you define the uh you define the event as a uh top level and here we have uh some PR you defined some event and we also have a start and stop event so basically it basically it we we use type anotation here to check whether for example here we have a start event then then you turn the um call event then you will check which which which Step will accept that element event then run that step okay that's basically the logic you get the last pairer thank you for the presentation um I'm really new to this I'm just wondering what's the difference between Lang graph and llama index what's the pro and con I think I think for uh for our workflow it's basically the event base if you use use a uh like uh if you if you try long gra you you might know like it's basically the graph it's a d graph and if you use workflow it basically it's a EV based right so from like from API side you can see it's basically uh if you try like new Js there's event emit right it's very similar toer but if you use the long graph it's basically the uh it's actually the graph based so I think there some some difference in API side but I think in in the bottom layer is actually very similar actually yeah yeah um I got a question is like um is there going to be support for like Dynamic workflow so let's say uh because right now most of the workflow are predefined workflow um is there any plan on generating the um I would say Dynamic workflow by itself by AI or controlling by some sort of algo that can that can make make there's uh make sure that there's no human in the loop and and and making all the making the actual autonomous agent happen so so do you mean the the AI generat workflow yeah no um so like um I'm saying not just AI generating workflow that's based on users prompt um but more of like U AI figuring out uh what are the steps in between similar to the steps in1 build like is there a uh way interest of the promer engineering technique that can be uh engineering into the code I think workflow if is very basic level you have to you have predefine all your steps I think it's I think uh but it's possible to have so what you say I think yeah I think yeah that that possible yeah other questions yep yeah thanks for a great talk I wanted to follow up on that uh Lane graph versus Lama index cuz I think it's a big architectural decision you kind of have to make when you first get started on the a agent design and then what's your take on like when to use event based architecture versus graph based where you're actually pretty finding like the agent interaction if you have any take on like when to use what like which strategy I think like the benefit of event based like if you uh really want to integrate like this front end like you might need the event base because you might need display some progress on on the front end like so I think that's might be the the benefit event base so yeah any more questions y um so uh is it possible to say that a workflow is actually an abstraction on top of U tool calling um uh I think not really like you can you can Define whatever you want in workflow like it's basically the it's like the the frame more a framework of the like even some even system you for example I you can Define some uh shopping workflow actually you don't it's actually don't need any LM actually but okay but in a tool call you can also Define whatever you want yeah but like uh if you want for example you have a MTI agent you you have a mut agent stab you don't for here you have a uh you don't for example I I have a uh example here which use [Music] the which which is a mut agent step uh in our plus Tre which is 2m and also have some uh human interupt for here like in in some case you might need a user to have some more interaction with input that for here you cannot just say the you cannot just use torque because it's it's a it's a long you need long communication right so okay let me bring the mic to you it's hard to hear in the back so I'm kind of grappling with the same same question as these guys have been getting at um and it sounds like basically yes it's another abstraction on top of tools you have these tools API calls but the API calls you can't string together combine this is my take I'm not super familiar with this um so this would allow you to maybe hit it and then you're kind of going down like a graph restricting the next action that they can take is that so like now I ask about I have a call member call in oh I'm John all right pull up John's information now we're in the context of John tell me about my spend on this um at this area and then tells you about that but and then you could and then it might say would you like to know about these three things whereas the other one it wouldn't be able to prompt you and say would you like to know about these three things is that is that yeah am I am I don't really answer I don't know the question is I'm sorry uh uh I'm guessing you're talking about like why I not just use tools Ely yeah yeah is this an abstraction On Tools uh it's not really like uh if you have a like MTI multi level agent or like if you want to customize your custom more step instead of a instead of TOS like you don't some sometime you don't need a uh just for example this very long step in the P if you just put in the tool it might take uh very long and you have Define your like for example debugging like uh parallel processing for for that so if you use workflow you already have some predefined uh Logic for parallel and buing and some for example the UI that yeah one more yes um in this session is like about new forj and also like L index like pull together right but I see like a L of like overlapping or um what is like the use case that can have like you on top of new for I think like for L we are data framework like we already support many like we already have many integration not only for uh like graph graph store like we also for many other like vectors simple waxer store and uh like for the graph store it's already have it has good performance right so we can uh easily integrate with workflow and also like you can custom customize your customize your logic uh using a graph graph based database with the agent right thank you I have two questions uh so if I have multiple methods that take the same type of event are they all called what so if I have multiple steps that take the same type of event and admid an event are all these methods called with the same event or only one of them uh I think the think the same event right I'm not so for instance imagine I have an llm call event here yeah right and I have two steps that take an llm call event as a parameter are they both called both the steps or is only one of the two steps called uh I think it's multiple step okay yeah of course like in the in the in the to call handling to call is basically give the B you put the the context into that event then in the Handler you you extract the for example the ID inputs from the event then you return the result event yeah but what I meant is imagine you have two steps that take a tool call event as a parameter yeah we you can put like into into the context or okay yeah or backass some yeah my other question was um could you actually have the llm emit events directly so that basically the llm emits any number of one out of 10 different events and you can just return whatever event the llm generates and then the llm gets more so you have to do less manual coding but you have the llm actually admit the that you want to return from your method for instance uh have you tried this and and something like this not none really I haven't tried that so okay was just curious okay thank you all right I think uh that's probably a good time to thank Alex and everybody for has a question [Applause]