Event-Driven & Multi-Agent Architectures for Gen AI Workflows | Mary Grygleski, AI By the Bay 2025
[Music] Thank you everybody for coming to my talk. I know you have other options and you chose to come here. So thank you. But if you feel this isn't the right one, feel free to move to the other. I won't be offended too. So okay. So um this is me. I'm uh Mary Gilleski and uh let me see
I'm very happy to be here. Let me go over here. That's my agenda. um just a quick intro and then um also then this is kind of like what I'm intending to do since these are like short like 30 minutes or essentially 25 because I have to leave time for question and answer. So it's just going to be primarily lecture through it. Um but I make I'll make myself available to you for future conversations as well. So I've talked about the demystifying genai, agentic architecture, event- driven world and AI and distributed computing and agentic AI uh towards more on the data engineering side and then applying event approach to rag and then to also agents and workflows. That's the plan and then uh that's about it I have time for
So who is Mary? So just really quickly picture is worth a thousand words. I'm currently the um associated with the AI collective but let me first ask how many of you are already familiar with AI collective. I'm just curious this meetup group. Okay, a few. Okay, wonderful. But I'd like to invite you too because it's actually born in SF and it's kind of amazing. Um now it's spreading all throughout the world and we're starting to have chapters. These are volunteer-run organizations although then there are uh you know a few of them who started it and they're based in San Francisco
So um uh we're expanding. So I'm I started off a year ago in Chicago as the chapter one of the chapter organizers and now I'm I don't know how I I just started asking I have friends who wants to start chapters in uh Europe and then they say come join us in the global team and then I now am I'm the VP of global for western hemisphere which is North America and Latin America as well and we also have somebody from uh Africa who's doing the Europe and uh APAC region. So just so you know and so yeah we're welcoming there are plenty of events too so I can share with you if you're interested. So other than that okay with the community thing I'm also involved with the with the Java community. I'm a Java champion and also I I run I'm based in Chicago so I run the Java users group. I've been doing it for 10 years so far and still doing it. I'm pretty much a you know I started off software engineer for almost 30 years in mo mostly Java kind of traditional into Java and all of these new things AI but I'm also involved too with a lot of community work and that that that's what I love to do too that's about me and so my um interests too are mostly kind of distributed systems that's kind of my core thing but then as it's as such it's apply can be applicable to many discipline as well so that's kind of where I'm coming from and yeah we'll be talking about AI today So and then I'll be sharing with with you my contact information afterwards too. So first of all just demystifying genai and agentic architecture
How many of you are already working with generatively AI? Let's see just Yeah. Okay. Great. Yeah. So that that's cool. So now this one too this talk I'm less talking about the consumer level so to speak. I'm not talking about okay use chat GPT and do some things and some agent um tool. they have great too to do your daily work but I'm really talking about the engineering level using you know agentic AI um distributed computing kind of principles to apply to building actually enterprise level type of applications so okay so really quickly since everybody already kind of know about it so I won't get into all the details but it's basically wanted to emphasize is that AI how do we look at this conceptually um there's the data that is actually the core of it too as we all know
So that's kind of uh without data there wouldn't be any AI, there wouldn't be any machine learning as we all know. Um and then where Gen AI sits is right in the middle over there is basically heavily depending on uh NLP natural language processing and uh LLM and deep learning in that layer. But of course you know AI is the outer kind of all-encompassing umbrella and then there's machine learning and then the deep learning inside too. Okay. So then uh without kind of uh kind of further introduction to there are many generative models to as we all know all of these are the popular ones um and that's what it is this genai is working with it's all depending on the LLMs and there are different variations of such too and then of course there are also the platforms too and wanting to mention for example like Amazon has the sage maker or Microsoft Azure all of these are cloud optimized kind of AI platforms too from a more application kind of perspective. Um, of course, Google cloud also has like vertex AI but then there are plenty of things and we are all building like um thing applications on on this level. Okay. So now then the talk is really about um agentic and why and because LLM itself as we all know is very good right it's it's excellent it can you can ask qu some questions and it can answer questions really well kind of but think of it though it's small oneshot deal if you don't build anything around it it can't do it much beyond that um so lots of limitations there are no memory of its own you can't add attributes to it it's opaque essentially and also to not only that LLM the algorithms is all non-deterministic and that's an issue when it comes to enterprise type of applications
Yeah, we can write, you know, beautiful poems and uh generate beautiful images and like a fascinating videos, whatever fake thing. But the thing is though, it it can't, you know, when you kind of ask it, well, how about I want to go, you know, I'm going to be flying from, you know, Chicago to Berlin and I need to plan all these things and then if you ask the LM to do it, there's just no way it knows unless you build something around it to make it happens. So, okay. So as such LLM is one shot usages zero shot essentially and uh the thing is though in real life kind of the way I look at it I've been working mostly at you know enterprise kind of level applications scalability like workflows those are essential too and also not only that the memory how do you me remember the state of data when it deals with data that when it deals with like distributed computing the state management is absolutely important too so so those are kind of things and what can make it better at this point in time all the researches and marketing too it's pointing to agents everybody is talking about agents and as such yesterday when Eric Mayer did the talk I really love it it's kind of the marketing has a fascinating way of talking about old stuff and make it sound so you know kind of like amazing but the thing is it has been we have been using agents for a long time if you are a Linux person Unix person it is essentially agents right all of the demons and doing things they are just essentially agents. So what it is is that agents are essentially we wanted to be focusing on some task do it autonomously and um being able to be intelligently and do some things very well that's what it is and also the adaptability aspect of it too. So essentially too we're now looking at agentic architecture how does it apply to AI this field right so kind of reiterating basically autonomous execution also too it should integrate well with tools other tools and this scalability should be seamless as well as well as resilience and recoverability those are kind of aspects too and as you can see if you also are have been kind of a serious developer let's say working with um reactive systems, right? If if some of you may have u asynchronous style, all of these two um and also AKA who is here the the sponsor, they are into reactive manifesto. All of these things are really dealing we're dealing with serious applications that can recover do all of these things, you know, that's on a large scale too. So that's what we're looking at agentic architecture to make it LLM truly useful in my opinion too needs we need to address that level of concern
Um so okay so some key concepts too about a gentic architecture. So it can be like you know architecture wise you can have vertical architecture meaning that you know basically too I think also Eric Meyers or somebody some other talk I was hosting yesterday that was so true. Um essentially one agent alone can't act. So you need many agents because they each take care of some specific task and then you need a leader to kind of do things and much like if you are working with Kubernetes and of course there are also like leaders too that needs to orchestrate the whole thing. So anyway you can have vertical architecture which means one agent will be the leading one and then or you can have horizontal architecture in which you can have multiple agents kind of collaborating. They each don't really you know don't lord over the other. It's more like they are collaborating to on that level. But the thing is though ultimately too you do need some way of some agents being a able to orchestrate to make sure that you know they can all perform
Let's say I'm I'm citing the example of uh booking of flights and it basically touches on systems on you know your your bank account and then also your schedule um and also availability of flights on the carriers and the time of day all these things. So you need somebody to really be orchestrating making sure right there's nothing kind of out of whack. How does it flow? The sequencing is very important too. So I like to also kind of compare it to like an orchestra when you have um you you need to play music and you have many instruments that are doing music you need a conductor to be there to to kind of or that's the orchestra conductor and then making sure the music comes out beautifully too. So that's kind of a same kind of concept. Okay. So then agentic architecture also essentially have to deal with these complex task execution. Uh they're sequencing all of these things and also decision making needs to be autonomous supposedly all autonomous
But I think right now there's no true AGI yet we all know. So it's basically um it's still we still it still needs some human in the loop to try to help right to kind of close all the gaps too. But then ideally it's supposed to do a lot of things for us. So let's see where it goes to and then of course the reasoning part uh the planning all these things too um and also the adaptability the continuous learning aspect too and it's basically at this point it still needs human intervention as we all know but we'll get there so the event driven world and AI let's kind of talk about a bit about it at the heart of AI is data and data itself is not stagnant too it's basically there's there's a flow of data data in many directions. So just wanted to kind of point out too essentially you we're working with data and the tools on the architecture architecture level that's event- driven architecture to kind of design your systems and then also you have to handle data in different states too. So the different states of data essentially is at rest meaning that they are could be on in storage in a database or like in some file system or like somewhere right they're being stored on on a cloud server all these things is at rest or it can be in transit meaning that data is traveling from one node to the other for example they they're doing a communication so they are there there could could be like data is in is transitory in some nodes too and then also can be data in be in use as well meaning that you are um operating on some um you know kind of processing and the data gets extracted somewhere and it needs to be transformed or manipulated as well. So these are kind of essentially the three different states of data and then also the techniques to kind of deal with flowing data that's kind of ever changing would be like event streaming and streaming is essentially the techniques or kind of essentially data that's you know we're talking about data that's being triggered by some events and then they will be flowing from one system to the other. It can be also like you're watching video we're listening to music all these are like streaming type of um operations
uh and then also there's uh event stream uh messaging as well. So I kind of like to site some example for example like Apache Kafka or Pulsar or Red Panda who's here too um solless all of these two there are also event store all of these are like kind of techniques that handle event messaging streaming and they have broker to kind of deal with for example if you using popsup right publish subscribe type of mechanism to do messaging too. Um all these things are kind of you know how you can work with data on an engineering level. And then one more kind of an interesting techniques too is called event sourcing and CQRS2. I sort of combine them kind of together but they are kind of essentially different techniques but event sourcing is a let me quickly kind of talk about it is basically um in in let's say cite an example in an order entry. So normally it's some in some traditional way you can managing an order you have order entry in your database and you have a column that says status and the status can be new or you know uh pending um waiting whatever it is and and you have your status column. However that kind of um capture of the data isn't very good be because what if there are some problems you want to be able to do a replay go back in time and looking at it. So now we're talking about using a technique called event sourcing
So in this case what we're trying to store is basically that order but the change of state of that order. So every time you create it in an order it gets entered into your database some way right it can also be Kafka for example. Kafka is essentially like a logging mechanism too. So you capture it and each time let's say your order is new and then change to pending then another record gets created is basically associated with pending and the time stamp. So as you can see every time the data changes with this order is being captured. So the nice thing about it is that if there are some problems that happen you can go back in time and say okay between you know 12:00 noon to 12:15 something happens I want to know what is going on. Then you can take that group of data that you have stored that that those events you save and do a replay and look at exactly what is happening to that order as it changes over time. So so that's event sourcing
I'm sorry. Okay. Okay. So that's event sourcing and then I just want to bring up to is CQRS is essentially it stands for command query responsibility segregation. Now that's something AKA is also doing too. You can also go talk with them. uh Tyler was here talking like the CEO. Um so CQRS is essentially the the write and the read becomes very um important too in that kind of a system because it can delay things or if they're also dealing with cap theorem which I'll be talking about in a moment too
So again just want to kind of point out to you. Okay. So now uh talking about generative AI going to that um talking about their streaming usages. So basically in a gen AI type of uh application like these days are so common chat GPT and we're doing prompts your prompts can be from a command line some program that's running on your phone for example from a browser or from your mobile apps and those are kind of think of it every time you send a request over to your you know to your LLM it's basically is streaming type of uh scenario too and then also too then behind the scenes now it receives you know your app receives that uh the the the prompt. So now we could be doing a rack too, right? Retrieval augmented generation kind of rack. It's basically augment what the LLM doesn't have. As we all know LLM gets trained with older data because it takes a long time. So then for example, JGPT when it came out two years ago, right, in uh 2022, but it gets the data that that's used to train it, it's only up until 2021
So what do you do with that lag, right, with with the time kind of in between? So you use techniques like retrieval augmented generation rack to do it. So the data you can be extracting from could be from file system from message queue from a socket as well right and all these things. So this is essentially is to supplement what the LLM didn't get trained with and then with that you you eventually send the request over to the LLM and then of course too LM process everything and comes back the generative model of the rack system will beautify your results and then get sent back and that's also gets streamed back to the requester as well. So those are kind of like a streaming usages that we can think of. So benefits of this event-driven approach is basically we're handling this dynamic nature of data and its flow patterns too and also the nature of data you know the events happen can happen anytime so they're handling the asynchronous kind of nature of these flow of data and also too in in you know in normal usages case is not a big deal but let's say we're dealing with enterprise level then we have to handle like scalability of system let's say you know you have um like a flowers.com on Valentine's day you get tons of uh requests coming in. All of these kind of scenario basically too if you use this kind of approach will be better because of you know using let's say a popsup type of messaging you can handle data um all of these orders coming in in you know many folds because you're using a broker to handle it and also all of these components that's sending the requests and receiving requests they are all independently managed right they're loosely coupled so then makes kind of the scalability aspect the flexibility very good in that case all you need to do maybe just to increase the number of brokers to handle all of the messages that comes in. Okay. So now in a genai type of scenario especially we're talking about genai type of application you're interacting with different models
So it helps to you have different agents and my agent this particular agent is handling video things. So I'm talking with Sora for example OpenAI the video uh specific LLM and the other one could be dealing with just general thing like a clawed model and so all of these they are indiv independent agent but they are all kind of co collaborating in this larger application ecosystem as well. Now the thing is though with event driven as much as it sounds good it's also not very easy to do sometimes because it's harder to debug. um think of all the data that comes in it can happen at any time and you know and all these it's pretty pretty much a bit unstructured so to speak right it can be having some messages that comes and so harder to debug harder to track so but the the thing is though is I'm not deterring us from doing that it's just that it means we need to do better work at observ observability side to uh track the messages that comes in if something goes wrong you want to be able to track it and debug all of these Okay. So, distributed computing and agentic AI some data engineering kind of information. So, CAP theorem and how many of you are kind of familiar with CAP theorem? Yeah, maybe one or two a few. Okay. Just wanted to mention now cap theorem is kind of very important aspect when it works with data in the distributed uh computing sense
Meaning that you know if you're just a single computer like within the firewall just everything together it's fine not an issue. But in today's world so complicated where having data live in different servers in a cloud environment for example then it's a distributed case. So what cap theorem is saying is that only you know and any kind of distributed systems um essentially there are three aspects right the um part well I'll go with consistency meaning that all of the read is from the most recent write um otherwise there will be an error meaning that you know the you you want the data to be accurate essentially right it's consistent you have multiple clients asking and you want it to be consistent and that that's one aspect and then the other thing is availability ility of the data. That means each you know request coming in they should get a response right is the availability aspect of it but it it doesn't care so much about accuracy at that point. Now the reason is because the third aspect of this um kind of a you know scenario is that we're dealing with partitions and these are referring to network partitions. So anytime there's distributed computing you open the cloud you rely on the network but the network is never reliable as we all know. So you could run into problems of that's what it's kind of described technical terms is partitions uh tolerance and basically network partitions you can have the connectivity goes down. So in that case you know if I'm having requests coming in what do I do with it right it can be do I sacrifice consistency do I make sure I have a response back so it's availability or having consistency do I wait until the network comes back and then I have more consistent result so that that's kind of essentially what it means the cap theorem essentially saying that only two of these three aspects are guaranteed you know that kind of thing so okay so then this is just a diagram I think you may be able to see it too it seems like popular somewhere
So the thing is too is it's not like one is right over the other. Should it be avail more more available or more consistent and the idea is that even if you sacrifice consistency let's say to answer a question or get the response back we believe that eventually the network will fix itself. So then you will reach consistency eventually. So that's eventual consistency. So now it also depends on the kind of applications you're trying to build. If you're kind of wanting to do stuff that you rely on data being integrative very important then you probably should pick like consistency because you want the data to be more uh accurate right and that kind of stuff but in in case let's say with generative AI it probably we don't care so much consistency and to some certain extent we want to always have some response back as quickly as possible. So then in that case you want to pay choose like availability. So this particular diagram is just showing you depending on your usage scenario you might want to pick if you have to select some database to do your work then you want to pick pick the kind the the one that respect certain rules better than the other so so just over here too is that you you can find this thing too say for example um uh the the consistency side is basically uh on the left side it's HB or Google big table MongoDB reddus or hypers scale these are more on the consistency side but if you want more available availability then uh Oracle, MySQL, Vertica, Esther those are bit more on the availability um side too
So and and actually even over here uh Cassandra Dynamo all these are more on availability kind of aspect. So so just just something to tell you. Okay. So then one more thing and I know I don't have as much time so just really quickly there's a parcel too. is basically an extension to cap theorem but maybe I won't get into all of these details because um you know unless you kind of work on that level too but basically uh what kind of quickly summarizing is that beyond cap theorem too basically in in a normal case right if let's say there are no problems of partition issue your network is always on then you still need to choose between latency or consistency is what it is latency refers to you know how long does it take for you request for the data and to get it back. So that's kind of the delay, right? So depend again on what is important to you. So you may want to also be aware of that aspect. Okay
So again I talked to about geni maybe we want to geni types of apps tends to be kind of moving more towards availability side of things. So so that's that and then there's also the CQRS and agentic AI. So but I won't go into all the details because I want to also quickly talk about retrieval augmented generation too. So what is rack right? Rack is kind of being talked about everywhere too. It's a hybrid framework right that integrates the two components of the rack model and one is the retrieval model the other is the generative side right so the purpose is to produce text that's contextually rich too and also uh or contextually accurate and also informationri is what it is. So this is kind of a nice diagram that kind of describes it but as you can see there's a lot of you know streaming techniques that can be used in between. Now this one I'm hiding a lot of um details too. Um basically too as we all know when we need to um read data in from a database let's say to supply you know in the rack situation or in this case to a user number one is user will submit a prompt to the rack application
So the rack gets it, but then it'll also try to maybe go to a vector database to get some additional data. And then with that, um, it will also kind of be bundling up and ha and enhance the prompt and send it over to the LLM. So the LLM now will have added context in order to gather, you know, the the request, put the request together and answer it and then send it back to the rag. Now the thing I didn't kind of um that I didn't talk about in this diagram I'm kind of simplifying is is the fact that um as we all know doing rag too right in some cases we're dealing with so much data um that you know sometimes too um it the LLM can't handle you know it's beyond the capacity so in that case it will involve techniques such as like chunking splitting up all your input data let's say I'm a law firm I have tons of cases I need to feed them into my rack app then I have to split them all up all these things and they require a lot of um special kind of techniques to handle you can't just chop up the data just like that because if you do that then it will affect the results that's coming back from the LLM so there are special techniques to that you need to be aware of if you're working with rag too so that's just a note I wanted to point out but the thing is though to kind of come back to our um agentic kind of um application all of these two there are also a lot of um scenario in which companies are doing like um agentdriven types of rack application that can help with all of these chunking for example um this kind of techniques to to improve um rag so that it helps to to reduce hallucinations too as we all know that's a big issue in generative AI type of usages okay so retrieval model I already talked about it so retrieval will go and retrieve data from some external sources and then essentially feed it in to augment the the the prompt you know for the um for for for the LLM to get better results and then the generative model side will get the results first get back from the LLM but it will also be using it to enhance the results too and making it more contextually rich. So that's kind of the two models of the rack. Okay. So I don't have too much time left now but now to applying event approach to agents and workflows. So essentially I talk a bit about agentic architecture but agents as we all know it's it's somebody who is authorized to act on behalf of another and that does some certain work well
Um and then genai agents too can be used we can build that to orchestrate the complex workflows and coordinate activities of multiple agents and process logic and evaluate answers too. And then you know in reality too we're talking about multi-agentic systems because you know we're living in a complex world only one or two agents are probably not enough. So in a lot of cases we we're kind of utilizing multi-agentic systems and wanting to also point out too then when you know it's now it gets into another level in which you know we may want to consider using some protocol because when you're talking about agents working with LLM's different resources um they can we we can build them in any way we want but then of course anthropic came out with model context protocol MCP to essentially standardize the the interface so that will be MCP in here there's also A2A a from Google to now A2A deals more with the with the um agent to agent level and then the MCP is more with the with more the resource kind of layer so they don't really kind of overlap each other they complement each other in that case so okay so don't have too much time left but I'll just skip over I'll be sharing this with you to the slide um essentially too uh this one is an example of using autogen is Microsoft like um the the open- source uh uh agent SDK to build kind of conversational style of agents. Um there are also different design patterns to deal with agentic too that I thought I share in here but I won't um spend as much time I really don't have much time but there these are the different things but like I said I'm always available so I I'll be happy to to talk more too and then there are different uh things in here and also there are a couple of multi-agentic libraries I thought I can share this autogen and crew AI um langraph from langchain and strands is from AWS and also to agent SDK from Google and if you're Java u there's also Caucus and working with lang chain 4j through CDI as well and then some code examples in here from crew AI and also the Java if you work with Java there's also uh a particular yeah okay and uh Java and um and and AI workflow there's also another example in there so challenges just wanting to kind of make sure you know we're dealing with Gen AI types of apps and just make sure these are complex systems and design all of these things and we have to ensure the data consistency to across different distributed agents uh different different network boundaries too and also that by itself too the genai itself you know has issues with hallucinations and not only that even though all the chips manufacturer all these things are doing better it's still very expensive actually to gen to do generative AI work um the data center is actually using a lot of energy just the nature of of LLM so um and also too um AI safety and responsibility that's another aspect um agents safety to uh data protecting the data is also very important prompt injection also is an issue so yeah so okay so I think that's all I have and then the slide deck can be accessed here if you are interested um the QR code and the bitly link um and I also have my contact information on the slide deck too but just wanted to share with you over here I also have actually a stream on YouTube and also Twitch although haven't been doing as consistently but I promise to be So if you want to follow me on there and also just want to say thank you again and this is how you can get a hold of me. Thank you so much and uh please let me know how you think of you know agents and using event- driven um kind of approach to do it. So thank you very much. Yeah, thank you. [Applause] Yeah
Okay. I I guess um I guess it's lunch time but if you if you have more questions or want to talk to I'm I'm still I'm here too. So anytime I'm still here the whole afternoon, so feel free to reach out. And also if you're interested in AI collective, let me know. If you want to start a chapter in your area that hasn't already got one, I'll be happy to help you start a chapter, too. So thank you. Yeah. Okay
[Music]