Trusted Agentic AI:Architecting Systems with Reliable Data at Scale| Steve Anderson, AI By the Bay25
So, I'm going to start out with something I recently read. Uh, I read an article about a health care organization named Northwell and it's based in New York City and they were able to create a AI system that scans images of your abdomen, your stomach. The the the important part of it was the scale at which they were able to do that. they was able to go through 10,000 images per week in order to uh detect uh cancer early. So with the with patients consent even if they were going to see something else with their consent they were able to scan 10,000 images to detect uh and look for large lesions and and um and masses. So as a result of this AI system, they was able to have positive patient outcomes. So they was able to increase the survival rate of cancer due to uh this AI system. And that's what I think when I think of trusted AI or reliable AI, something that have a that creates value for people and and and that's that's what I think of it
Uh, hey, my name is Steve Anderson and I'm going to give you a little about me. I'm the founder of Naravez is a aentic data visualization platform and we'll talk about that later. I also teach data science and AI at UCLA extension. I have a new course right now called data store and data management to bring context to AI. Uh, so check that out as well. Um, my prior experience is at JP Morgan Chase, GAP headquarters, Charles Schwab, UCSF, the NBA team, the Memphis Grizzlies, and a large database technology company. So, I worked in most industries, uh, and been doing this for many years. Uh at UCSSF, I did a lot of uh research using machine learning and statistical models to help influence healthc care public policies here in California
And there's an agency that I helped out in California and that a agency was able to save the state of California um um um nearly a trillion dollars and make people healthier. So with that uh just another example of you know trust in AI using AI to create value for people and and that's what I that's what I personally think think it is and some of a lot of that work is in peerreview publications so check that out. Um so what is what is trusted agentic AI and we're just going to start with aentic AI. Aentic AI is differs from traditional AI in a sense that traditional AI predicts whether this customer may uh uh uh buy our product or not or whether this patient has this condition or not. So it predicts aic does things autonomously. So it can do a task with no intervention which means trust is even more essential and more important and and we as humans has to be be cognitive of that as we create solution. So we're going to go through today we're gonna go through example a use case um that that goes through and hit on the principles of what uh trust trusted AI is and I and and we'll also check out the platform that I developed that and and go through how how those principles are are are um are within in the platform so we can hit that those concepts. So these are some of the key questions that I think about before implementing AI use case like what problem is being solved
Uh is it materially practical? What what what outcome I'm trying to achieve like meaningful outcome not just uh a small outcome. So in the Northwell example, the outcome was positive patient outcome, survival rate of cancer in increasing um and and so forth. Should it even be implemented? How does it affect people? If it doesn't affect people that uh positively like to me why why why do it even if it's a B2B solution, you should find you should really think deeply on on how does this affect people? like is it going to increase revenue so I can I can keep my pay my employees more or whatever the case may be. It has to have some type of impact on people. Uh and then last get feedback from the beginning the ideal stage all the way through the life cycle of the AI use case. So, so we're gonna talk about some of the things to consider when uh when you make to make it trusted. Aentic AI trusted from quality data to data governance. We're going to talk about transparency, human oversight, impact, and we all know it all starts with good data and and and it takes pre preparation to have accurate predictions and accurate content and accur accurate tasks done right
It takes preparation. It takes good data and because the data is what keep uh uh uh bring the context. So if you have a if you have a good model and and you have um good data, you got an amazing uh AI system. If you have a okay, if you have an amazing the most sophisticated model out there and and your data is not good quality, then you got a really bad AI platform. Same for defining a problem. like when you define the problem um it just and you bring a solution to that problem you're more prone to getting to the outcome that you're trying to get to. So you have a very good solution. So, we're gonna we're gonna we're gonna get to the technical diagram and talk about the technical aspect of it and then we're going to talk about the people aspect of it because the people aspect are just as important as the technical aspect especially today because uh we as I guess technology people are we're really putting a lot of products out there and and AI products and we really need to think about the technical and and the people aspect of it
So, the the use case is this platform I I built not narrative is it's a aentic data visualization platform and and with that platform um it basically take data from desperate separate many system and create a dashboard within seconds for the user and it created in a way where it thinks about their role and what they're trying to accomplish so that they get actionable insight that they can do something actionable. So, um I'm I'm I'm going to do a just a 30 probably a 30 second demo of it or one minute demo of it so that when we go through the the technical diagram to talk about uh trusted AI that you uh have some sense of what I'm talking about. So, so with this platform you can just connect to the data and the data can be it can be a flat file like a Excel spreadsheet. It can be snow uh it can be snowflake it can be data bricks what whatever whatever you have. So I'mma pick this. I used to work in the NBA and I got this sample season ticket holder data. I used to love working in basketball association by the way. So it'll connect to the data
It it knows my role. It knows um uh it knows that um that I I work in marketing blah blah blah blah. And I just say generate story and it goes and and generate a story here. Uh, and with this story, I can customize it however I want want to. I can pick additional additional metrics to add to the story. And um, and and many other things. I'm not going to go through everything, but with that, right off the bat, I know that on average, my season ticket holders attend 30 games. I know that they spend roughly $154 per ticket and a merchandise spin is 200 and something
Um, you can you can look at it at it by customer segment. You can look at it by whether they may renew their season tickets or not. So, so you can treat these differently so that you can increase your renewal rate blah blah blah blah and you can talk with this agentic data analyst uh through wherever you wherever you work in the most. So if you work in Slack the most, you can go to that agent and analyze the data there. Um so so and it give you a prompts that you can use or you can or you can don't use these prompts, however the case may be. and we'll we'll talk about the technologies behind it as well. So, so that that's the platform. Go to the next slide
So to to kind of start very fundamental of of what it's doing, it's it's a AI, but I'm going to start with generative AI. The user have a prompt, but as you saw, the user didn't have to prompt anything. It already knew their role. They already had context or their goals and created the story. And then it goes to LM and then do a output to the user. Then build upon that you can add metadata to it for context like hey the role of the person guard rails limitations and all type of metadata that you want to go and send that to the L&M and get a output to the user. The next is a retrieval mechanism to take in external data that you may have PDF uh customer contracts whatever the case may be and specifically for that platform I created a context text workflow and and that's what what I call it uh and it has metadata so hey I was a marketer role in the example that I gave you when I went through the demo. Um, it knows my goals
It knows the audience. So, when I pull something, who who am I pulling it for? I've mostly pulling for the marketing leadership team. Um, and then limitation of guard rails and and of course, we've seen that in the media lately. Um, we saw example of uh the gu guard rails not being um in place where people created like Martin Luther King disrespectful videos and things like that. Um and um and um they had one where he was in like a WWE wrestling m match with uh Michael X. Um and and that that really went viral viral. So we really got to think about, you know, what guard rails need to go. And now those guardrails are in place because you can't go and do that anymore
So So with this context workflow, you really have to think about in in in our platform, we think about, hey, what are those guard rails so that we can put that in with the with the metadata. Then we have a process to go out and learn about that industry and that's a separate process that goes out and it only take best practices and research from highly credible entities. Um and then there's another thing that we we as in uh me and the people that uh who's now helping me uh I I have several years or several decades of data experience. So I wanted to put my experience in it. I don't want to just I don't want AI uh just taking its experience. What what do I know about the data field that needs to be put into the platform? So so so I have that as well. Then an adaptive visualization selector. So it goes through and process the the what the user need or asking and say hey is this a data visualization or is this an analysis or how how to show it is a bar chart a pie chart um do they need actionable recommendations and it's parsing that out so that it gives the most relevant uh and helpful thing to that user and then there's a retrieval mechanism and then how that fits into the picture
If we're we're looking at this, you have the user on the left and then in the middle you got the user query and it goes to through that context workflow. Uh and then that context workflow go through all those things that we just mentioned, all those different processes. Um and then if there's external data that that's needed, uh it get it get it from a a vector uh database and retrieve that and then and then that's packaged up metadata and goes to the LLM and then as you but as you see the the user data from here to here so with that user data you see that user data never go to the LLA. them and that that keeps their data uh secure, it keeps their data private. Um it it doesn't train the model, it doesn't touch the model and so forth. What what you send to the user data is how to process that and you sit in where the user data is at and then you send a response back to the user. And when you send that response back to the user, the user look at it and say, "Hey, was this helpful or not?" And they can say, "Hey, yay or nay." So there's a feedback part of it. There's all there's also a monitoring dashboard so that there's a human in a loop and there's accuracy um there's accuracy metrics and health metrics and trust metrics and and so forth
Um, and that's that's pretty much with that diagram that that's what it's doing in the in the background as it create data visualization for people and and businesses. So some of the traits is hey minimum hallucination because if you want to build trust you want minimal hallucination. We read the platform error out before it gives you something that's that uh that's not true or that's not accurate. Most of what it gives you is from your your data. So not from the L&M. Use the LM for human language and use your data for the knowledge. So if your do your data is not good um then your your output is not going to be good. data data traits is data privacy
Uh data is not is not used for training. It's it's not it doesn't touch the LLM. There's a human in a loop. There's a monitoring dashboard for health and it's outcome focused. Like I said, it has to create value and it has to create value for people and it's secured and encrypted. So the this is not all the technologies of course Python Python you can do web programming and then you can do some analytical programming superbase it's deployed on Google cloud lang chain and um and that's where we doing all the um agent orchestration and all of that and then duck DB is some of the embedded data processing and then we're using uh we actually using two LM. Uh as for a a few tips, I I always spend a lot of time on the the problem statement. Um the the the AI piece is is is pretty easy
like you can build an MVP pretty quickly using using AI to create AI. Uh but um but the problem statement is really important and clearly defining that and hopefully this is going to be put out. I got I got some slides in appendix that you can look at on some uh methods of creating a problem statement. Create your goals and come up with outcomes and KPIs on how you're going to measure this is important. the sixp model for AI readiness. I don't know if that's actually a model. I made that up, but I'll show you I'll show you what it what what I go through in that 6P model that I made up. And the documentation documentation is really important because you're going to always be optimizing your AI system
Um, and what I find is is that people and organization don't take the time to to to uh document every change that they're doing. Of course, it's it's in your Jira system is in all these system, but uh do you have something that you can give someone and it and it say everything that your AI system is doing. And I think that is really important. uh sensitivity analysis and and that keep accuracy uh high. You definitely want to do that. And then as for the sixp model for readiness, I answer all these questions before I start in in high detail. One I'm going to break down in a lot of detail, but but overall why do this? Why should this uh be done? Uh who's involved? How would it work? How to measure success? Are we doing it responsibly? And how how do it scale? So there's purpose, people, process, performance, and principle portfolio. And the scale piece is from multiple dimensions
It's from a technical dimension, people and and a and if you're in a corporate world, corporate strategy. Um, so from a a AI perspective, how how do this AI use case fit within the overall AI strategy? Is this going to help me get to the goals that my strategy is trying to get to? Am I going to hit the KPIs within that? And then even greater than that, am I going to hit the KPIs on on the overall corporate level? and and you have to be thinking about that all the way down to a a little small use case that you want to do. And from a from a technology standpoint, uh you have to think about, hey, if I do this, does it fit into my tech stack? um am I'm doing this with tools that's open and connected and with massive parallel processing and all the things that uh make that that really help you scale out a solution uh because you got to be thinking about hey what what is the next solution I'm going to do uh so so really taking the time to think about that scale is highly important Oh, and to to kind of sum this up, uh having good data is uh highly important and example I gave you, you see we don't even send the data to the LLM. So your answer is coming from your data. So So that's really important. Defining a problem clearly is essential. Um it has to create value for people and a human has to stay in a loop uh uh in a reliable and trust trustworthy AI solution. And to sum this up uh yeah if you you want to check out the platform there's the URL um right now I have the platform where you can sign up for free
So, go out there and check it out. Let me know what you think. Um, and also I'm on LinkedIn, so uh feel free to follow me there. And other than that, if there's, uh, two or three questions, I I can I can answer those.