Shreya Rajpal: Practical Guardrails for your AI applications

Shreya Rajpal: Practical Guardrails for your AI applications

Recording: Shreya Rajpal: Practical Guardrails for your AI applications

Hey everyone, thank you so much for coming, especially at 8:00 p.m. Uh I hope I'm not keeping you all from your dinner. Um all right, so at this talk I'm going to be talking about practical guardrails for AI applications. Uh and in real-world AI applications, how to ensure safety, reliability, and robustness. Uh I'm uh the founder of Guardrails AI. My name is Shreya Rajpal. Um and I'll talk about how Guardrails AI can help you on this mission. Um awesome

So, why do we need guardrails? I'm sure everybody here already knows this, but I'm just going to give a quick refresher um in case anybody anyone doesn't. Uh LLMs are absolutely awesome, but they are brittle and hard to control. Um common issues that we all see as we are building with them in practice is that the LLM works while prototyping, but is flaky especially when running in production. Um getting correct outputs from LLMs always is hard. Um and common incorrectness that we see in practice are hallucinations, falsehoods, lack of correct structure, etc. Um furthermore, the only tool that is available to you as a developer is the prompt. Um so, what this actually ends up meaning is that whenever correctness is a critical requirement of the application that you're building, uh LLMs are currently severely restricted. Uh so, why is controlling LLMs hard in the first place? So, there's three different ways that as a developer you can control an LLM output

Uh the first one is through a prompt. Um and this is not uh you know, this is not an ideal way to go about it because LLMs are stochastic. Uh and in practice, what this means is that even with the same input, so with the same prompt, uh you will see different outputs, you know, just running it again and again. Uh prompts don't offer guarantees. So, what you'll see people doing in practice is like adding a lot of exclamations, adding like instructions in all caps, do not print, you know, very bad things will happen if you uh generate this output and the LLM like will listen to you sometimes but also does not many times. Um the second option is controlling LLMs by tweaking the model itself. Uh so, there's a few issues here. The first is that it's expensive and time-consuming to either train or fine-tune a model on custom data

Uh and if you're not training or fine-tuning a model, then LLMs are often hidden behind APIs. And as a developer, you don't really have access to, you know, model upgrades, etc. Even without the model version changing, the LLM uh under the hood might update and, you know, all of your prompts, etc. will end up breaking. Uh our philosophy at Guardrails AI is that combining LLMs with output verification is the way to go. Um and what this means is like application-specific checks and verification programs that ensure that the LLM output is always correct. Um and that's a mouthful and over the course of this talk, I'm going to go into what that really means. Cool

So, this is the flow that, you know, most of you if you're developers are are used to, which is that you have some application logic and within that application logic, uh you have, you know, a piece of the pie, which is the LLM. Uh and how that operates is that you end up getting a prompt as input uh and you send that prompt over to the LLM. Uh the LLM generates a raw output and you send it back into your application logic. Um the Guardrails view of the world is a little bit different wherein like you obviously have the LLM flow wherein, you know, prompt, LLM API, raw output. Uh what Guardrails does differently is that the raw output is first passed through to this verification suite. Uh so, this verification suite, you know, contains a bunch of uh independent programs that verify whether that output is correct. Um so there's a bunch of these uh examples here. I hope, you know, people at the back can see what those examples read, uh but essentially making sure that there's no PII or personally identifying information, making sure that there's no profanity in the generated output

Um if you are building a commercial application, making sure that there's no mention of competitors. Uh if you're generating code, then making sure that the code is executable. Uh if you're doing something like summarization, then the summaries are similar, etc. So these checks can be anything, and they're often, you know, application or uh domain specific, depending on what you're building. Um so the raw output of the LLM passes through to this verification suite. Uh each test in the suite either passes or fails. If everything passes validation, you're you're hunky-dory, you go on uh onwards to the application logic. If validation fails, there's a way of like um handling the failure so that you can re-prompt the LLM, give it context about why it's incorrect, and then ask it to correct itself

And that is one of the wonderful properties of LLMs, which is that they can self-heal if you tell them why they're wrong. Uh this is obviously if you have enough budget and if you, you know, you know, if you have enough tokens or enough latency to support this. Um so what Guardrails AI does, uh Guardrails AI is a fully open-source library that offers uh a framework for creating custom validators. Uh for example, we looked at five validators before, but you can create like any number of custom validators. It is the orchestration system of prompting, verification, and re-prompting the LLM, and managing that whole loop. Um it is a library of many commonly used validators across a bunch of use cases, etc. Um and it is a specification language for generating structured LLM outputs and communicating your uh correctness criteria to the LLM. Uh so, I'm going to showcase this uh with an example of getting correct SQL

Uh so, the problem here is that you are building an application that takes natural language questions over your data as input, and you want to generate SQL queries. Um and the database here, just uh so that you follow along with the example, is department management. So, it's going to contain, you know, like, department counts, number of employees, etc. So, those types of records. Um so, we're back to our diagram from before, but uh for people in the back, what's changed is instead of like all of the suite of tests, we essentially have our application-specific tests, which are that the SQL is executable for your database. Um there are no private tables that are referred to in your generated um query, and there are no risky predicates. So, even if a user asks you to delete a table or something, uh you're not, you know, you're not generating and running a drop uh drop table query. Uh awesome

So, I'll walk through the workflow of what a Guardrails uh text-to-SQL application looks like. So, which department has the most number of employees? Um the prompt that is constructed, you know, essentially contains uh information about the schema um for the database, um as well as, you know, question about like the the actual question that the user sends. Given this prompt, there's some raw output that is generated, and this is a very contrived example, but let's say there's a failure where there's a reference to a table that doesn't actually exist in the data. Uh so, you end up getting uh the table name departments, uh whereas like departments is not a table in your in your DB. So, when you run this generated LLM output through the verification suite, um these two tests of no private tables, no risky predicates pass, uh but your SQL is not executable because that table doesn't exist. So, how Guardrails works is a re-ask prompt is automatically generated for you with all of the relevant context about why that predicate is in a why that query is incorrect. So, for example, this is obviously a simplification for the purposes of the demo, but an error table departments not found. Uh please create new SQL query that corrects this error

More information about the DB schema, etc. is all automatically created. We send this re-ask prompt back to the LLM and then end up getting a new raw output here. And this is what the output looks like. Um again, for the purpose of the example, departments is automatically um corrected to department. And then this time around the verification logic passes and so we uh continue and send this uh send this output uh you know, back to our application system. Uh awesome. So, that was the Guardrails overall workflow

Uh here's like a very very high-level overview of examples of some of the validators that uh exist in the package or that you can create with the system yourself. So, very very um um you know, fundamental things like JSON structure and type checking exist. Um making sure that, you know, your URLs that are generating are valid, are relevant to the text that you're generating. Um if you're generating strings, then there's no profanity in the text. Uh if you're translating text, then the translation is high quality. Um summarization is faithful to the original text. So, all of the salient points that you care about in the original text are captured in the summary. Uh let's say you're building a chatbot for a healthcare company, then there's no healthcare advice within the generative text

So, you're not being prescriptive, which is um against the law, as far as I understand. Um a customer service bot doesn't ask private questions. Generated code is bug-free, uh etc. Um so, in the demo, I basically talked about, you know, re-asking as a way to handle validation failures, but Guardrails supports, you know, a bunch of other configurable options. So, if any any any Guardrail specifically fails, you can obviously re-ask and all of the re-asking orchestration and prompt construction is automatically handled for you, but you can also programmatically fix it if it is, you know, possible to programmatically fix queries, filter offending values, filter, you know, specific text that are, let's say, have profanity, etc., refrain from answering, maybe your response is so bad that you don't want to handle it. No up, in this case, no action will be taken, but, you know, logs about why validation failed will automatically be stored for you and you can do whatever you please with the logs, and then finally raise an exception. So, I have a quick demo. I am very close to being out of time, but maybe I'll just quickly showcase like what this looks like in practice

So, here is the text-to-SQL endpoint for Guardrails. This is basically configured with this like configuration like markup. Essentially, what you can do here is talk about which validators you care about connecting. And then, what you can also do here is like hook this up with a database that contains like previous examples. So, this will automatically, you know, figure out the relevant examples for you and inject it into your prompt so that, you know, the so that your LLM is able to more correctly generate responses. It will do verification for you and it'll do re-asking, etc. So, I tried this earlier and my internet is kind of slow here, so this may or may not run, but this is an example of calling it for the example of like, "What is the name of the department with the highest number of employees?" And I think my internet is kind of broken, so this won't run. So, but I'm going to give it a shot

But, what I really want to demonstrate while that is running is what happens when we have an incorrect output. So, with the Okay, so it ran and you know, this is the query we ended up getting. But, in case there's an incorrect output, you can pass this incorrect output you know, with correction for validation and this is what it kind of looks like. Um whoops, I think I did not run this. Um this is what it looks like, but under the hood, this is what it's doing where this is the raw LLM output that we end up getting. Um select name from departments. After validation, we see that this is why it's incorrect along with all of the context of why it is incorrect. Um a new prompt, this is automatically constructed under the hood um without you having to do anything

So, this new prompt is automatically constructed with all relevant information. Uh we end up getting a new LLM response back. Um and then the new LLM response is validated, no validation failures are found, and so this output is returned to the user um directly. Um so, once again, what Guardrails offers: framework for creating validators, orchestration of prompting, verification, re-prompting, library of many commonly used validators, and a specification language for communicating with the LLMs. Awesome. To follow along, you can um I hope you can see this, but you can check out the GitHub project on ShreyaR/guardrails. Uh the documentation is at getguardrails.ai, or you can follow me on Twitter at ShreyaR uh or guardrails_ai where I talk about this constantly. Thank you so much.