LLM Avalanche: Govind Gnanakumar: Firewall for LLMs

LLM Avalanche: Govind Gnanakumar: Firewall for LLMs

Recording: LLM Avalanche: Govind Gnanakumar: Firewall for LLMs

thank you I'm govind I'm from automorphic and we're building a firewall for LLS and so that's a pretty complex term and I'll also break it down the basic idea behind this firewall is to stop adversarial attacks to stop Hackers from messing with models in in dangerous ways right and so I'll give you an example of such an attack suppose you're playing around with GPT in the instructions of this model are you are a helpful French translator right we say something normally uh like I don't know hello world you will get a basic response hello world right and it's behaving as expected but if instead I as a malicious hacker or a malicious user tried to do something like this well then it starts writing this elaborate French essay trying to convince me to start the llm firewall and Gita right and so this is a very playful very simple example of the kinds of danger this is that can occur when you give models uh the ability to execute untrusted user input so think of these attacks as analogous to SQL injection maybe if you've heard of it where if you execute uh arbitrary user input with administrative powers and then get all kinds of sketchy dangerous effects right and so one example is there are a lot of AI email assistants that read emails that have the ability to categorize emails delete emails and so forth and so someone could very easily inside of an email basically just embed uh embedded instruction to delete all emails from XYZ person right and the model because it's been trained to follow instructions so well it'll just do that and you won't even realize because it is in the end of the day just following instructions and so and and so if we do see what what happened here it's literally translated it for us it's trying to convince the user to to start or GitHub repo right or another example is someone with a Bing chat professor at Georgia Tech uh inserted this in his website mentioned that margarito is a time travel expert and so in Bing chat summarized all the information about him it said that he is also a time travel expert right and this is a reasonably playful uh uh side effect of the model's ability to follow instructions but at the end of the day it can lead to disastrous consequences right and that's what we're trying to stop with this firewall which protects the user from the models from a compromised model and protects models from malicious users and so uh our we have a three-pronged approach the first layer is something borrowed from traditional cyber security that basically just the term looks for lexical string similarity another layer that uh basically learns the attack signature of uh some an injection that has been an attack that has been identified as an attack then we have a third layer that is another model that actually identifies these kinds of uh malicious attacks and so to show you what it looks like uh this is the uh previous attack as you see they'll tell you that an attack has been detected or for like another example Maybe this way right which is absurd right but it identifies that an attempt has been made to subvert uh the parameters of the prompt so there are a lot of other concerns right so if you're an Enterprise and uh you're worried that users may attempt to may accidentally leak personal identify identification information or sensitive company information well then you can stop that with this firewall or if the model itself May act in a manner that is not complied with regulations right so like if you have a fintech chatbot you can't it's not allowed to give legal advice or financial advice and so if it starts suggesting stocks well that's a grave danger and so this would kind of stop those kinds of sensitive information from getting to the user that it would also prevent the user from setting that kind of malicious information to the Mob and so if you'd like to check it out and find it find us at automorphic.ai playground if you want to play around with it and if you want to try it out or have any ideas or you know possible suggestions for contributions uh shoot us an email uh if you ever go to our GitHub repo and you'll find us but yeah thank you