LLM Avalanche: David Talby · Delivering Safe and Effective LLM Applications Using the Open-Source...
thank you so what I'd like to talk to you about is responsible Ai and an open source library that you can use today to evaluate llms compare them and make sure you get to production safely responsibly AI today is is not optional it's not nice to have it here on top most of my work is in healthcare generally in healthcare and life science it is illegal to go to production like with one patient to One customer before we actually proved in writing and validated that you're not going to help someone but even in other area we have cases this is now people are being sued as we you know as we speak whether you are in e-commerce you cannot offer different prices to white people and black people the same same thing for recruiting consumer protection a banking credit decisions you need to be able to prove that you are doing things correctly that you are not violating laws that are already existing to protect they protect the populations so in order to you know to tell what everything is okay basically every every company every Lab company came out with this beautiful graphic that says you know it's ethical and it's human-centric and it's responsible and etc etc and very quickly we had more of these and then we had more of this and really there's no large company or consulting company or government organization does not have one of these like beautiful things but the problem is what do we do when we actually go and implement the system and if you read just about every paper that comes out on the topics of robustness bias fairness toxicity a truthfulness data leakage what you find is basically and these are this is by the way the same thing I'm seeing right now kind of in practice in production the situation is really really bad you take a prompt you change one character it breaks you test it males versus female white people versus black people you look at data leakage you try to find protect the information you find it the the one thing that's common to many of the papers of this year all of them say you know here's what we found and in the discussion section this is really not surprising because we are the first ones to ever check okay and what we need we need to go really beyond that step now so here are some things that we have learned that you need to pay attention to in deploy system so things that you you need to do first of all you need to actually test your models whether the kind of classic ml models or llms so so really right now especially with llms we are kind of where we were with software engineering in the 80s you know I call the thing it's autoflex on the machine okay push it right and then we landed there's this whole theory and practice of software testing that goes to many many layers and has many many tools so that we actually know that when you say something works it actually works and now we're at the point where we need to do this and if we don't do this really there's no surprise that the situation as it is today the other thing we need to do which also is obvious once you write it you have to test Beyond accuracy okay if you want the system to be robust right to be able to deal with you know typos right or the fact you change the order of words if you wanted to deal with bias if you wanted to not click protected information if you wanted to not be toxic you actually have to test for this stuff right and just like we have you know unit testing with every version before it goes to production you need to know that you will save the two important things that you've landed you cannot do because we actually deploy this for real customers and get feedback from them the one thing you should really never do is is pick a model that someone published with the paper some published a checkpoint kind of said this is my model it has the best F1 score it's really great getting Bendix I'm just going to go with it okay basically we find these models they usually they perform okay on the F1 score not always generalization is something to usually test but in general the current standard on how they perform on any other requirement and any other metrics you look at would be far below what you normally expect so that's one thing I can tell you just never do this the second thing you do not want to do you do not want to rely on standard benchmarks okay so if you know for example Stanford they have a great portrait called the helm holistic evaluation of language models there are lots of benchmarks for toxicity for truthfulness and there are two problems with it one problem is what what a lot of companies and systems do today they do what's called classic over fitting we just they train the llm on those tests okay and then what you get of course you perform Beauty if you're on the test yes you you know you test on your training data that's fantastic and really then what you say like really change one world in the prompt or one word in the data and then the thing completely fails and the other thing that you get is really General robustness issues okay and really I have an unproven hypothesis that a lot of the gains that we see now with very large language models compared to the same models you know 500 billion parameters also 50 billion parameters is ill demolition just memorize more of the web okay and if you give it a medical test or a legal test it's just very likely they actually have the exact copy of the same test and they can just give you the answer which is not what you want to see when you you know when you want to have a generalized model so to deal with some of the issues one of the things behind the John celebs we've built the NLP test Library which is and this is completely open source it's Apache 2.0 license use it abuse it take it do whatever you want with it it's open for commercial use the goal is to have something that's very simple so you'll see in a few lines of code you can tell it please generate tests for me and right now those I think about 660 plus some types of tests that you can generate automatically you can also do things manually if you want to testing also is something that you don't need to be a data scientist to do right you really want to be really an end user or kind of a domain expert to be able to come and say Here's how I test something just like classic software testing we want it to be comprehensive so actually test in one place everything that you want to test before ready to go to production and of course make it open source so the NLP test Library does three things for you so instead of the usual the current path which is you train your model accuracy looks good F1 score seems to be stable enough push it to production if we want two or three steps we want to generate some tests then run to a run the test and get a pass Trail score right it is something we need embedded you know release or envelopes Pipeline and in some cases we want it it can also do some data augmentation okay so that we can automatically add more training data and improve the model so let's see how this looks like in practice so um these three lines of course show you the really the whole thing end to end a form NLP test we import the library then we create a test harness which is an object and you basically tell it look I want to test this model in this case we're testing the most likely it was model nagging phase over a built-based case a form form hanging face there's a you can also specify the test that you want to use but there's a default Test Section for each tasks so whether you're doing an ER or you're doing classification doing question answering you're doing summarization either kind of standard tests sets that you start from and then you use the harness so you tell it to generate tests okay which means start with the test that they gave you but it started it's a it's a kernel generate other tests from it and we'll see what kind of test you can generate once you generate the test the test two just available as a python data frame you can export them as a CSV you can add in Excel you can load them you can do whatever you want then you run the tests okay meaning actually run influence and see if the test each test passes or fails and then just generate the report how am I doing for each kind of test and in the end based on the config file provided did I pass or did I fail in terms of what you can test kind of the hubs you can use so this is what's available right now out of the box so you can test any John Philip's model any hacking face model any open AI model anything that works with long chain cohere open Assistant AI 21 open AI of course all the dilemma and lamb variant models so all of those are there so those Integrations are there and also the way just the system is designed if we add another kind of Hub around those source of models all the test work we can generate and run tests of all of this and also if we add in other test type so for example we recently added the toxicity tests tsunami can do toxicity tests on all of these together okay so it's kind of it's designed to separate those two concerns um and now let's talk about what you do and the first thing you want to do is actually generate tests okay and and what I want to explain is what this means so robustness test basically want to show how your models IO model performs if you can if you make small changes to the input so what happens if I introduce typos spelling mistakes what happens if I change the speaker what happens if I change the time right from present to past what happens if I had punctuation what happens if I change a question to an answer or an answer to a question okay can you still can you still answer the question there are different types of fairness tests okay the simplest one would be look classify whether this is talking about a man or woman then I want for example whatever accuracy Matrix is let's say F1 score I'm only generally here's the minimum I want that's a test I want it only for male it needs to be at least that only for females it needs to be that so that's kind of your basic fairness per group there are different types of bias tests and this is where you'll see a lot of things really failing for example um names so really when we started the project we actually had a real customer and we were doing medical data de-identification anonymization and and they let us know that look if you type in you know Mike Johnson came and complained about you know knee pain it was 99 accuracy if it was an Asian name like Wei Wu came and complained about knee paint was only 1991 accurate okay so really it made me like five six times more mistake okay with this group and then when you look at this camera so Katie bug report but really uh you know first of all okay shouldn't we also check you know white people black people Hispanic people um Jewish people Muslim people we have no test here for example of Hindu people per cast Okay because the typical ample cast we have a first name we have last names is we can automatically do replace do the kind of generation as you can see on countries we replace countries automatically in the those U.N lists of countries that are kind of richest poorest and then there are two more classes in between and we can look at it as a age group per religion and there are other tests that look at kind of specific biases like young old Rich poor basically you can add adjectives automatically so those are some of the other things that are there and then there are accuracy tests okay because because if you comprehensive test the model yes you also want to pick the most accurate model or at least you have an accuracy Baseline before you go to production so we've implemented all the metrics around you know the Clio classic summarization metrics question answering metrics that's there as well um and we have representation tests okay so for example I can say something you can say oh I want the accuracy for you know women only to be at least 90 but there's another thing sometimes what you find is you have a protected group and you don't even have 10 women in the test set right and you can say look I'm just going to fail this on just lack of representation okay kind of go and work your tests as well overall there are 10 categories of tests that are there when you run the test let's say you can generate the test and either automatically you can edit them manually and you can see the example here so in the first row we've added a typo in the second row we've added just some context some hashtag in the third row we replaced kind of the name right to another ethnicity and the other tools we look into the presentation and accuracy usually even if in a simple case once you generate SD you can have really form a few hundred to a few thousand tests okay and really the nice thing that's different from you know just just unit testing on software to this is now we can actually generate a lot of this and you can edit it you can play with it you can do whatever you want you can save and load those tests and then after you call run we run the test we run inference and you call report the report basically groups everything by category and then in a config file you tell it okay from this category I want a minimum of 100 past 50 past 80 pass and it tells you whether you pass to fail okay and this is something you integrate the spout to release pipeline so that every new version of the model needs to pass all the tests like any other piece of software okay on top of that if a regulator comes and says oh why do you think that this model is not biased you can tell I'm I'm happy you asked here are the tests and I can leave it as an Excel file right you don't need to be a data scientist here all the things the testes here are the benchmarks here are my here's my release process right and I can I can show you that at least these baselines are being met so you can generate test you can run this the other thing you want to do is data augmentation yeah because if you think about it look if you can generate you know new tests with you know Hispanic names black names female names first names last name country names and you can show that the model is garbage or does not do as well well the next thing you come okay well if you can generate a limb can you just add them to the training data and just retrain with it and give me a stronger model and that's what augmentation wants to do the only thing is you cannot what you cannot ever ever do is take your tests and use them to train the model because that is classic data leakage right you cannot then test on what you trained on because then of course it's going to do better right yes I trained it on the test material so what age dot augment does is it generates new examples only on the training data so if you see the example age dot augment you give it the the your training data set okay know the test it separate it generates new example you tell it where it needs to send the output okay so I generate a new enriched augmented training data set for you then you train a new model and then you can do whatever Library whatever however it is you train or tune your model that doesn't matter and then what you do is you basically you want a regression test okay which means that they want to create a new harness with the new model but I want to load the old tests okay so we have that one liner and then you just call dottran and it does that okay so that's basically the workflow here and what we found is especially on a robustness and sometimes on bias this is actually enough to solve quite a lot and assuming of course the the protected groups behave really behave similarly right so it's really just a matter of text um what you can do now with this we have been working on this for a few months now including kind of you know real production news I would say things to know right now if you look at the current elements in the market the Baseline is very very bad especially a robustness In fairness and bias situation is not good if you're working within any regulated Industries please test what's going on so that you know the position next year or the year afterwards with people rightfully sue you okay so you know at least you should test whether your staff works this is completely open source uh kind of we build this for you um and you know I mean we use it internally we have a full team on this uh one nice thing with John celeb is that kind of we're not you know like we're profitable you're not busy fun day just like we this is our third open source project so we know we can support it for several years it's not depending on Grant it's not depends on funding it's not depending on anything so we're committed to open source Community just like our other projects to do this for the long term and with that thank you and I hope you find this useful [Applause] David thank you so much for all the exhaustive capabilities and abilities for us to take advantage um maybe Jon Snow knew nothing but Joe snow Labs definitely know their things so that's very exciting we have a couple of more minutes for one question okay hi uh so I work in AI ethics and I wanted to ask you you said don't trust the benchmarks don't trust the model out of the box but trust me so what like why should I trust you why should I trust your test I just want to know oh no definitely do not trust me I would not trust me okay my kids no forget it no this is an open source project okay what you do here is you generate your own tests okay so what do you think for example you start with the question answering data set okay usually if you take that you know any of the data sets on Helen you just run them you're going to see better results and what your customer will see in practice so here's what you should do you can start with it they generate more tests with it okay try different valuation try to break it in terms of once it right see what happens what I see is that often once you look at the bigger data set that you've built right nothing but nothing I did right you'll see that there's also I was substantially than what you'd expect in terms of ability to generalize the ability to really handle what you think would be trivial changes in in code and then you can tune your models you can augment your model you can fix that and I'll say we do work in healthcare another thing we often do is we generate the test then we have medical doctors who look at them okay because some of the tests they can say oh this doesn't make sense like this will never happen someone says oh this is great I want to add 10 more like this so you can see in going XL and they basically kind of say oh try all the medications that are you know no don't Down The Spill right like you know all medications or devices right yeah because you know you present it differently the dosage is different it's going to break so all I'm saying is if you only rely on the academic standard academic benchmarks okay I think you'll be overly optimistic okay and I think really and the main thing that's happening right now I think really is overfitting right because really right now you know like if I want to show that my alien beats all the academic management for healthcare it's super easy I just train an element right and it's going to do beautifully because it doesn't look up right essential all right please give a big Applause to David thank you so much for sharing your knowledge with us today thank you foreign