Bay.Area.AI: John Andersen, AI-Powered Personalization Drives Decision Intelligence
Recording: Bay.Area.AI: John Andersen, AI-Powered Personalization Drives Decision Intelligence
um so so as Joel mentioned I'm gonna introduce cognitive scale and kind of our product and how we're applying that product uh to help solve for things around AI personalization I don't know if we we invented personalization but uh we definitely have had a lot of time productionizing some uh very cool use cases across different Industries so um what I thought was we can you know have some time in PowerPoint and help set up the context but um you know my role as at cognoscale is really leading all solution architecture and and Consulting so that's everything from sales engineer to client engineer so I'd like to also show you guys some of the capabilities within the product itself um and see what you guys think right okay so just a brief overview of um cognitive scale so we're located in Austin Texas that's actually uh where I'm I'm from and uh you know as Israel as Joel mentioned I just want to thank you guys all for hosting us um it's great to be out here in an in-person event and always happy to come to San Francisco uh so if there's more of these I hope that I'll be able to come back if I don't butcher this uh presentation right um as as we mentioned we're founded by uh senior Executives from IBM Watson so this was really the first group that uh productionized or commercialized the Watson um QA system that was coming out of Jeopardy and that's that's how I got my start really and I was talking to Lexi earlier you know what brought you to cognitive scale um and it was the people right it was the the individuals that really started the Watson labs and really looked at ways of you know how they could take something from you know spawned out of research and apply that to you know curb martialized product and that's you know what really you know spawned this startup in Austin um you know as a small go to market we do Focus us primarily in a couple of different Industries so primarily that's in healthcare as well as financial services but you know even though that you know our I would say our breadth of clients is an overly large the clients that we do have have a very large member population so we do service actually over 100 million individuals here in the country whether that's coming from large Banks to large Healthcare companies now what we do is we're not I would say a data science platform right we're not here to author you know new machine learning models there's a lot of different you know Technologies and products out there today that are doing that but the space that we really help fill is how do you take you know an Insight that's being produced out of some model and how are you bringing that into an application at a system level and this is something that's come up you know with the Foresters and the gardeners really in the past you know 12 to 16 months where they're looking at this discipline around AI engineering or forcers calling an AI maker platform which is really you know understanding that there's many different personas out there that need to come together when it looks to being able to build out an end-to-end application so you have your data scientists your data Engineers but you also have your software developers you have Architects you have you know business owners smes even you know legal and compliance it's how are you able to start to bring all of those individuals together in a very you know consistent and a repeatable way to start to drive some of these meaningful outcomes that can be leveraged within you know personalization framework and that's what our product cortex has really aimed to do think of you know us as you know this intelligence layer that's going to sit between your system of record and system of Engagement you know on the left hand side of this bookend you have your existing data science workbenches you have your existing data engineer and data pipelines now we're not here to displace that and in fact what we're here to do is augment that and sit on top of it to help maximize the existing Investments and preferences that organizations have already been put in similar story can be said on the right hand side of this bookend which is really your system of Engagement you know you already have out there tools and technologies that are driving your digital channels your web portals your mobile apps your CRM systems right what we want to be able to do is help bring together you know all of the different data assets all the different models and be able to start to drive this personalized intelligence into your existing you know end user experiences and we have in this framework you know a platform that really is enabling you to connect into any type of machine learn model or non-machineal or model so this could also include things like rules engines deterministic algorithms business logic even you know rpas um to be able to package that up in what we call as a cortex skill think of that as you know an integration point or an interface around your existing um you know models and engines and what we're able to do is we start to publish these different insights or data elements into what's called a profile and this is really being able to start to build out you know a rich knowledge representation of an entity whether that entity is a person place or thing so in healthcare we build profiles of members but we also build profiles of claims we build profiles of providers and banking we'll build profiles of products in real estate we build profiles of houses and what you're able to do then is start to draw out a longitudinal record or an understanding of really that entity and how you want to be able to apply the appropriate level of Engagement um that we think is you know the best way of servicing that individual within your existing engagement we have apis that are plugged back into the profile which then allow you to you know learn which ultimately that can help become your feedback store which is you know leveraging your model development to retrain those algorithms but also we use that feedback as a mechanism to start to monitor or measure kpis that are meaningful to your line of business and then underneath all of this is a governance layer because the way that we view personalization you know the the trust comes from not just how often are you able to get the answer right to that individual but to make sure that any of these insights or decisions that are being made are being done in a trusted and transparent way now to boil that down kind of like double click you know this this profile you know this this nebulous entity what what is this really able to do well you know the company of cognitive scale has been around for about nine years now um and while we started in the healthcare and financial services space we also did a lot of work in retail digital Commerce and a lot of the things that you know we had as requirements coming into our product uh came from retail where it was you know asking us questions like well how would you you know track an anonymous person on our website how would you generate an Insight in real time um how would you provide this more of an omni-channel experience where if I had somebody you know working on the web portal but then they switch over to the mobile app and then they call up at the call center you know can you keep all that contacts and so we brought in a lot of these different requirements and capabilities into this product that's called profile of one and we distilled it down into you know five key functional areas the first is just let's build a profile um you know that can come from a series of connectors that we have as part of the platform which is connecting into your existing data Pipelines being able to package your existing models and we can do this by you know presenting new leads into different types of organizations the second thing that we do is then we're able to start to select you know what is the right Outreach that we think is appropriate for this individual the third is that we then learn you know are these are these messages resonating we also can measure the effectiveness of this as it relates to you know key kpis and then we apply Assurance to this now conceptually the way that we think about this is how are we really influencing kind of an end-to-end user journey and we we like to call this kind of like a waze-like experience so similar to if you think about like your favorite GPS tool Apple Maps Google Maps ways right what that's doing is it's going to guide you in real time to get you from point A to point B and as events are taking place it's going to trigger some type of Outreach and alert you to maybe you know switch directions or follow a different path well the same analogy can be applied to an individual as it relates to your Healthcare member experience or your your banking customer experience you want to be able to publish these events onto this profile you know through some streaming mechanism and as events are being published you want to have that feed into other models that are listening in that can generate new insights and that can then help with understanding the new types of engagements and if it's a trajectory that we're observing around an individual that we want to avoid like it could be you know we think that this person's about to call into the call center what you can then do is you can trigger you know some engagement strategy that we call an intervention which is then being able to start to surface a message to that individual in a meaningful way and get them back on track of where we're heading right uh a key thing that we have with all of this is really being able to run through simulations so you know rather than deploying this out in the wild and tested these types of messages in a you know pilot or a B type of way um what we want to be able to do is run simulations prior to that deployment to be able to see how optimal that this you know engagement strategy would be it's it's very similar to you know we're just talking over here about F1 um so in Austin there's an F1 race coming up in two weeks in coda you know you would never you would never expect that person or that that you know team of people to build an F1 vehicle assemble some parts make modifications and then just throw it on the track and hope for the best right but that's kind of what's been done you know in the industries around how AI is being run it's you're cobbling together a bunch of models or some other types of rules and then you're kind of hoping that you know the very first time that you send it out you're gonna reach the kpi or business value that you're expecting um and then if you're not you're going to collect that feedback and then eventually over the course of several months you're going to get there but you know businesses aren't that patient and so we want to be able to do is shrink that time down as much as possible and be able to start run kind of these test beds within you know a way of simulating the effectiveness to get you to that kpi as fast as possible and then the last thing right is whenever we are making decisions when it comes to personalization it's important that we do understand that you know these decisions are trusted and so when we think about trust we think about you know being able to apply several different factors um around an Insight or a decision that's being produced and what you're seeing here is you know six factors that we bring into the the product as we look at you know trust and what's interesting is this you know pie graph is being represented on the right hand side these are metrics that are more geared towards data scientists right so if you have a model you know you should be able to start to calculate things like how explainable is this model you know how robust is it is there any potential bias right on the left hand side you want to look at other metrics of trust that aren't necessarily the responsibility of the data scientists but other groups like the effectiveness of the model is going to be more around like the business owner data risk could be coming from your Chief risk officer compliance is you know your legal team it could be Auditors um you know it's interesting that when we talk to data scientists about this they do not want to deal with regulations like nobody wants to go through and understand what's happening with CCPA or gdpr as it relates to the models they're being built um but that's what your legal team is asking you and so there's this friction right that takes place between the output of what your data science teams are producing and what your legal and compliance teams are expecting and what your Auditors are actually tracking and so we want to be able to do is provide this level of trust and transparency across all these different groups to ensure that that level of collaboration of transparency is done so these models can go into production as fast as possible you know technically the way that we think about this is you have all of your models that are being developed you know that that is your IP that's not something that necessarily should be opened up and so we treat every model as a black box um all all we need is really just a input data set and an API that we can run the predictions against and the way that we you know go through levels of explainability is we use a genetic algorithm that's able to start to produce counterfactual explanations um and what that ultimately is going to do is it's going to start to assess you know which features within this you know Vector set what I have to change to come back with a different outcome so you know if I have a model where I only have to change one feature then that model becomes 100 explainable you know the more features that I have to change the less explainable that model becomes and what you can do with this counterfactual explanation is actually start to produce additional scores as it relates to things like uh you know fairness and bias is there any potential discrimination that could be happening against some demographic group based off of gender race ethnicity Etc right so that's kind of at a high level you know the the behind the scenes at least how we look at trust and how we apply that across any of the models and so any model that's being published and registered and applied to these profiles they're being scanned with these metrics um as a way of being able to track over time you know is there any potential drift so at a you know broad level the framework that you know we we follow um you know it starts actually with not the data or the models it starts with the business goal which is it's a little unique in thinking about this so when we we work with um you know different business units it's always the you know what is that kpi that you're looking to drive um are you looking to improve conversion bias start by a certain Target are you looking at you know improving the funnel of leads coming in for your marketing team are you looking to deflect calls etc for your call center and then from there what we then are able to do is apply some of our industry standards around these profiles and we're able to start to plug into your existing you know data sets be able to hook into the existing models that have been developed and we start to publish that onto their profiles right you can then segment these profiles into you know what it is that you're looking to Target we call these cohorts and then what you're able to provide then is a series of interactions we call interventions which is what are the different ways that we want to be able to outreach to these people whether it's you know sending them an email when some event takes place to a push notification on their mobile app or you know having uh them interact with the chat bot you know these are all different things that actually get plugged into your you know system of Engagement um which is actually then gonna go off and send those interventions and before you ever go and actually you know send some you know personalized Banner over your Adobe experience um or your Pega your CRM system what you have is the ability to simulate refine the effectiveness of how well we think that this is working go through the workflow deploy it then it'll start to send off all these interventions and then what you have is the way of being able to start to Monitor and measure this um which we again have a way of being able to calculate things like bias fairness robustness explainability or you can also plug into you know companies like weights and biases as you're looking at things like performance of a model over time right so that's at a high level kind of the framework that our product fits in um but I think you know based off of that let's let's actually show you some of this so you can see you know visually how all this kind of comes together oops okay so just to kind of introduce uh uh the use case before I you know just kind of jump into it so I'm gonna I'm gonna walk you through um a specific example within the healthcare industry this is going to be a call center example so um here what we're going to do is we're going to look at ways of how we're able to start to apply personalization efforts to do things like improving your call deflection rate right so um you know it's actually a pretty substantial problem with you know large Enterprises that have you know billions of individuals that they Service uh they can have up to 100 million actually well depending on some companies they got 200 million calls coming in easily per year um so if you can just deflect you know a very small percentage of that that's a crazy amount of savings right um so the first thing that we can think through about this particular use case is you know how are we going to be able to start to build out this profile um to be able to start to provide some level of personalization um and what you're seeing by the way on the screen before I move off of it this is called our cortex Council so this is you know what different roles are going to be using to start to construct that workflow that I just showed you on the previous slide and it is role based so you know you have you know specific capabilities for data Engineers for data scientists for software Developers for domain experts Etc but what we have is a way of being able to start to hook into a wide variety of different data sources we support over 260 different connectors uh that give you flexibility into first party data third-party data structured unstructured social media business applications like Salesforce you know for uh call center that's that's you know pretty pretty common and what we do with those connectors is then we start to construct data sources and then transform you know that data into profiles just given the time that we have the demonstration what I thought would be more effective is just to kind of show you what an end result of a profile actually looks like now this profile is again it's a longitudinal view of an entity and so what I mean by that is you know you're able to start to keep track of all the different elements of this entity and how that entity is evolving over time and within this visualization you can key into what are some of the specific attributes that make up in this case a member's profile of one the way that we think about a profile is really broken into you know three distinct elements the first is what we call as declarative information so this is leveraging the connectors into your system of record where we're able to start to bring in some data that's been explicitly found um like you know general information like their demographics the the second type of data though that we bring on top of these profiles is called observations or observed Behavior so this is streaming data that's coming in um this is you know click stream data mobile app utilization call center inquiries um where you're able to start to very easily through an expression editor create what are these calculated attributes where you can provide analytics or trends of the data that's coming in and start to do things like you know roughly how active is a person by time of the day versus day of the week and then the third type of element that we bring on top of these profiles are the models themselves so this is again any model that's being developed in your existing you know workbenches regardless if that's a Jupiter notebook sagemaker H2O AI Etc we have sdks cli's apis that enable you to start to publish these models or the outputs of those models as inferences on top of these profiles so now what you're seeing is you know for this particular individual what's the likelihood that they would end up you know picking up the phone and calling into the call center so it's a combination of kind of those declared observed and inferred attributes that that make up this temporal profile of one and as I mentioned earlier right we're not just building profiles of people right we build profiles of any entity and hopefully I didn't lose my connectivity when I showed this earlier but you know I'll try some meet up anyways um but so for an example what I mean by that is you know for this particular individual member what you'll see is that we have you know a list of claims or historical claims right that are tied to it if I click on one of those claims what that takes me to is a new profile so this is a profile of a claim and and what we're able to do then is start to draw out a network of how these entities relate to one another that's going to give us then a foundation that's driving personalized you know efforts within some type of business process you know I was in Orlando a couple of weeks ago and um for an amusement park and uh the way that we were explaining it to them was look we'll build your profiles of customers you know visitors that are coming into your theme park today but will also build profiles of rides you know your amusement I mean if we know that today a ride is only maximized at 50 of their efficiency like they're only going to take 50 of people coming in that's context into you know how you'd want to be able to recommend should we show this person to go to this right or not um so it's those types of things that we look at as being able to bring together the profiles and the context around those profiles to drive you know these types of initiatives now the profiles themselves as I mentioned you know everything is enabled through an API it's actually optimized you know to hook into this profile API within your you know end system so you can call this directly through you know into your mobile apps or into your web portals or crms you can we also have direct connectivity into Kafka if you want to just you know stream it that way or if you want to use some you know jdb jdbc connector and run this as like a bulk batch or as a spark that's all enabled you know within the platform itself um okay so what do we do with those profiles though once we've started to build them uh I'll just kind of show I guess this last piece I guess we have what time do we end do we end it seven or okay I'll show this last piece and then if if there's any other questions we can we can talk into some other stuff um which is you know how are we are able to provide the right engagement to an individual with with this uh profile data right and this gets into um really that level of understand the trajectory of an individual and how would you start to simulate what the right engagement strategy is prior to deploying so again service experience we're looking at ways of deflecting calls within the healthcare space what you're seeing here is you know these three different components the first component is how do we start to identify a cohort right how are we looking at a segment or a population that we want to Target today um and behind the scenes all this is is just running a simple filter query on top of the profile data that I just showed you so like I mentioned earlier we have an expression editor that you can you know write all these expressions in and very quickly create segments on the Fly um and that'll allow us to start to generate these cohorts the second component here is the goal what is it that we're looking to achieve you know here we're looking to improve call deflection again if I you know click on this and open it up behind the scenes what this has given you is a way again of creating a very simple expression off of that data where you can start to Define that that analytic you also can start to identify you know what is your start in value where do you think your call deflection is today what is your target State like what's the ideal Target state that you want to get to uh how long would you want to be able to start to run this type of Engagement strategy and how frequently do you want to be able to measure this this this ultimately becomes your feedback store that you can then bring in and start to measure and monitor those kpis and then finally the last piece here is our mission which is how are we going to go out and target the people in these cohorts to achieve this goal so as I open up this what you're seeing in this visual experience is actually a way for individuals to start to create these interventions as I mentioned right an intervention could be something like you know I want to proactively email a person whenever their deductible balance changes or I want to um notify them you know over their mobile app uh if they're about to get an expected Bill even things like I think some people are likely to call in around this type of you know inquiry rather than letting them call in I'm going to send them you know a link to some chat bot that's already been configured to answer those types of questions and each of these interventions is actually linked to an action so if I was to deploy this right it's going to go off and it's going to integrate with you know some type of like email server which is going to go off and start to send these emails but what you can do is not just you know create these interventions in this environment but you can simulate it right so what I'll do is I'll kick off a quick simulation just so you guys can see the the visual here and you know what this is doing um and as this loads I'll kind of explain what's what's happening here so the first thing is that you know the the platform itself it's a dedicated instance we're optimized to run on you know all the major cloud service providers AWS Azure gcp or you know even on-prem um it's it's all deployed natively on top of kubernetes um and what's happening is we're just going to spin up the appropriate you know compute uh necessary to you know run this uh job right uh once this loads up it'll then go through and it's going to go through uh initialization and a training phase which is you know given each of these interventions how are we gonna you know basically you know train this model which I think there's just you know could be a you know like a recommendation model uh against this particular goal um we call this our amp engine AI Mission planner engine what you're well you're kind of helping out with is you know for a way of reducing the time it's going to take to test out this type of Engagement strategy I mentioned like in the wild but also what you're doing is you're you're basically training your own machine learning model you know for this particular process um so what's happening here is the training phase is starting on the on the right hand side it's going to run quickly so don't don't take your eyes off the screen um what you'll see here is profiles that are found in the cohort and it's starting to map up you know what do we think is the right engagement versus uh for these you know different profiles and that was you know completed very quickly just because I ran you know only I think 100 Cycles um but basically what you saw kind of flash on the screen was as a as a profile was coming in in that um you know Space Invaders type visualization the Green Flash meant that by sending that type of message to that person we think that their call would be deflected red means that um you know we don't think right they still would end up calling in and what you're left with is kind of now A View into you know what the success is for each of these different interventions so let's say in our cohort of maybe 10 000 people for you know this one intervention of the simulation thought that about almost you know 947 would be good candidates uh based off of how this was configured with preconditions as the end result we think that 255 calls would be deflected um and I could look at that and I could say yeah that's that's good enough for what I want let's go off and deploy it or I can say that doesn't sound right um I think it should be more or should be less we have a whole like process in here that you can actually go through and refine these plans which you're basically you know going through and starting to uh give your own label data and that then goes in as a way of being able to help retrain these simulations to get them more um you know precise over time and that workflow you know eventually goes into a refinement stage and a deployment stage which will then you know once this gets deployed it'll then go off and start to execute these messages against your cohort um and then we track you know the TTL for each of these messages and we're ultimately bringing that back into this you know kpi store so business users can then start to see you know is this moving the needle in the right direction of what I'm expecting uh from an application right I think that's unique in the way that we think about this because again different groups are focused on different metrics you know oftentimes when we're working with data scientists they have a model and they say well it's you know 95 accurate or some measure of their precise preciseness um and that doesn't that doesn't you know cross over to what the business is expecting right what they want to know is well how many calls are being deflected and so this is a way of how you're able to start to help bridge some of that that friction or bridge that Gap across these different groups so as data scientists you're focused on what you're doing best which is develop author new models um and as the business is looking at this and evaluating the kpis then that's giving you more flexibility and freedom into you know retraining those models adding new features to it and make it more effective so this is kind of you know the end-to-end life cycle as it's being built you know within this this platform cortex um so that was kind of everything I had I mean there's a lot of other things I could walk you through in the demonstration but um I think that was kind of everything this particular talk so um probably end there and see if anyone has any questions foreign