Scale By The Bay 2018: David Andrzejewski, Privacy aware data science in Scala with monads...
you thanks everybody for coming and thanks the organizers for having me I said my name's David from Samoa logic and we're gonna talk a little bit about how we can do data science and kind of a private and secure way with some monads and can type level programming techniques so first kind of word from our I always say word from response for this time is literally true response helping sponsor comprehend super excited yeah so sumo logic we are a machine data analytics platform as you can all you can see right there it's it's right there up on the slide we recently launched in Japan so we actually now have a international site with with that language and I I can come summarize it by piping that back through Google Translate and this is this is what we get actually pretty good and as a matter of fact I love this summary you know we're going how can you have data-driven insight to build execute and protect excellent apps in the cloud so I'm sure a lot of you are building excellent apps in the cloud so if you're looking for data-driven insights to build execute and protect that you can talk with us while we have a booth over there of course we're also hiring and what that specifically means it's kind of sending machine data to our service so logs metrics and you can kind of analyze they have to troubleshoot your system check security things everything like that and so that's actually the context that kind of brings me to thinking about how we can do data science over some of this while maintaining some privacy and security around that data so myself I've actually been SME logic quite awhile by our standards 2011 like I said we're hiring of course we're selling what we think is a great product I also Corgan eyes the bay area machine learning meetup so if anybody ever wants to do some meetup tech talk or host a meet-up Tech Talk at your company find me at this conference before that I was doing machine learning research and before that I was doing a PhD and a few weeks ago I was can anybody know that can guess the costume you know who we are puah and hey hey come on all right so the the talk it's all pretty much there in the title privacy aware data science and what I'm going to propose is that I think data science and machine learning folks could be looking into more some of the information flow control literature from kind of the intersection of programming languages research and the security community I'm going to propose you know how we can kind of use types as guardrails to maintain some security and information flow control and if we kind of wrap all of that in sort of a monadic context we get some of the benefits of a nice composable abstraction that engineers can understand and that other libraries can kind of interoperate with you know that said this these ideas are absolutely sort of work in progress I'm not going to propose that this is the the final ultimate way to achieve all of this I would encourage everybody to think of this as kind of a starting point hopefully you get to see some interesting ideas that kind of get your wheels turning maybe some useful references and I think there's actually still a lot of open questions about how we can kind of best leverage type systems functional programming abstractions and things like this to achieve various guarantees and goals around maintaining privacy and security for data that we're doing data science and machine learning over so previously on scale by the bay last year I had that privilege to be a part of a functional programming for machine learning debate panel and the question was is functional programming the future of machine learning and we were sort of randomly slotted into teams because given this conference I think otherwise everybody would have just been on the yes team I was actually on the know team and and we had to think of a line of attack and it was data science notebooks for the way you know they have the hidden state they're very mutable and this is what people love these are kind of like dominating the market and that was our claim and the other side said that's a very bleak pessimistic way to think about it how and you know they sort of won over the audience and we lost the debate but I'm gonna put a asterisk on that because did we really so if we look at 2018 notebooks are absolutely ascendant this is to the point where they actually have a contrarian backlash in the form of a talk at Jupiter kindness last year so pure functional programming should be so lucky as to actually you'd be having contrarian backlash talks at major conferences that'll be a big milestone I think so you know pushing notebooks to production data science with notebooks all the cloud providers kind of pushing data science notebook as a way to get work done so this this is a really kind of key trend and then as a matter of fact sumo logic is now a part of this trend so we we recently announced some data science notebook support where essentially you can we have some API level integration with a notebook running as a docker image that can really easily get your data out of the service and interact with it in a notebook and we're trying to kind of lower those barriers to adoption so that companies you know data scientists or whomever can do some interesting things with the kind of notebook workflow notebook libraries on their machine data their logs their metrics and things like this so by all means I would encourage you to check that out but this adds some interesting concerns and this is basically my like talk proposal that I sent the organizers you know what if you got sensitive data in the notebook or you want to sort of be a little more careful its interactive you're sort of iterating really fast it's hard to have like guardrails to make sure that you're not doing something you shouldn't be with the data or letting data leak or bleed across boundaries or abstractions and so there's kind of a threshold okay you can't just be careful this is not really a solution that's not really very scalable and you know we're going to walk through how we can kind of try to do better and better by leveraging the type system leveraging you know other pieces of technology so that's the ultimate goal now that can roadmap here we're gonna talk about why the data can be risky what we're gonna try to achieve with it what we're not going to try to achieve here how you know this whole solution that we kind of walked through and then why that's actually going to work and do something useful for you be composable and bring it back to the notebooks context so who here knows who clive humbie and weena dun are all right so their claim to fame was inventing the super market loyalty card program so this is huge our entire kind of ad tech surveillance capitalism dystopia in some sense we have done nothing for it and they have this data is the new oil right and you've heard many remixes data's electricity data's all this other stuff so another uk-based source data is nuclear waste right and and this is kind of in the context of breeches personal information security all of these kind of so besides being something very powerful that we can do a lot of useful things with you always kind of have to be mindful of some of the liabilities associated with data and you know this you have you PII you have various other kinds of financial data medical data everything like this many companies across a wide variety of industries they have some data that's kind of free-for-all and some data that they really want to be more careful with and so you know if you want to do data science you need the data and often you want to do data science over some of this more sensitive information you need to explore it clean it iterate wrangle fit models evaluate models troubleshoot models do this whole loop and you're not going to do any of it without the data and that kind of brings along these risks you know often increasingly we're seeing sort of a regulatory environment around data with GD P R and whatever the California version of that is called I forget as well as just companies you know they're often trying to set an even higher standard than the regulation and they have their own regulations so there's a lot of reasons that you don't want to sort of be fast and loose with the data furthermore there's also this notion of statistical contamination that we'll get to next but you can actually sort of get yourself in trouble from a machine learning quality perspective if you're not careful about how data is flowing through your machine learning pipeline and workflow and so notebooks just kind of put all of this on on steroids I mean the interactivity the quickness you can sort of make mistakes and do things sloppily quicker than ever as well it's definitely a double-edged sword so this really becomes an interesting question in the data science machine learning community so this is a what I like a modern classic paper I would absolutely encourage you to take a chance and read it sometime and it's actually one the best paper at at kdd 2011 and what they look at is a lot of these kind of data science competition Cagle like things and look for what they try to characterize this phenomenon of leakage and a lot of people who have done machine learning a practice can can tell you sort of various horror stories about this in the paper example they actually have a medical one where you're trying to predict page outcomes however the patient ID was sort of correlated they're issued sequentially and they were correlated with like what facility the patient was at and one of the patients or one of the facilities rather is actually hospice care so the prognosis for those folks was was generally not great and you could use that to really get accurate predictions even though they had nothing to do with the medical prediction tasks as such you're sort of cheating and in production machine learning systems you're usually just cheating yourself if you have that kind of leakage another example sort of you can think of it you can imagine that sales force you know they've got a bunch of great talks here this year maybe they're trying to predict which sales leads are going to close but they have a field in Salesforce that only gets populated when the lead closes right like contract signing date or something I know if you just dumped all of this kind of oblivious into your machine learning workflow having any value at all in contract signing date there's going to be a great predictor of leads that close but but you really haven't done anything useful and the this paper is actually interesting what they propose is something they call like legitimacy tagging and they don't really get into the mechanisms of how this might work but they just talk about conceptually you know they what fields what parts of data are like fair game from the machine learning and what are sort of protected or restricted and actually again back to scale by the debate last year our erstwhile debate nemesis from the other side Oscar actually did a really interesting talk about machine learning and feature engineering and and this talk basically had a kind of functional reactive feature engineering technique for events dreams to achieve what Kaufman at all in in the paper called no time machine and basically the idea here is again the issue with that contract field in Salesforce this hypothetical thing is that it's information from the future that that's not really you shouldn't have when you're trying to make predictions and by deriving sort of a declarative functional feature engineering technique from the event streams that can kind of eliminate from by construction this sort of time-travel problem and anybody who does kind of hedge fund quantitative this kind of stuff they're also constantly plagued with how do I avoid the time travel cheating so these are cases where not for just sort of privacy sensitivity security but just for the quality of your machine learning you really want to be careful about how data flows in your program so what are we not going to care about in this talk cryptography right we're just going to assume the data encrypt data is handled securely all the data is done in the clear if you sort of search around for machine learning and privacy and security and things like this you'll find a ton of really cool resources about how you can do the machine learning on the data while it's encrypted and homomorphic encryption and things like this but well we're going to table that for now by the way that car is like a block from my house I saw this oh my gosh all right that's San Francisco crypto lifestyle here also not a goal right now or not a tool that we're thinking about but it would be interesting for you to know about is differential privacy this is a girdle prize-winning stuff a Cynthia to work and others kind of work on this and it actually very very interesting and principled approach to this that has some very high-profile adopters if you actually search for Apple in this you can see the Apple white paper about it the Census Bureau is talking about it it some kind of a they pose it as sort of a computational a very conformal definition of what it means to be private in aggregate again not going to talk about this we could eventually imagine composing this with the sort of techniques we are going to talk about also not in scope our actual kind of malicious adversaries so this is a lot of the research and the papers I found about this we're sort of Haskell oriented and they could make these kind of absolute guarantees about security and absolute you know you cannot do this programs will not be able to do that on the JVM there's sort of a side-effect side effect backdoor and almost anything you do ever and so if somebody's actually writing code against your data they can always do things like just write it out to a mutable variable or something or this absolute top notch exploit right here if you if you run this in Scala you can for most part you'll get what you want so this is also something we're not concerned about we're not trying to actually protect the data from hackers or anything we just want to give engineers tools to help them do the right thing more easily furthermore we're not looking to replace you know an actual organization level commitment to responsible data handling so this is our compliance and data protection officer or I guess this is the assistant to the compliance and data protection officer at Summa logic so the the actual employee is her name is Jen this is her dog max but you know there's all sorts anybody who's kind of worked in an organization where data security and privacy is taken really seriously knows there's tons of you know procedures thinking procedures for the procedures monitoring of the procedures auditing of the monitoring of the procedures and so on and so forth so if you tell your you know PCI auditors that you have a monadic type safe lair and and kind of clap your hands and walk away it's not a great idea so this is absolutely not a silver bullet for these kind of things it's just kind of one more tool in the toolkit so what we do want to do is do useful work with the data but don't allow kind of direct access to the sensitive data or unintended information flow of that data and the real question how can computers help us do this this is why we have them right and the actual approach this is kind of a requirements sketch what I would like to have is my data in an imaginary box and when it's in that box I cannot directly access it I can manipulate it in place I can de secure it but only in specified controlled ways that kind of go through some auditing or some review and you can sequence those transformations and mix and match them and the security level will be set by álast right wins so walk into the a l'amour detail it's in the box you cannot directly access it there's no dot yet you can manipulate it in place and so you know functional programming enthusiasts will already kind of see what's going on here you know I have the data in the box I have a function I want to apply the function to the data in place in the box you get a new box with the result of that function inside I can DISA cure it likewise function programming folks probably can see some patterns this is I want to transform the data and simultaneously as a side effect change the strength of the box so now it's maybe a box that I can get the data out of but maybe I do another analysis where I pull in some other data from a secure source again and now the data is once again back in the box and so I want to be able to compose pipelines of these data transformations and you know basically classified declassify side effects and and achieve this so if we can look at all of these and and kind of box them out we can sort of see some map these to some ideas that we know and love from kind of Scala and functional programming right so we have the box it's kind of this container type this parametric type and you can't access it directly but you can map a function into it this is like a functor and you can process it in this pipeline sense and you can mix and match these processing sequential steps and this is like a monad flat map in the scalloped Harland's so so let's dig in a little bit more and there's actually all this idea can comes from a research paper that I found searching for all of this stuff so when I when I kind of searched I think we had these requirements we had these needs and I figured somebody else had thought of this stuff before and a lot of people had in some sense this was one of the few ones I could understand this is kind of this this one was within my my range of being able to parse what was going on at least somewhat and what they proposed is this kind of monad wrapper layer where s is sort of the level and a is the type of the internal data so this is that they implemented in Haskell the papers in Haskell all the papers for this essentially are in Haskell but basically this is a you know container type with two type arguments one that says what is the level high security low security we'll get to that in a second and the other one what is inside the box and so the levels so this is the lattice model of secure information flow and and this woman who sort of invented this in 1976 sidenote also did the early work for the theory of intrusion to modern intrusion detection at SSRI in the 80 so this is a really interesting idea where we're going to define a formal model of how these security levels interact and then the idea is you have some mechanisms that enforce the model and and what is the lattice and why is this a good way to think about it so a whole lot going on here it's basically just an ordering relation a partial ordering relation over your security levels and the idea is that if you're lower you do not allow higher data to flow to the lower level but you can send lower data to the higher level and that's the partial kind of ordering sense lattice just means that all pairs have a unique greatest lower bound and least upper bound and and what this means is that given two pieces of information I can sort of compute a lowest common denominator level that if I merge the two pieces that's what it should be in the original paper they kind of do this with subsets just to motivate it but the real key takeaway here is this gives you a mechanical way to determine whether code X at level X can read data at level Y and if I combine data at level a and B what what is it what's the resulting level and intuitively the resulting level is sort of the lowest level that covers both of them that's higher than both of them so you don't want to let data be d secured by combining it but you sort of have to find that common common ground so one kind of difference for how we're going to do this versus the paper is the sec lib paper defines a monad family so you have one instance per level and you can kind of map and flatmap within this but they're only connected via these explicit transformations so it actually the paper out of the box doesn't have are like classified declassify effect that we're looking for so we we kind of tweak it a little bit we want basically last right winds join so if this is the Haskell type signature of what the join would look like you have a secure wrapper around another secure wrapper and the inner one is sort of the last last right and so when we join we want the inner one to kind of quote-unquote win so if we sequence a whole pipeline of actions they become nested flat maps which you can equivalently think of as these nested joins and the very last action wherever we end up that should be the final result this doesn't exactly work out I'm not gonna dig all the way into this given the way that it's implemented in the paper basically we have to add an identity element a neutral security level so certain operations are just saying I don't really care about changing the level this if combined neutral with anything you get that other thing and now that you have an identity again kind of stepping into some of these concepts you have a non commutative monoid over the security levels so I can combine any two security levels and there's an identity element that combined with level high gives you high combined with level low gives you low again for the lattice theory from the Dorothy Dunning paper you could have these very very rich security lattices for our purposes and in the psych lab paper we just say there's high and there's low so it turns out that kind of tweaking it in this way is in some sense equivalent to accidentally reinventing this other thing called graded monads so this is actually credit here to Professor Russo from the sec lib paper I emailed him some of my questions and he said oh what you're actually talking about is this and I went to those papers they look fascinating I could not understand anything of what was going on in those papers you can check it out yourself I'm sure it's very interesting work but I really couldn't parse it however professor Russo was was kind enough to give me the summary that it's basically a family of monads indexed by mano AIDS so here again we have the mono ID of the security levels we can combine security levels and kind of hop skip and jump across the monads and in this case the join another way to think of it besides last write wins would be this like join has this least upper bound and this is kind of back to the the Denning idea but what we're gonna do is we're going to do last straight wins and in order to make that work out with the monad and applicative laws we had to add a new security level called neutral and and this is fine so this actually achieves our goals so now we have some conceptual design for this data wrapper box that sort of does what we want and the next question is okay how do we how do we actually code this up right so we have the idea we have the theory we have some haskell snippets now we want to implement it in scala so the attempt 0 and if folks have some better idea after I just rewrote the Haskell code directly in Scala and actually did not really do what I want in some sense because I couldn't pattern match and it's like other type argument if folks who know more about type tags and such than me want to talk after about how this could have been done I would love to hear it but but this I play around with it for a little while cannot make it really do what I like so next one I just say we're going to value encode the security level so we just create a sealed tray to you know some type over the different security levels and then that kind of gives us a straightforward way to implement flatmap here we go that basically if you have the neutral you get the other level and otherwise you get the last right wins in your level and then the reveal method which is sort of outside the monad API but part of the security API is we're going to say when I want to get the data out if it's at the low security level it's fine otherwise it's not fine and for this scheme to work the important thing is that by kind of again we're not protecting against hackers we're just trying to help engineers by convention you want to have only sort of pretty reviewed and trusted code actually owning the ability to write data at the low level in the in a flat map call so you sort of fence off your API or your library design in some way that you have sort of reviewed an audited code that can declassify high classification data and you have other code that just kind of consumes it and as long as you kind of maintain that boundary engineering-wise you don't really have to worry that other code is going to accidentally reveal the data in the in the Haskell cycloid paper they sort of they make this demarcation between trusted and untrusted code and they they have some stronger guarantees again because it's tasks old but here we're just going to say organizationally if you kind of do this you should be ok so can we push this into compile time this this is the kind of type level thing so this is the truth table for how we think about the monadic joint right we have the outer pipe the wrapper and then the inner type this last Rite wins and we have what we want the result to be assuming that the functions that are constructing instances of our security wrapper output fixed and Static levels we don't have data dependent processing here this is this is what we want right so how can we do this I would encourage you to check out this awesome presentation by stuff in this Uyghur about type level computation Scala he has a really really nice example with boolean sort of a to build it up the way that I think about it after reading his presentation several times um you know the trick is you have an abstract type associated with a trait so you have the trait you have some abstract type in it you actually make it tape a parameter if you were going to make this B ultimately it's an endo function it's going to just map back into itself so we're going to add a sub type constraint that the type argument to this type is itself you know a member of the perm perm is gonna be permutation you have some instantiation so in this one we're going to say we have red green and blue and they're all extending that trait and they all have this abstract type member and the abstract type member or member it's going to take a type argument that is also a permutation type and return something okay and now we can actually encode a you know permutation or a bijection unto itself by just supplying the type arguments to kind of fill this out so by you know this this trick right here actually at compile time all of the sort of type annotations kind of flowing through your program the compiler is going to be able to evaluate okay given this type here this concrete type here supplied with this argument I can thread it through and compute what that argument should be and to do by doing leveraging this kind of machinery we can kind of combine it with our idea for how we want the privacy air to work to prevent unsafe data access from even compiling so taking that truth table and boiling it down to something much simpler we can say the inner type when it's neutral this we're going to just accept whatever the outer type was when it's low level last right wins and when it's high level last right wins the inner type is sort of the last guy in that monadic pipeline it's the final security operation that's the winner unless it's neutral which is this thing that we want to kind of make a lot of operations work out and makes sense and in this case we're also going to encode what we want with reviewable which is going to be another trait defined only over the security levels that we are kind of comfortable exposing the data as so we kind of take that same trick from the permutation stuff we just talked about we're going to basically say that any privacy level has this outer that takes an argument that's the outer guy in the monadic join and it's going to return a privacy level type as the result and this again this type constructor in the abstract type member of our security level acts as like a type level function it's all evaluated by the compiler at compile time and it's going to output the results of what the monadic joint should be for this level so that's kind of the way we would implement that trait itself now how we actually use this in the function again we have the reviewable low always wins last straight wins high lasts right wins outer differs to the other guy so the actual join type signature ends up looking like this and we can kind of define map map in terms of join given the way we talked about last right wins for me it was a little more natural to implement join okay so there it is but let's let's break down the components we have the outer level which is a privacy level privacy level outer privacy level inner and we have the argument to the monadic join which is a double nested security which is basically again sort of the result of what you get in the flat map implementation and we want the output to be a new security a single nested security wrapper with the target level that we want and you can see that by sort of this hash tag kind of operator guy we're able to access that the type member of the inner security level and then send the outer security level as an argument to the type level function contain their and that becomes the type of I result so this whole thing just kind of happens and now we get the result we want and what this happens is that if I have a aggregate operation that you know it does something sensible and reasonable basically this takes a string so aggregate is the kind of function you might have in your sort of controlled library it's going to take the string and compute the length now the string is gone this is very simplistic and you just have a number so maybe we say this is okay from a privacy and data leakage point of view and it returns the length wrapped in a low-security wrapper and so if we and you can do this in scala test which i found out in a process of doing this that i didn't know before you can actually supply string to scala test and say does this string compile or not and so that's actually kind of a cool trick but you can see that if we say reveal on the high security by just we just map so again we do the func 2f map and say underscore length okay now we have a high wrapper around our number but you can't unwrap the number because we didn't declassify we didn't have the monadic effect from a flat map to change that security level so just simply kind of functor classic map of the length function is not sufficient to allow us to access the data but this special you know approved aggregate function because it wraps the result in the low security it actually essentially has the monadic effect of declassifying your data and that one does compile so that's it that's what we kind of hope to achieve right and now why why do we do this again let's circle back after I'm diving into the Haskell to Scala translation and type level functions resurface to kind of think about the context of what why we would think this is useful so again we're in the data science notebook and you want to really explore and iterate and play with your data really quickly but we don't want to accidentally pull data we shouldn't look at that we shouldn't combine data in ways that we shouldn't or likewise let data from the future leak let data that has sort of features our side channel information for our data science problem leaked across those boundaries so we want to just kind of have the freedom to not worry about that while we're interactively hacking in a notebook and so we outsource that thinking to this kind of wrapper library layer and again if by kind of Convention and organization you have the code that is capable of unwrapping under some more stringent paths maybe different reviews different audience things like this then you can interactively hack to your heart's content and not really worry about that leakage the machinery is going to kind of make sure that you're in some sense doing something reasonable here so that that's that's the big idea and why can I do this when we were when I was looking through all sort of the related work in the research why did this was ammonium functor and everything and why worry about if the laws work out you know there's another thing you could do you could do what's down there you could have a secret object that you say dot updates and you pass a function and now it changes in place you can say dot set the security level and now it does this you could get and get it out now that said it's not really going to be it I guess you can read it and sort of understand what it does it maybe it's a little it's got that I guess but it's it's going to be really hard there's got a it's kind of a custom API people are going to have to kind of wrap their head around what's going on here it's not um in some sense in the functional side it's gonna be little more standardized and also there's gonna be very hard to compose right like you're not going to really be able to reason about what's going on at any given time the compiler just knows I have a wrapper thingy it's not going to track the type information it's going to be really challenging to make anything reject anything at compile time you know so this they we get a lot of benefits the other way engineers can understand it other code can interoperate with it and so a sort of an example you know and again just to bring back this operation is map or flat or a functor map right this operation is the effect is sort of flat map or bind or something like this so again if you if you kind of are familiar with these concepts already the security thing it should make a fair deal of sense to you out of the box and then finally let's say that we want to construct a privacy aware data pipeline for our notebook right however some operations may fail and return no results oh boy if only there was some Scala language construct that could handle an operation that fails and returns no result at all so so let's let's walk through this so say that this is our pieces that we're building our little example out of we get a query from the user from something maybe it's there and maybe it's not we get the user data it's going to return a monad transformer of option and the security layer we can analyze the data and return a low-security layer we can do a leaky analysis that still returns a high security layer and then we have a reporter that just takes a string and gives you it or it takes a double rather the result of the analysis and outputs it as a nice string and so how do we string all these guys together okay we can kind of put it all in a nice for for loop that we all love or the for comprehension and so we can sort of lift the option target query into a monad transformer option and security we can get the user data which is already going to return the option security monad transformer we can lift the results of the analysis into the same structure and we just yield the result and we have a nice double wrap thing at any given time if the security does what it does or doesn't do and the option these effects interact natively option knows how to make security security knows how to make option everything quote-unquote just works this is done with a Katz library and so that's an advantage of implementing the security layer and these kind of standard formalisms this monad kind of stuff and you can you can imagine if we have a different one a leaky one that keeps it at high or low again we'll get the result we want everything will just work we can kind of hug plug and play these pieces all right so some question for a future work I'd love a even better way to do it in Scala especially focusing on type tag I'd be interested are there use cases for richer layout of C's I'm not sure could we do data dependent declassification like differential privacy well I'll do question one second again I think there's a ton of cool ideas in information flow control and you know PL security research that could be used for machine learning and data science could use property testing for anti leakage and there's a whole new kind of machine learning area of like trustworthy AI about how we can build kind of tamper resistant AI and machine learning systems that I think these programming language ideas could be really useful for so that's the the big picture thanks again for listening I'd love to take any question if we have one minute great to hear these kind of algebraic solutions to problems that are usually just another security level have you thought about how this works yeah so yeah that would be like the latus stuff like hypothetically you should be able to encode that in like the richer lattice model this implementation just kind of does low high and maybe you'd have like a more complex truth table but you you should be able to encode the whole idea of the lattice is that's always well-defined you you if you define the lattice over security levels you always know how to combine them there's always a truth table as long as you do that [Applause]