SF Scala: Meredith L. Patterson Interview

Recording: SF Scala: Meredith L. Patterson Interview

[Music] who everybody I am Alexey Krylov the organizer of a chef Scala and actually have a joint meet up today with SF skull and Bay Area Haskell user group and we're at formation the eye which is a company known for using Haskell and Scala and elm and really good functional programming languages for real startup production work so we're very happy to have Meredith Patterson here with us she's the founder of flying sec like second network it's a it's a it's a idea that you can model inputs to a system as a programming language in my cookie understanding and Meredith will tell us more but she is doing the talk today preceding the Lange sack workshop by xxx ik workshop which is held here in San Francisco so very happy to have you here Meredith thanks for having me so maybe tell us a little bit about Lange second what got you interested in in everything to create it well it actually kind of started with an argument so back when I was in graduate school I was studying formal language theory with Professor named Theodore ruse Romanian guy wasn't doing security at all and there was another student in the department who you know I used to hang out with him sometimes and get dinner and you know we were eating dinner and he was talking about SQL injection mm-hmm and you know so he described the problem you know the idea that you could terminate a an SQL query early by like you know inserting a semicolon into the into a form and then adding whatever commands you wanted to after that and then just you know finishing it with a comment a symbol to you know to comment out the rest of the query that you know the the original developer intended mm-hmm you know and so he's explaining to me that you know people often try to prevent SQL injection using a regular expression blacklists or whitelist and I said - I said to him that's kind of strange why would you try to use regular expressions - you know it's a mask off things in you know a context-free language you know Keogh SQL lets you you know SQL admits you know lists of things for instance yes and he was like okay well what would a context-free whitelist or blacklist even look like and I was like well I can see it in my head I guess I'll just have to turn it into code and so that ended up being my 2005 black hat talk a bit got called ejector mm-hmm for I mean it was it was kind of it was almost a poor man's prepared statements in a way it was it was basically a shim that sat between you know your database and the form then was you know the the that users were submitting things to your database through mm-hmm and it would pattern match the the parse tree of the query against a set of templates that you know that match you know what you had used when you when you programmed it mm-hmm and if it didn't match that you know if the if the parse tree was something that was something different than one of the templates it was expecting it would reject the query mm-hmm and you know other people had had similar ideas you know in roughly the same timeframe like you know I think it's about 2007 that said Shaw for instance decided to write a write a web server yeah this was this was one mongrel or you know so like he he decided to write the he decided to write the the parser using a state machine generator called ragel and you know the idea being there - you know rigorously recognize the the language of requests and be able to throw away you know malformed queries because the thing is you know most malformed queries you know turn out to be somebody trying to you know exploit your web server or you know whatever their whatever or just wasting your time wasting you better yeah exactly you know wasting your bandwidth wasting your CPU cycles you you know you should really recognize that quickly and throw it away just to just to save you you know time and money hmm so yeah like when after he finished writing it you know they decided to just as a test to you know throw up mongrel behind Apache mm-hmm and mongrels stopped like 80% of the you know the attack traffic the Apache just let through interesting yeah so I mean it was it it took you know several years of you know of people sort of independently realizing gosh you know if we introduce more rigor more formality to how we handle inputs you know were were able to achieve greater security and greater reliability just by you know not trying to process things that don't make any sense right these are bunch of people learn anything from that I mean normally like you would just back work longer all parser into Apache right like did they do anything about it I mean that hasn't happened to Apache as far as I know all right I mean a known or anything like you know nginx nginx still uses a giant hand-rolled recursive descent parser which has given rise to plenty of exploits if it was you know hand-rolled parsers are difficult to debug interesting so so that's kind of your current focus on occurrence although workshop but you known for different things so how did how did you arrive and you know like I I know that that you you will to study accommodation linguistics and right and so I know how much of you know folks at Penn where I you know went to grad school and kind of I I see the connection right so a lot of folks like like linguistics capitation linguistics they end up writing parsers right and so how does it work for you like how what attracted you to computational linguistics in the first place well so I mean this was Josh back in like 2001 right I I had finished my undergrad I had actually you know gotten an undergraduate degree in theoretical linguistics largely because it was the first major that had managed to hold my interest long enough for me to graduate I I switched majors a bunch it took me about six and a half years to get my bachelor's so then I spent like a year doing tech writing and decided that tech writing and I didn't have a future together so I started applying to grad schools got into the University of Iowa in there linguistics program mm-hmm and I figured you know computational linguistics would be interesting I had a bit of a programming background cuz you know my dad had brought home a PC Junior when I was like eight years old and I'd learned basic so you know I knew a little bit about what I was doing but you know it turned out that the the program at U of I it was you know pretty heavy on the theory side you know we we got in a corpus linguist from mitre my second year in the program but I ended up getting poached into the cs department by the formal language theory guy that i mentioned because the cs department offered a computational linguistics mess and you know he was tenured so you know he sort of handed me a couple of papers like over the course of the semester and you know invited me to his office to discuss them and you know i didn't quite realize that i was sort of being groomed for a PhD but you know that's sort of what it was you know yeah and so at the at the that semester he was like you know would you like to come to a PhD with me and I was like sure I'll try that and then yeah so that but the thing is like he was a he was a formal language theorist and he was more interested in you know rule-based mm-hmm systems which kind of went out of vogue like towards the end of the AI winter yes and you know I was I was paying attention to the literature and I realized that you know the field was progressing in a more statistical direction yes so when the department brought on a data mining guy from UIUC I was like okay I'm gonna I'm gonna jump ship and that actually turned out to be a mistake because the guy was a total micromanager and eventually got fired for plagiarism but by that time I had already rage quit and moved off to the valley because I got a I got a good job offer here interesting so it's so in a way kind of solve them you know am a logical question right we've seen that basically happening so AI used to be logic and you know illogical sisters and inference and now it's all statistical approach same thing with language she used to be no nice categorical grammars and other kind of formalisms and now it's all deep learning and so you know I like I've been that email ope 2015 and Belcher gave a keynote about learning and basically a linguist has like is there anything Tamara cannot do like do you need linguists guys so so it's so in legs ik you basically you're doing grammars here doing kind of traditional parsing right which used to be the bread and butter of computational mystics and now of course everybody wants to do security with the hi and you go to the airport and you see you know this you know somehow silence science and things like that just like everybody basically wants to do patterns but but they want to do the blurring so so what's your take about this I mean this is something that the talk tonight is actually going to touch on to some extent right but because when we're talking about you know like a file format or a network protocol you know we're talking about something that has a defined specification yes you know even if the only defined specification of it is the code and that is a thing that happens it's still ultimately deterministic yes you know it's unfortunate that more and more often these days we tend to see you know the only specification is the code because you know that's very difficult to reason about yes it's you know and you know one of the arguments that that I make is that you know if you use the tools of formal language theory you can reason about your about your protocol before you've actually committed it to code and so if there are any mistakes in you know what you're specifying you know you can reason about them at the spec level and you don't have to go and you know dig through a bunch of code that you've already written to find the mistakes that you made and you fix them and the goal of course is it's an accretion of people cuchara and right and exact right like it's not little truly specified because nobody really knows exactly all that you know and like you know people's understanding of what the code is supposed to do ends up diverging from what it actually does yes you know but crucially you know input languages are definite mm-hmm this is not the case for natural language you know there's a lot of ambiguity in natural language yeah the title of the talk tonight is ambiguity is insecurity and you know that divergence between the human understanding of what the code does and the machines understanding of what the code does is where vulnerabilities are I arise from right so you know it actually strikes me it's like it's really something very important because the I think the security which people try to do machine learning deals people right basically people who don't kind of crazy stuff and people are not automatons so we don't know what they're gonna do right like we're gonna specify the behavior they're gonna try to send you some packet to support and then it'll try to log in and then like poke at the touch sensor here and push the button sound like there is no real order alright but obviously if you send you know binary packets through the wire then you know there is a port above somewhere right all your like internet forums there is some some code behind this but I'm curious the world becomes multimodal right so a lot of people now who do for instance a blockchain which is usually handled by some apps talking to a distributed system they basically the input language is you know touchscreen like you're touching an app you're touchin touched answer you are doing this iPhone X face recognition you're basically so so can you specify kind of reason about basically all of this essentially like you want to record all the key button keep like basically videos legitimate signals I mean you have to decompose it correctly right I mean you're talking about a couple of layers here right so the the you know the the the the physical layer the the you know the person interacts with right they're gonna touch the screen and there's going to be some system that recognizes you know where the where the touch occurred yes and the output you know like that the coordinates that are output from that will be input to the next layer and then figures out okay what region of the screen as far as you know we've divvied up the the regions for this app you know what's you know what's been touched you know what you know the the coordinates are the the input to the system that tells you all right well what got touched yes and then the thing that got touched will have some functionality that you know is in turn a third layer down mm-hmm and so on and so forth there's some loop like through the UI loop essentially she waits for for your events so so what do I mean would you kind of reason about like would you try and what people let's say let's say there is a bug in there right let's say the OS like because it's you know it's iOS and there is a bug on some kind of UI kids so you touch in one place it registers occurred other coordinates or you push a button right so so like in otogi bugs you need to somehow a lot of things like you know which people do to it you you touch like screen the push buttons so do you think do you know like can we sort of in trying to think like can we enhance it expand the definition of language of input to basically everything you can do like to my little iPhone right like it's its its inputs its surfaces buttons so the language of inputs to the iPhone consists of you know your fingers touching it you know you're looking at that and like everything it's camera sees accelerometer recording movement like basically that's what you do to it right it's I mean these are these are yeah these these are a kinds of language yeah yeah I would I would absolutely say that okay you know and I'm there there's an example involving cars in tonight's talk that you know that basically speaks to that but you know another another talk worth checking out is sig Cox's talk from lambda Con 2017 mm-hmm that's you know she hurt her that talk is mostly about user interfaces but it's about sort of developing a vocabulary and a language for you know for interfaces Oh knife and similarly parry Metzger's keynote from the lying psych workshop last year you know one of the things that he talked about is you know formalizing the you know the state transitions of of protocols as you know as languages as well mmm-hmm interesting so so basically probably it's just we will have more like more a booth we need to roll the notion of language right like we need to include all this stuff yeah let me figure out how to characterize them mm-hmm so so like what is but like what would it be like you did the work on on data mining Postgres queries so what would would you like plan for the next year at you like you know like what would you propose to the machine learning people who do security based on traditional data mining like load over logs into a chicane Jake datastore and like unleash deep learning of the right now somehow but I don't know how like it's very hard to get and the day the data of exploits right because it's sparse and I'm gonna get the training that data said well there's there's two pieces of advice I would give the first one is you know learn from the past mistakes of others you know because big data has has been a buzzword in security since you know 2009-2010 and people made a lot of mistakes trying to you know trying to identify you know malicious traffic automatically mmm-hmm and that actually feeds into my second point which is never ever ever underestimate the creativity of an attacker mm-hm I mean the languages of of network protocols and message formats and file formats are are constrained and human creativity is unrestrained and this gets back to what I was saying earlier about the pigeonhole principle right you know if you're if you're trying to you know if you're trying to get a a regular expression you know to match like you know a the end be to the end that's it's not gonna like you can basically always construct a string that will that will not be what you were expecting yes and and and additional complexity when it comes to you know when it when it comes to language power just provides attackers more degrees of freedom interesting and maybe you know the final question obviously it's on everybody's mind you reside in in Europe you know we just everybody receives everyday global privacy but updates and both the wonderful GDP are right and then basically I live in Brussels usually Russell right this is the Citadel of the bureaucracy right the west side of Brussels I don't live on the park you know I thought I thought this is this stuff is not touching me because it's just a bunch of emails and cookies but it really hit hard because I like domain names and I like the cookies and one of my demand providers sorry you know this thing is killing the Whois we're gonna we're gonna delete the quiz we cannot give you the quiz anymore because gdpr is killing the Whois and like like no like you're like this aggressions not gonna Stanley you're not touching my cui's is gonna be an interesting one yeah yeah so how do you think in a world of the beginning of this and so currency basically reaction to GDP are is to power you know private privacy update and then email like you know gtp are at our company comm complain to that right so basically like it's not really doing but amou see like an attack on a who is a strong one what do you think how do you think this will affect the security field will attack on security of everybody will it lead to new innovation it's just another brazos bureaucracy thing globally you know affecting global users I think one very practical effect that GDP are is likely to have is that you know for end-users you know I I I think or at least I hope that you know a lot of the data that is currently collected being collected on people is just not going to be collected yes yes you can't own what you don't have right so if nobody's collecting it then it's it's not gonna be but it's a good thing right like it's like a mega thing yeah I mean at the at the end of the day I think it's ultimately going to be a good thing for users but it is going to be extremely challenging for you know for enterprises particularly small and medium-sized ones to implement all right well let's see what happens and we're looking forward to your to talk thanks for being us thanks please [Music]