sfscala.org: Alexy Khrabrov interviews Paul Kinsky
Recording: sfscala.org: Alexy Khrabrov interviews Paul Kinsky
hello everybody I'm Alexa crab off the organizer of SF scholar and here we are on location at lithium where we have an amazing meet up tonight we have two talks one is fast parses heavily and the other one is about extremes with bulking skin here we'll also have Paul Kinski he's also my colleague at nitro so we share a lot of common interests in Scala and interactive processing so I'm going to spoil a few questions so first of all how did you get interested in in Scala and maybe you can outline a little bit your your career path and kind of your technical track and like how did you arrive at this set of technologies all right starting at the beginning I guess originally went to school for electrical engineering realize they only really like digital circuits didn't like all the radiofrequency engineer and calculus stuff all about discrete math then when in the industry started working initially of this big company pega systems working in Java and started working on you know some side projects playing around with Python because you know enterprise job isn't that easy to get started with or easy to do interesting things with eventually you know I noticed that it's harder to refactor stuff in Python harder to like work without the power of like a type system even though say something like Java you're sort of restricted by the verbosity of the type system so through started looking into Scala there's a scholar Coursera course I think like everyone watching this is probably heard of it that's topped by a Martin or dare ski took that sort of fell in love the language then yeah that's amazing right like every time you know we can learn new things I didn't know you did all the Python before so we have to actually have a lot of data scientists who predominantly use Python and it's they think it's hard to switch to scholar maybe you can kind of explain to some of these folks hopeful watching us in the future you know why kind of scholar make sense for social engineering for data processing and is it really dramatically harder than Python or kind of what kind of a good way to approach skull if you are a Python developer um yeah all right so one of our engineers merrick he's gives us talk on a functional programming and using scala for data science really great but one of the examples from that that really just comes to mind is using python running this like 12 hour job doing a lot of like just training some model or something and then at the very end having like just some formatting error and string or some type air that wasn't caught until runtime just like torpedo the whole job causing them to have to run it again so one thing i'd say scala just because everything the types of checked at compile time you can avoid that happening I don't know I mean generally there's a great community of people working in Scala there are a lot of people who are data scientists who work with complex math complex models as like their main job who create libraries in Scala there's a lot that's already there you not like it might have been like five years ago or something you don't need to build all your tools from scratch mm-hmm cool so speaking about community right so I think the first method no physical conference which is a fantastic example of scala committee maybe for folks who who didn't yet go to school events maybe you can summarize a little bit what kind of scholars committees for you how it helped you to to work a scholar and what's special about it yeah so we met at any scholar which i think it's a great example of the scala community because at first there and it's a standard conference you just do talks at our pre-scheduled that is like sort of a call for a proposal that goes out beforehand etc but the second day it's really cool really i'm sort of highlights the community what they do is they just have everyone come in and among themselves organized the day of talks they just in the morning everyone writes in a whiteboard like what they want to present and people hash out like when and where they'll presented during that day it's called an unconference and every time I've gone to any Scala twice they're always like really good series of talks that people are trying out for different conferences or that people just have like sort of come to come up with that they're just able to just like sorry that they just sort of like have on the back burner that they can just show off at the unconference mm-hmm and you actually the very interesting session together with the speaker about extremes and reactive streams right so I just wonder you know maybe you can share with us how did you find this technology and what's interesting about about extremes and folks who don't know maybe you can summarize like what a case or streams are and like how they connect together okay right starting from the beginning acha so actor model you have actors they send each other messages they have each have a cue they process messages it's really great system sort of great for working with concurrency you don't need to deal with mutexes locks all that like multi-threaded stuff that makes a concurrency so hard to work with you just get this like nice low level model that lets you describe like message passing current systems so that's great i was actually at the time working at a company that had all of her back in systems built in akka so you sort of start to notice that you can't really compose actors you lose a lot of type safety if you're working at that level just with actors but they're really great for systems where you can sort of like draw out like a finite state machine of all the different states beforehand they're just like the best thing to do if you have like the specification in your head that you want to just build so akka streams it builds on top of a cactus right so what it does is you use it as you use this dsl to describe your stream processing systems you create these sort of blueprints at compile time these are just like immutable vowels they describe stuff they're not like actually running stream processing systems and you use a active actor system to materialise them right so you turn these blueprints into running stream processing systems that use actors to pass messages so you get all the benefits of akka like the speed the concurrency etc and you still have type safety so all these blueprints your building and composing our fully type safely every fast of them is described by their types like including something that like their materialization types right like so when you run let's say you open a stream that's like connected to a socket you might get like on the sides value that lets you close the socket say and like that's described in the type system then you get these like really well described black boxes you can just sort of like bug together like legos and build up a system from so the fact that it's a skull is actually very valuable because you kind of connect them through types right you can connect okay cool that's disintegrate so and I know that you are writing a book on reactive streams yes I am an extreme specifically right onaka streams for for my name and so we know hopefully that will kind of Cosmo folks to learn about this and you want to share this knowledge I wonder why do you think people should be programming in this paradigm where I like I think one of the reasons to write a book is going to share your approach and teach more people right so how do you think this is going to improve software engineering practices how is going to improve some developers lives and kind of reliability of systems why this is an important approach you know we should learn about and try all right well I think streams are just a great mental model for describing a large class of computation right let's take a few like big data you're dealing with more memory data than you can fit in memory you sort of necessarily have to process a chunk by chunk so you have a stream of chunks that you're processing stream processing right you have like a stream of log data you have a stream of events from like Internet of Things devices for all of that like the most natural way to process it is as a stream of events or a stream of data then you have for example like low level stuff on Linux right so your say cat a file you grab it you may be pipe it to another process that's a source of like bytes being piped through transformation pipeline and finally to standard out to your console right that's a string mm-hmm you're looking at like TCP that the stream of bytes order you're looking at like web sockets streams going both ways modern services right billion dollar companies like twitter twitter i think it's a billion-dollar company anyway companies like Twitter will cut that they're just like streams of messages being fed from multiple sources aggregated fed back to people Netflix stuff like that it's all streaming media so it's like sort of a common denominator everyone's working with streams and if you have something that can like natively in the language of like streams and transformations over streams if you can define your system like that then you can do it very concisely and you can do it in a way that accurately models the property oh man this is the songs also so we're looking forward to a token thank you very much for sharing with us let's go I'm glad to be here thanks