sfscala.org: Alexy Khrabrov interviews Chris Vogt about Advanced AI
Recording: sfscala.org: Alexy Khrabrov interviews Chris Vogt about Advanced AI
hello everybody I'm Alexa crab Roe the organizer of SF scala and here we are on location at ticketfly which actually recently moved to a new office with Chris forked senior software engineer at X dot AI out of New York Chris hi it's great to have you here it is a scholar it's great to be here it's actually the second meetup we do is ask Allah I think it's why with you last time we've done one on slick when you were a type-safe and slick was relatively new so you were explaining everything about slick so let me ask you first you know how did you get involved with scala what kind of attracted you to to it how you chose Lucas your project can you give us a little bit of your scholars history sure so I mean I was studying in Germany at this university called athen and at some point I had to write my master's thesis and I was looking around a little bit was like the idea of writing it externally and Scala seemed really interesting to me I was doing a lot of Python back then and I really was looking for something with a strong type system and functioning more functional programming than than Python and Scala it was so I got in touch with it with a lab and joined them from my thesis and one topic that I've been interested in for a while back then was the whole topic of object relational mapping and the problems that many people have with these frameworks and in the end the mismatch between this object oriented imperative world and the declarative sequel world seemed to be the clue the missing part like the complicated thing and basically doing something like this in Scala which is functionally using the functional subset of Scala allowed a much nicer correspondence much nicer mapping so that was interesting and epfl was interested in the topic at the time and type site shortly after with slick so yeah that's how we started and building building interesting so is you mentioned Python because now you know you're with X dot the i obviously using you know muscular and data mining and python is actually most widely used language in that sense committee so kind of a challenge which skull folks and spark people whose column face daily is a bunch of Python people asking why should i switch to scala and I wonder given the opp I thon background what would you answer to these people what do you usually say in this kind of situations how to explain why they decide to seduce collar versus Python well data scientist in particular I mean I'm not a data scientist but the reason why X today I chose scala was when one reason was that we wanted access to the Stanford and machine learning libraries and they are written in Java and we didn't want to go for java mm-hmm plus scala was more Python like which we expected the data scientists to know mm-hmm plus several people had some sort of experience with Scala so that's that's the language they chose before I joined right and in general why would you choose generally Scala of a python is really I've worked on like growing and larger Python apps and it's just hard to change your mind about decisions you made hard to and you make mistakes you have to refactor and big refactorings are painful and they require very very solid test suite and even then you have like holes and just the type system makes stuff like that much easier and also pushing functional programming further which we should we see with spark it's easier to distribute these things it's easier to reason about things in an immutable referential transparent way and yeah I think we're we're doing great on Scala cool though this is right here so how did you come from tools side look sleek is a tool right basically it's you know it's kind of a tool which are many people can use it kind of its general purpose or you know or or right of Scala powers but xai xai is a starter by the product right so you basically went kind of from the you know open source package you know contributor a type-safe 22 engineer at exit a I'm making kind of a product and using Scotland and compensating can you talk about how these things are different in obvious like open source community versus build a product and you know you probably cannot open source a bunch of stuff you do it exit the i right so can you talk about how these things are different how these two roles are different like why why I you know poor like it's more interesting to you at this point it's a when you're when you're working on libraries and especially when you're working inside of the small horse collar development team and ecosystem then you have to correspond a lot with the community to actually figure out are you building the right thing are you covering the right use cases and one thing that that interested me was being closer on the front lines to basically I am I am someone who likes to build tools who likes to build core components that accelerate the overall progress but very crucial is building the right thing that was that was my intention to kind of go closer into industry also the the grant which we had to to help you evolve slick was limited for a certain time from the yeah from from the Swiss government okay right and in fact there is a lot of tool in the building that needs to happen a text audio as well right we're over 20 developers now and so I I work for example on on our civilization library and we open source that all right and we are planning to open source more in general is just just extra day I was also such an interesting like focused company like we do one thing and we do this all the way and in AI is like I'm a computer guy I am I'm excited about a I write is excited right like this is her ex I is actually one of the key kind of appealing pieces of you know I think that the company so how did you meet them like you know what is most interesting about x for you how I met them was quite random like a friend of mine sent me a link he found somewhere i was almost signing at another company at the time but they just seem to they seem to have such a clear method a message so i got in touch with them i talked to them about how they're doing things what they're doing with what how they see things and all lined up so I I just joined them and they use skull before she live already made a decision yes they were they were committed to scala but they had one full-time skull engineer and one data scientist who knew some Scala right so they needed someone with experience to help basically build out the platform and scale it and scale it to like having 20 people work on it right so you have 20 people just working scala mel is a true we have we have like half half and like core engineers and data scientists with a little bit of Python at a little bit of node them but the majority of this stuff is is in Scala yeah and the data scientists right Scala code as well exciting so in the topic of your talk is FB essentially for III and I mean this is actually something we try to promote strive to promote in skala by the bay big data scholar as of scholar communities and you know folks in arbutus understand if p not many people actually do machine learning so on the other hand people could do machine learning very rarely even normal function program in this last or not know why this is important right so so typically the design is not a Python which is not much to do with front programming to begin with and even if you can try hard it's all right so it's on I'm very interested and you know I don't like you can I give it a go talk about this but just from kind of your silly boy this is in my mind it's very good and cool way to should do both things because you want composable abstractions and look valga rhythms can be kind of composed but this sense don't think this way right like the don't compose all right they just pick something off the shelf and check it together and kind of a split in so how did you especially like you mentioned some data scientists who work in scala how how do you see the data scientists can can they can they beat up this you know can you know I can they be even interested in this because some of them just want the graphs they want the results it look here how you get them right and they will carry stuff together right so how we convince somebody that composable things i do like what what is interesting about this for you is how do you try to promote is what we seen in the industry I mean an advantage for the data scientists to pick up something like functional programming is that they are very schooled in mathematics and they have a lot of analytical thinking skills and they're generally not afraid of math or of theoretical concepts and so they're not that's not usually scared about those things I find that many software engineers when they hear terms they haven't heard of before which comes somewhere from from research they kind of scared off a little bit but I didn't see that with the data scientists they obviously learned a lot of Python and and there's some transition period for the thinking and that's something we have to work on some people pick it up quicker some people need ya need some more time sometimes it's not that important when the components rather self-contained and in our case Marcus who runs the the data science team he himself he is very committed to two functional programming so I'm sure that helps as well we ran workshops explaining how important it is to work with mutable data structures and reason about things obviously we're we're in a learning process of the company nice so we should team up on this right so kind of you know my dream is that we have a library of abstractions for data science and algorithms machinery which is composable which is excellent you know functional programming style for a scholar year right and but that can lead to you know orthogonal design for algorithm so do you think you know is possible to cast a shoe library why don't get it now like do you think that you know multiple startups team up you can get this but what I thought so can a guy is beautiful library of compulsive obstructions I prayer I think so i think what needs to happen first is that we spread the knowledge about the fundamental building blocks and this is still to a degree lacking so my talk today is is actually most of these things can be applied to general software engineering but we happen to use them in our company just working with the data scientists and working with some of the existing tools they use i noticed that those tools are generally not build with composability and and immutability in mind and actually we have exactly the problems that result from not doing that when we try to use these things like like things don't multi-thread well or think like this so i think first kind of we need to establish the basic understanding and then yes I'm pretty convinced we can if you rebuild some of these these libraries in a more advanced engineered way all right now this is a great plan so I'll kind of rip up you know with this hope you know if anybody's interested in collaborating with X dot AI with my choice of Scala you know please you know fine kind of like minded community members come to class and let's make this library happen and we're looking forward to talk thanks for having me thanks