SF Scala: Q&A with Jon Pretty by Alexy Khrabrov
Recording: SF Scala: Q&A with Jon Pretty by Alexy Khrabrov
hello everybody i'm alexis kraborf the organizer of sf scala and here with us we have a very special guest john priti who is one of the most experienced scholar veterans uh he uses scala since 2004 and in paul phillips's famous quote it took him to 10 years to gain 10 years knowledge of scala which is most impressive uh and a very very paul did add that it took him i think about two years to get the same amount of experience right we're going to ask him next time when he will apologize that's that's great to have you with us we have a uh uh job previous is talking today about uh rapture and we're gonna do a workshop and follow along uh but first i will welcome you here thank you it's great to have you with us here thank you for the kind introduction and uh obviously uh it's not very often we meet somebody who did so much scholar and uh and some from so early so maybe you can tell us how did you discover scala what interested you in it and what kept you in it for for all this time so i i discovered it back in summer 2004 and i was i was setting up a business with a with a friend of mine we wanted to develop our own language we had aspirations that were completely infeasible but we wanted to create our own language and as part of as part of the the research you ought to do before embarking on a large project like that we we sort of looked at what was out there already and i stumbled across this website from uh epfl mm-hmm and uh it was it was my my business partner who was the brains behind it so i i i said to him have a look at this this this sounds like how do you stumble how do you stumble upon something like fpfl.c i i googled google i think google was was there yeah okay 2004 yeah yeah and uh yeah i found this found this language okay i can't remember what did you google for like interesting languages i must have searched for functional functional and presumably object-oriented i i can't remember maybe it was just programming languages i went through a list of 50 well 50 pages worth of of all the languages out there and somewhere somewhere there was scala i'm sure it wasn't on the first page because it was it was only uh like three or four months after it was first announced interesting but uh we tried it out we we compared it to a language that was um it was around at the time called nice nice i have no idea what happened tonight but we i remember it we we did some tests and the choice was between nice and scala okay and we went with scala and uh so skull was not nice scala was not nice it wasn't nasty most of the time but uh yeah we were then building the the the business plan remember this is this is this is two guys fresh out of university without any real world experience our our plans were to to develop a language or or some technology that was that interested us funding that through some consultancy work now as as a as a startup um without an experience what we ended up doing was spending all our time working for no money on on a few web projects sounds like that's that's san francisco startups maybe we said there's no investment involved this this is this is england right yeah we uh not happens not very common thing here either right right yeah so so we were building websites in scala we we ended up having the first what what could be called the first commercial piece of software written in scala and i mean the experience working with it back then uh this this was scala 1.3 it was um it was very pleasant most of the time and then occasionally we'd stumble across some bug that absolutely through everything the compiler would fail to compile it would crash for no reason i mean it wasn't like we suddenly started using some new broken feature it was like we'd be doing exactly the same thing we'd been doing for the the first eighty percent of the project and then one line would would cause it to crash so there were a few occasions like that that were incredibly frustrating so it's hard to even imagine right like the live before the big bang what uh what was there in 1.3 and what most common things were missing so the thing that i use most now that was missing then was implicit values okay implicit conversions existed okay you could you could automatically convert from one type to another if you put a value so that was there from the beginning that was there from from the very beginning uh i tried to have higher kind of types were there in some form i think they were there in a limited form um let me see the the it's interesting some of the syntax changes that were different like cell self types were there except they had a different syntax so you'd put a colon after the the class constructor and then the type the self type um which i quite liked as syntax and then later on that changed to the requires keyword requires was a keyword for a while and then we got the current syntax which i always thought looks too much like a lambda when it's just not a lambda so i'm not a big fan of that how but the collections collections well they were they were far worse than they are now i mean i know the collections get a lot of criticism uh from from certain quarters what about the api was they like the function names and methods the method names were generally speaking the same as they are now except they were implemented only on certain certain types of collection certain types of uh thing like so so um you might find that list has a sliding method but uh actually vector didn't exist but sequence possibly didn't or something i can't remember it didn't feel intuitive because you know one thing which struck me when i started doing scholars is how intuitive the collection methods are and i think personally that one of the reasons of uh adoption of scaling also spark i think really benefits from this that that is one of the best things about the collections library uh the the consistency or at least the the attempts at consistency people will find places where they're not consistent and that was there in in the early days too there there was there were aspirations to have it consistent but there were many more cases where things weren't consistent mainly but i think by virtue of the implementation being less type class okay and well in fact less inheritance based so yeah so certain certain things were sporadically available on certain collections just it wasn't consistent so how did you uh kind of uh i would say crystallize this color because you wear in one of the initial seas of the scholar community right so how did you got in touch with the pfl and where did you see community grow from there right so the first i guess the first couple of years myself and jamie webb my business partner had i think i had i had no um no sort of face-to-face communication with anyone from epfl i remember jamie who was more active than me on the mailing list back then uh had a meeting with with martin when he happened to be over in the uk and they they chatted about some stuff okay and uh we had occasional meetups in london though there were a few of us these are the the the the early london scarlet user groups meetings do they call them user groups did they call them it was called now it's a user group i think didn't didn't the word meetup sort of evolved from from meet up the company i i assumed it was it was called the user group and we'd meet in the pub and there would be about i don't know 10 of us at most and there'd be a guy there who who said well i'm i'm i'm using java i'd really love to use scala you didn't project any code on screen no no no there are no ipads either right no ipads so so many things didn't exist back then all right so you couldn't but the laptops were there so you could show some code to people laptops laptops were bigger and heavier right right i think it was rare we actually took our laptops out and looked at code i think i think we were mainly just chatting about things we could and couldn't do and all the all our favorite bugs so you were actually kind of uh industry users right so if you follow this academic organization this mysterious place from which skull emanated and you guys took it and used it in industry yeah we were industry users but nobody none of our clients knew we were using scarlet ah so we we had a few sales meetings where we said yeah we're using this really cool language scala and i'd say call called what what what what is this so they want to build their sites to build their sites to auto build the back end back and stuff and uh they they've never heard of scala obviously and uh if if they if they did then go and look look on google for for information about scala the fact that was so little was it was a good sign to them that actually it was uh wasn't wasn't a good long-term prospect and that they would have had maintenance headaches and they're completely right in that decision so we just told them it was java so we said we're building your software in java runs on the jvm it's java smart choice so that that yeah that's what we said and uh we we i think we were only really successful in in in sales when we didn't tell people it was scala and it wasn't later until about 20 2011 i think so after after scholar solutions which became type safe it was founded i think it was only then that scala actually became a positive rather than a big red flag a big risk for anyone i think it still happens we see echoes of this yet still some companies using clojure scala yeah they like to have uh java apis and they kind of like to although they like to have php i mean if if you're judging based on on how widely it's used people yeah people will uh people make bad decisions so basically i'm following the trajectory so you are doing this in industry you have occasional meetings with visitors from a pfl right when when do you see the momentum picking up when do you see this really changing the trajectory well the big thing was twitter i think in in the last uh 11 years the the single the one single thing which has made the biggest difference scala has been the adoption by twitter and their success and and the fact they managed to reduce their downtime quite significantly at the same time as they were expanding very rapidly and that was about 2008 2009 2008 2009 and you were in europe i mean i was in europe so how did you see i mean we live in the bubble world here right like we kind of know well i was in a bubble within europe called britain that's right but the the the ripples from here are spreading slowly sometimes the rich midwest in a few years right in the east coast and so i was i was really i was really pleased to see joe barnes from alabama so scala has reached alabama that's right and it's been there for a long time yeah but uh so i'm curious how the twitter uh phenomenon uh reached uk and like how did you see this skull explosion affecting it it wasn't it wasn't so much that the average developer even knew that twitter was running scala i think it was more a matter of the the tech leaders who whose job it is to be aware of what's going on in the industry became aware that twitter had used this that they'd taken a risk on this this language right and had been successful with it right and then maybe they should consider that as well right so it was it was i mean suddenly not all decision makers were aware of it and certainly many quite reasonably i think still back in 2008 saw it as a risk but um the fact that twitter had been successful uh with scala i think that that was an enabler right and it enabled the the formation of right and you know it was called skull solutions it was cool solutions i i did a couple of weeks of work uh for them i i i did some consultancy back when they were setting up i'm so glad they called it type safe the skull solution sounds like uh well there was it was it was actually type safe was formed from two companies the scala solutions and scalable solutions which was jonas bonaire's company and uh you'd have thought you could find a sort of a common name between scala solutions and scale right solutions i think it's subsumption but yes but they went with types i think i think for the best yes absolutely that was a great name so and did you stay with scala all that time yeah so i haven't written any java code since about 2004 maybe 2005 i wrote some mm-hmm so were you ever tempted or kind of uh thinking over whether you should do something else i it's probably best described as blissful ignorance i i've i'm aware there's a language called haskell yes another one called idris yes they sound interesting but the the the free time i have doesn't really allow and i think they're both made in the uk now uh yes yeah yeah so they're local they they are local yeah i'm well i'm actually living in uh in cambridge in england at the moment which is uh where there's a lot of a lot of research into right the microsoft research microsoft research yes also f sharp is quite big there but as we know from scholar days you know scala is a gateway drug to hospital and husker later is is the future as we've seen yes i'm looking forward to that so uh so uh so this is very interesting so uh i mean can you tell us a little bit about uh kind of uh running business with scala for all this year so obviously especially through early days right it's it's it's a non-trivial uh proposition to to work in the industry in this new language and and probably the the nature of the business evolved can you talk a little bit about the the the early dates i i um we weren't successful we uh we we wrote a lot of scala code we we were i mean one thing that did draw us to scala was the fact that we could use java libraries from scala right and it was very convenient being able to take existing like uh apis and libraries off the shelf and use them within scala but i did find a huge amount of my time was spent writing wrappers around those libraries and i guess you could even say that that's the uh the sort of inkling of where rapture came from it's it's certainly it certainly taught me a bad lesson i would say that uh that you should reinvent everything and i because because scala didn't have anything back then there were no scholar libraries it was always very tempting just to rebuild stuff reinvent the wheel over and over again but make it better yes and i'm not advocating this as a solution to anything it's i still find it harder today to to pick up somebody else's library rather than just create my own interesting and uh i mean yeah i mean it's a it's a it's a flaw that's developed from having worked with the language from the right i think i see now because every programmer has this little set of code they drag along with them all the time right they develop their own methods and their own utilities so i think it sounds like rapture is your bag of tools which you have taken with you along different projects so i i can trace i can trace code in rapture maybe one or two methods back to 2005. uh jamie webb wrote wrote those methods um the ones i'm thinking of stream uh data synchronously from one from an input stream to an output stream or equivalent that that is the code that's lasted with a few a few updates it's lasted over 10 years well no uh about 10 years i would say what uh source control system did you use oh we we used from the very beginning we used darks okay yeah it's a great it's a great system with a theory of patches the theory of patches which uh didn't work for uh for the first couple of versions of darks mm-hmm at least i i was i was never sure whether the theory of patches was wrong or whether my my mental model of theory of patches was wrong but we persevered we had occasions where where darks would get tied in knots and it would be just left spinning for ages and ages and and might ultimately complete the operation we asked it to do and we might get our code back so you could you could haskell code lovingly nurture this budding scholar yes source yes have you ever been tempted to look at this haskell source of darks when it was buggy no i mean i don't i don't read haskell yeah did you build it from source um we we would have done i'll tell you why because i've been using gen 2 linux for the last 10 years so yeah so i mean i didn't i didn't type the commands except the the standard installation right on gen 2 but yeah it was built from source interesting interesting you still have this dark repository i i probably have them somewhere yeah you should you should contribute to the computer science museum right because this is this is one of the longest existing repositories in scala you should make sure that it's preserved at least you know she's all important to github right and then it will be a matter of public record forever yeah and you know future i continue with darks until um probably about 2011. i use it around the same time right so i think the scent of git kind of switched everybody to git yes yeah um dark starch 2.0 was a big improvement that that made it usable really yeah well we have some of the sage issues and yeah but we were early adopters with with darks again and darks didn't go anywhere really right scarlett right that's right some that's right that's right uh so can you kind of highlight you know some kind of the most interesting things you've done with scholar during these years right okay the most interesting things i've done um well my talk at scholar days i thought was very interesting i was quite excited by that so i'm i've found a way to to provide checked exceptions in scala i probably don't have long enough to talk about the details but java's java's had checked exceptions for ages and and it always seemed to me like this was a point where where java had better safety i mean at a cost of like horrible boilerplate code and and busy work uh coercing your your your exceptions into the right mold but it did have this additional safety checking that scarlet never had and the reason scarlet never had it was that it you needed to be able to accurately track the exceptions that could be thrown so if you if you map across uh if you map a lambda across a collection say that lambda may throw some it might throw i don't know uh what's an exception some some exception exception but uh the the map method you can't write the signature in such a way that you can derive the uh the exception type right dependent on the lambda that's not possible in in java but i've found a way of doing it in in scala which is involves mixing together like three or four quite advanced features but uh i've got working prototypes so i'm confident that i'll get something working interesting interesting yeah uh i think it will be really interesting uh uh to look at that it i i would yeah i would hope that within a month or so i'll have something that people can actually use um so you so you you give a rapture talk at um scalability conference last year which was inaugural scholar by the bay and i think it was a great success because it was one-liners each of them kind of self-contained showing some of the uh beautiful syntax and ease of use and i hope we get to do something today alongside you know following you in the rebel uh but uh i wonder uh how do you came up with this idea and how do you see people reacting to that and how much adoption do you see happening around the world when you present this uh this library so so people people do seem to like the one-liners talk because i've got about 50 slides each one has one line or maybe maybe two lines of code and because i've designed rapture to be as intuitive as possible you can just read that line and know what it does and i've managed to do it in most cases whilst remaining completely type safe taking advantage of quite advanced features of scala at least at the library level whilst not requiring that the people using the library need to understand them so for example going back to my my talk at scala days yesterday was it yesterday yeah it was yesterday it seemed seems so yeah seems so long ago so much has happened since uh i i introduced the concept of modes now modes allow you to pick and choose how you want your exceptions to be handled so do you want do you want to return an option do you want to return a try are you happy with just an exception being thrown maybe you want to return a future and execute the code on the on on a fork join pool these are all possibilities with mode and you just import the appropriate mode and methods that have been written in the in the right way which is basically anything in in rapture will actually change their return type based on the fact the import is there so this this is a really complicated thing or you need to understand a few complicated things to implement it the implementation is simple but the the concepts are i'd say quite advanced but users end users don't see any of that it's a lot of i think it's a common feature on skull right there it's it's a couple seconds it's a common feature simple once you understand what you're doing yeah and occasionally occasionally it leaks occasionally that abstraction leaks uh but with modes it's been it's been holding up pretty well and you don't see any of the underlying high kind of types or um implicits or dependent method types which are involved in the implementation all you see is well my my method return type has magically changed and and it was easy and not just this so i think this is a theme on rapture right like by varying the imports you can make very different behaviors and i think this as i'm not just a or when you mentioned this yes right and i think this is very interesting so uh uh i wonder actually that that sounds like a very good modular way to alter the behavior uh so can it be for instance combined with the build system to produce different behaviors in different contexts right you can have something on staging something in production by varying the imports you will change the behavior of the library and the the underlying call so you'd have to well you you'd make one change to the code work yeah definitely yeah so you can have method return types that are dependent on the import that's how modes work yep um and yeah you can do different completely different things run different code dependent on import that that's that's certainly something you could do whether you want to have that configuration as part of the source or as a compiler flag or as an environmental variable or something i wouldn't recommend environment variables i i try to make the source code usually a specification for how the software will run that that's right that's i think generally preferably right so you can drop but you're dropping this symbol so you can use files right but i mean there are issues with that for example you can't change that behavior at runtime then so so right there are different approaches but i'm thinking about things like you know typical use of a cake pattern would be to replace a relish base with a mock right so it sounds like you can achieve a lot of similar things with rupture by changing the boards yeah the imports do give you a lot of flexibility and i've i've gone through using the cake pattern um i i used that for years and years i i was reasonably successful with it i think uh i mean i came up with a name i i i decided one day that it was uh well i i'd actually uh uh eaten the cake that day oh on the mailing list uh which which i answered which to which my answer was where you need to use this pattern here and it didn't have a name and i thought can we can we like find a better way of describing this and one of my colleagues had his birthday that day and uh we'd had a cake for him so i sort of the cake was there on on the table i thought well it's kind of you you're slicing every trait is a slice of that cake and then you've got layers within the cake which sort of represents stacked or nested yes traits and the the compiler neatly checks all these slices fit together perfectly and uh it seemed to fit people liked it and uh that's a great name it has a lot of illusions right let them eat cake to have the cake and eat it too well yeah i'm not sure you can have your cake but you can it does introduce uh many problems if you're not getting and there are of course a lot of uh deep discussions right i think it's a very common thread in the scholar forums you know are what are the you know how the cake pattern works you know what are the benefits should we use cake powder and there is sometimes there are revolutions against the cake pattern right yeah i i would say if you if you set out with the cake pattern in mind as your as your goal you'll probably approach the problem the wrong way i think if you find if you find you've written some code that ends up looking cake-like maybe that's maybe that's legitimate but uh i don't know i've i've gone off the cake pattern a little bit not as much as most people i think i think daniel spiwak still still believes it's a it's a good thing but we're sort of the last uh the last couple of people who are still sort of using it so what do you use nowadays when instead of the cake part um type classes uh i think i mean i mean to say that type classes are a drop in replacement for for cake is is maybe misleading but normally if i need uh normally if i need to to do something that that's not sort of obvious in in in sort of the the java subset of syntax that scala has i i would reach for implicits and type classes okay so i think there is an endless amount of things we could discuss right and probably i'll definitely follow we'll follow up next time we'll have you but you know i will probably close with your uh given you know like uh one of the person's most experienced right and i will compare it with the first 10 you know first 50 years of computing when acm celebrated the first 50 years of computing and i was uh at penn at the time where antioch was built so this was on the celebration sites they made it a hundred years of capital celebration because they celebrated the next 50 years and the first right so so you've seen first initial scala uh we're gonna probably meet here again and talk uh about 20 years of scholarship in 2024 2025. that's right that's right so what do you want this next 10 years to be what do you think they're going to be uh where would you like it to go oh more of the same more yeah that's great that's it sounds great to me all right here's tomorrow the same thank you thank you