Scale By The Bay 2018: Seth Tisue, You Are a Scala Contributor
Recording: Scale By The Bay 2018: Seth Tisue, You Are a Scala Contributor
you hi my name is Seth and I'm on the Scala compiler team at light Bend we share responsibility for Scala with the Scala Centre in Switzerland and with Martin's research group I've been working on the team for about for about three years from Chicago originally and and live here now this talk is called you are a strong contributor if you aren't yet I assume you're here because you're thinking you might like to be you can be and I'm the theme today is to tell you and show you how being a solid contributor isn't a big deal so everyone starts small on this if you open an issue if you comment on an issue or or someone else's pull request if you make your own pull request even if it's just fixing a typo then you are a solid contributor I say everyone started small this is a little embarrassing but I'll show you my own modest beginnings on this this was in in 2008 I spotted a typo in an error message so I opened a ticket on it I wasn't a committer there was no github so it wasn't like I could just fix it myself so I hope to open the ticket and Martin fixed it the same day that was exciting it's a bit of a lesson for it's like what else can I make Martin do just by opening so that me it's a lesson for open-source maintainer z' i think that you know be nice to people who show up with these little contributions because there's a good chance that they'll stick around okay so it is 2018 now and contributing to open source is vastly easier than it was 10 years ago it's it's really the advent of github has has been like like night and day for for open source so in all of the scala venues that we offer for contributors your github login is your identity so not only for issues and pull requests but also for the scala chat rooms and forums we have a special chat room and a special forum especially for contributors so if you are working on a contribution or just interested in getting involved and have questions about about what to do and how to go about it these are the main two places to go for help get er that there's chat technology I mean maybe everyone knows that this by now it's a either web-based or you can get a desktop or mobile app a little newer and less established is we switched from Google Groups to discourse for both of our forums we have one for users one for contributors that's web-based but you can also choose to get the messages by email if you need help or have a question I've arranged the you know the three main ways of getting help in ascending order of how formal they are ascending order of how much effort that they require from you and also ascending order of how likely you are to get a response feelings are sort of divided on chat like a lot of people really love chat as a way of getting involved and other see the point but that you know that what when chat is good it's really good like you dawn there you asked your question and almost instantly you have an answer or you're having a conversation with with somebody if you go on the Scala contributors chat and don't get an answer right away or if a bunch of other stuff you know scrolls your question off the screen please you know don't get discouraged that that's normal and consider moving on either to the discussion forum or just go ahead and open a pull request I think different open-source communities are maybe a little different about this and the policy in Scala is that it is totally ok to open a pull request even if it's not done even if you're not even sure if what you're doing is a good idea it's totally fine open a pull request and you will definitely you will definitely get help getting it across the finish line particularly if you are explicit about where you're stuck and and what kind of help you could use so in a way I've jumped ahead a little bit to the point where you're already like submitting pull requests I want to back up a little bit first and help you get oriented in terms of what's even out there for you to consider working on where to find things where to open issues and so forth all of the official Scala stuff are almost all of it is in one of these three github organizations so the Scala organization is stuff that's that is shared responsibility between light Bend and and the other groups involved that's where the that that's where the main Scala repo is there are also important repos and under the Scala Center organization and then one extremely important repo under EPFL that's the name of martin's research group at at EPFL and lizanne and so that that's where Dottie is Dottie which will become Scala 3 before long under the Scala organization you'll find two repos for the Scala websites it's not always obvious when you go to the Scala website that there's actually two of them because the the designs are coordinated so you won't necessarily notice if you've left Scala Lane org and moved to Doc's Galileo org but as a contributor you'll need to be aware of how things are divided up the main repo is downloads the blog getting started information upcoming events it's a fairly small repo where everything is expected to be really high quality and and really up-to-date and then doc stuff Scala learning is where more in-depth technical documentation lives it's where the tour of Scala lives and it's just generally bigger and Messier there's more of a spectrum of things which are which are well maintained and things which are less well maintained here the Scala compiler itself and the standard library and the language specification are all together in the main Scala repo it's a bit surprising that there are no issues in that repo that came about for historical reasons we used to use JIRA for issue tracking and we wanted to import those those issues in the github without losing the numbers you know which which are surprisingly important as I as identifiers said that people actually know so the issue numbers overlapped with the polar quest numbers in the main repo we had to put it separate so you can go looking for bugs on install a bar there's no code there you may also see references to tickets in Scala dev which is another repo that has no code only issues its admittedly confusing for people sort of wide split up and and where to look for something the the the two main differences are one is that Scala dev is used primarily by the core team and and that and by people who who work with routinely who are like regulars and then the other difference is that Scala dev is really supposed to be about tracking current work in progress and Scala bug is like a historical record of issues that were found in published scholar releases there's a helpful feature on github which i think is maybe not as well-known as it should be which is organization wide search if you go onto github and go up in the search box one of the choices is search this organization so that's extremely helpful if if you're not sure where the the thing you're looking for might live you know is it an issue or a pull request already you're in two different repos so or worldwide search hugely useful also under the Scala organization are the Scala modules these are separate libraries which are you know kind of part of the standard library and kind of not right so that they're under the Scala organization so that they have some official status but they're separate jars and are separately maintained the group that I've listed on the top are primarily maintained by by us at ly pen and by the whole Scala effort although of course community participation is essential and welcome the bottom group are modules that have now been spun off far and our community maintained so we on those style teams sort of advise and and give input but there are community maintained errs who are making the decisions about releases and and doing the publishing and so forth there's a hope of slimming down the Scala standard library even further and making some more modules there's a ticket link there if you're interested in doing that kind of work so that that concludes the the survey a part of like what's out there and now I'm gonna circle back and talk about the contents of those repos in a little more detail and about what working on them looks like and I'll do it in ascending order of how easy these repos are to get started in so and the starting point for that is the Scala websites even if you're quite new to Scala even if you're still learning the language there are still opportunities to do website work where you're where you're valuable in fact having a kind of beginner perspective on Scala is actually really useful for for doing this work like you can see things that aren't clear that those of us who have been doing for Scala for years are just blind to because we're because we're so used to it there are plenty of open issues in in the in these repos some of them are technical so having to do with the site generator and with with CSS if you so if you have web skills that that's useful and there are opportunities there if you there are also plenty of tickets that are about content yeah either content that's poor quality or needs updating or ideas for for content that's missing so if you have some writing and editing skills that's needed as well so is translation in particular the tour of Scala is something that we try to offer in every language every language imaginable if you're interested in either updating existing translations of the tour or doing new ones that would be fantastic I think there are about a dozen currently working in the website repose the easiest way to do it if you're if you're fixing a typo or doing light edits you can just do it directly in the github web UI the pages on the doc site have a link to the page on github where the source for that page is stored if you're doing work that's a little more in-depth or if you need to preview the work locally before sending your pull request then these are the steps tolling the repo both of the websites are built using a static site generator called Jekyll which is pretty standard for github based projects I'd these two commands are they are basically the only thing that I know about Ruby but but this should be the it should be enough for you to build the site locally and get a live preview that will show your edits as you make them if you're looking at the issues keep an eye out for the labels on the issue in particular this labeled good first issue so in the last couple of years this is has become increasingly standard on github that open source projects that are looking for to make things easier for for people to get started well we'll label issues as a good first issue so we are using that label in in the website repos as well as as in the other repos that that I'm going to talk about okay so movie moving on now from the websites to these skal modules there's actually not a lot to say about the modules specifically which is which is good news so you know one of the reasons to spin them off from the standard library was exactly to make to make working on them easier so the Scala modules or ordinary libraries they're built with the usual tools that are common that are a near universal now for Scala open source projects they're built with SBT there's CI via Travis and the tests are usually with either Scala test stall check or both because these are separate modules they're not tied to the scholar release schedule new versions of the modules can be built and shipped at any time including including quickly if needed which is almost never the case with with Scala itself okay moving on now to the main scholar repo as I mentioned there's several different kinds of content in that Revo the language specification is there the Scala doc for the standard library is there you know of course mixed in with the actual code for the standard library and then the compiler lives there as well this is a this is a sample month of data from earlier this year that shows well I guess you'd have to know who these who these user icons are but I did this up and the majority of contributions were actually from outside contributors so it's you know that there's kind of a spectrum of programming languages from languages that have huge companies behind them funding large teams to work on them and there are their languages that are purely volunteer and open source Scala is really a lot closer to the volunteer open source end of that spectrum but you know light Bend is a small company and the Scala Center is fairly small as well so I showed this slide for two reasons you know what one is is to emphasize that you know Scala needs you if it was just us at like been doing this then you know Scala would would die and the other reason is just to show that contributing in this repo it has as has worked out really well for for for a lot of people and I would I would like to think and I actually do think that the activity has really picked up in the last two years or so I think moving to an SBT based build especially really made it a lot a lot easier for people to help out until about two years ago we were still building with ant which was just kind of a nightmare the other thing that that has really helped us the is the advent of Travis that's been another game-changer for open source in general in the last few years okay so if you're interested in the language specification that's a fairly specialized interest but it's an interest that that some people do have issues on the specification itself our install a bug there's a there's a label for that the kind of work on the spec that we would most love to have someone step up and do some of is merging accepted Scala improvement process proposals into the spec this is supposed to happen after a proposal has been accepted and implemented and and generally has not happened so that's a kind of a bite sized chunk of work on the spec that it's possible for people to do usually the proposals are expected to already be written in the same kind of technical prose that is used in the spec so it's not a big writing job it's it's more a matter of sort of taking that the cipa part and finding out where where to stick all the pieces of it the tools on the spec are marked down in Jekyll just as in the websites mathjax for formulas there's a separate readme if you're interested in spec work for the other kind of contents of the repo the library and compiler the top-level read Me's apply there's two of them for no reason there's there's really no rhyme or reason to like what's in one versus the other so you just have to read or at least skim both the easiest part of the standard library to work on is of course the dot comments in order to work on that you need to learn a little bit of Scala doc syntax this is general this is not something that that requires a lot of study basically everyone just just looks at all the existing examples and and imitates them and as we'll see on the next slide you didn't need to use SBT but you basically only need to know one SBT command which is the standard one after you've cloned the repo and made changes to the sources you run SBT and then run the doc command in the library sub project and that'll generate the HTML for you to view in your browser we generally don't have we don't have issues typically reported on missing or poor quality Scala doc so if you'd like to do Scala doc it's more up to you to identify things that that need work so I encourage you to as you're looking things up yourself in the standard library to kind of set aside a part of your brain to notice whether the documentation needs work for working on the code of the standard library again we're still in a zone where the level of difference from working on any Scala as any SBT based Scala open source project are pretty small the main area where this is unusual is in testing because because Scala is you know Scala itself we we can't really have dependencies within this repo we did make we can't have scholar based dependencies because those would be those would be circular we actually did make an one exception to that which is that we took on Scala check as a circular dependency and we're willing to pay the cost in that case because we think that generative testing property based testing is like so important to have if you're not doing property based testing then normal standard library tests should should just be written with je unit in the Scala repo there's a bespoke testing tool you unique to this one repo called par test if you're doing a standard library work you probably don't need to learn much about par test because you know any new tests that you write should not be part test but based but you may find yourself needing to update existing tests that were written in part test in like the old days when you're working in the standard library often performance is just as much of a concern as correctness if you're not sure whether your pull request needs benchmarking it's fine to assume that it doesn't and just submit it and then either ask or the see if it's requested if it is deemed performance sensitive there's a directory under the test directory called benchmarks it has its own readme and its own benchmarking harnesses so that if you do need to benchmark change that you've done it's actually it's actually rather easy so finally the compiler itself so you know reaching the we're now at the end of our ascent into subjects requiring you know more and more effort and ramp up time there is definitely not enough room in this talk for me to really to get into depth on compiler issues so I'll give you a few practical tips and then also point you towards for further learning resources I mentioned par test most of the compiler tests are power test based and it means you have a little bit of learning to do but it's actually primarily good news because because it's unique to this repo it's like heavily customized to be really convenient for doing compiler testing it's typically we run a subset of the test suite locally because the whole thing takes a long time to run so you sort of get halfway confident that your changes is good and then just send the polar question and let's see I do the do the dirty work for learning about compiler internals my number one recommendation not only for learning but even just for getting interested in the subject is this video by Jason Zod from a couple years ago called Scala C Survival Guide so this is it's it's not a lecture it's all jason in the repple right from the start showing you how to interactively explore compiler internals and and see what the compiler is is is doing with your code so it's it's all hands-on I think it's a it's a fascinating talk the other like must see or must read and in this case is Eugene Vermont Co Scala hacker guide this is unfortunately starting to get kind of out-of-date so that's our so updating it it's actually something that we could use some help with as well this walks you through the entire process of making a change to the compiler from from from beginning to end it's it's very detailed and well done you'll want to learn what phases exist in the compiler there's a flag that will print out a little reference card of that and my single favorite you know compiler thing for playing and exploring is this X X print all flag you can run any piece of scholar code you want through this and it will show how the code looks at every stage of its transformation as it goes through each phase of the compiler it's really awesome I strongly suggest using IntelliJ for compiler work even if you're an IDE skeptic and in a debugger skeptic as I am myself everyone who does the heavy work on the compiler uses uses IntelliJ and considers it essential to their productivity it IntelliJ works fine on the both the compiler and the and the standard library even if you don't want to edit or build or run your code with IntelliJ you can also attach the IntelliJ IDEA butter to any running JVM that you've started with the debugger Flags enabled that's incredibly useful for troubleshooting compiler stuff if you're looking for an easy way to get started working on errors and warnings is a really good getting started task also other kinds of work anything you do with scholar reflection any macros that you write if you if you write a compiler plug-in I have a separate talk about how to do that the API is that you're using for all of those tasks are the exact same AP is that you use in compiler works so that it is compiler work it is training you to to do compiler changes so far I've talked only about Scala - I'm shortly some of you have been wondering have been wondering about working on Scala three I'll start by reminding you that the theme for Scala 2:14 which will be starting work on soon now that that 2:13 is nearing completion the the steeped the theme for Scala 214 is to align with dotty and to prepare for Scala 3 so by the time Scala 214 is done Scala 214 install 3 will have a lot in common they already have the same standard library and that will that will continue they already share the final phase of the compiler the byte code generator is shared the work on tasty will enable style 2 & 3 to share the same library ecosystem and what we're also in the on the compiler part we're looking to make more of the later phases in the in the compiler be shared the ideal goal would be that really the the only difference between the two compilers is that would be the type checker at that point we would admit tasty and the dotty back-end would take over that's an ambitious goal I don't know if we'll get there for Scala 2:14 but we can definitely at least get a lot of the way there so if you so work on Scala 2 is work on Scala 3 if you're working on style - you're helping Scala Cerises helping Scala 3 happen nonetheless III definitely wouldn't want to discourage you from working on script from making the jump now and working on Scala 3 directly I don't have a lot of specific advice about that for you because that's it's something that I haven't done myself but here are links for where the code is where the issues are and where that where the chatroom is the dotty project is heavily github based so they don't use discourse much the issues tracker and pull requests are like this crazy mix of you know I fixed the typo all the way up to like I'd you know here is this major new feature for Scala 3 what do you think it's it's it's a tough repo to follow because there's so much sort of small noise concealing the like really big things you want to read but worth following worth following anyway okay in a in a longer version of this talk I'd go on and talk about deciding what to work on and how to stay motivated so my talk slot here is a little shorter I don't really have time to go into that too much but because I want to leave a few minutes for questions but you know you can find these slides online afterwards and I have I have lots of advice about decision process on where to work on I have specific suggestions on things that Scala needs help with and a lot of advice about how to stay motivated I'll just I'll pick just one bullet to show from the how to stay motivated which is the the most important thing is is to understand that open source work is slow so it's going to you know occasionally you have the straight experience in chat where you didn't answer right away but it's more common that the the the path that your pull request will take to being merged will have you know multiple delays that lasts that can last days so you know fire up more than one thing so so you're never feeling stuck when when one is delayed thank you for listening are there any questions somebody [Music] how much okay so the question is in order to do compiler work uh how how much education and compilers do you need and how much does it matter how recent that that it education is if you want to work on the type checker then you you it would be best to you know be one of Martin's PhD students the rest of the compiler is is just not that hard so parsing while parsing is not that hard but it's also not like an area of activity the the later phases of the compiler are complicated but some of them most of the challenges that you will face are simply challenges of learning this specific code base rather than challenges of like I don't even know what these concepts are or you know I should have learned this in school I mean you it would help to have done maybe a quarter or a semester of compilers as an undergrad but I would say education beyond that level is it's not that important and the how recent that education is is is not important at all time for one more question but okay the question is when is lunch lunch lunch is now thanks [Applause]