A better Scala REPL?

SBTB 2015: Li Haoyi, A Better Scala REPL?

Recording: SBTB 2015: Li Haoyi, A Better Scala REPL?

cool so let's get started um for those of you who looked at pamphlet the title of the pro of the talk was about Scala text typ save programmable documents so that was out and for those of of you who who looked at the website the title of the talk became a better Scala rapple question Mark and since then it has since changed again so currently the title of the talk is beyond bash shell scripting in a typed object oriented language so if any of you who have a laptop so want to follow along the slides are at the URL url.com Beyond bash so feel free to go and check it out get ahead of me refer to slides I've already passed but to begin I'm hoe I get paid money to work at the developer tools team and Dropbox so continuous integration senium development environments all that stuff I don't get paid money to work on Scola JS and I still work on it anyway I've been using Scala JS professionally I've never actually used scal Scala professionally so compared to many other people here I'm technically an amateur um but today's talk is not going to be about skyar JS nor is it going to be about developer tools at Dropbox so today's talk with where the agenda is more or less spelled out here is meant to answer the following question how do we can we stop using the world's worst language to build our most important infrastructure so what does this mean um if you look at typical application for example you may have a client you may have servers multiple clients even if you're actually making money and you may have a database that all your clients and servers talk to for the people in this room the server may be written in Scala for many people in this room the clients might be some HTML JavaScript web scale standards compliance thing and the problem comes when you actually want to make changes to your system so maybe your server is written scalas using Scala Z very safe reactive back pressure all these nice things and that's great but when a time comes and maybe your database is fine so maybe it's mongod DB maybe it's postgress some are more safe than others but everyone's safe kind of when the time comes that you want to work on the front end code Suddenly It's Not So safe anymore so your your JavaScript May block with undefined is not a function your CSS may have one rule that makes a whole web page disappear and become unviewable or your Ajax calls which are over here have nothing stopping them from resulting in four Falls Vie to typos and URLs or if that was the case now it's scolar JS the front end is has no reason to be any less type safe than the back end and even the Ajax layer between the client and server if you're using the correct tools you cannot have a fall4 because the compiler will enforce that the Ajax requests are well formed so scal JS works is great check if you haven't checked it out you probably should um and there's some things you learn from scolar JS for example casting is great like many people here may not like casting but in Scala JS you do this sort of thing a lot and realistically in JavaScript every single expression in your source file is effectively a cast so having one or two casts in a large Scala GSF vation is like no big deal really compared to having every single line possibly blowing up due to a type error similarly weird unsound behavior is fine unless it's as long as it's less weird and less un sound in JavaScript and best effort error handling is great because JavaScript does not even make an effort in trying to get you good errors so what have we learned from scholar JS what we've learned from scholar JS is that when is that even though it may be bad in some cases as long as it's better than the worst thing that's great so you don't need to be perfect you just need to be better than what's out there and in this case JavaScript is not hard to beat and being not bad so so our application looks like this and does that mean you're done does that mean that we've never you're never going to get runtime type error ever again does that mean that all our code is written beautiful reactive high performance Enterprise grade Scala um probably not because for many applications underneath these beautiful architectural diagrams you have this kind of spiky uneven painful underpinning of Automation and deployment scripts for example at Dropbox our our developer tools team has a mix of bash python puppet Ruby and vagrant file to manage often calling into each other recursively so like bash will call python call bash call python call vagan call python again and generally it's not that fun so this is dangerous and so you have a high performance type safe server application you have now with SC can have a high performance and type save web frontend but it's underpinned by this Mex of stuff there's no there's no real description of it but this stuff is really hard to test and because setting up a ec2 cluster and making sure everything's confed correctly is really much harder to test than a function which takes a case class and returns another case class it is possible to test setting up ec2 cluster but because it's hard to test people tend to do it less and therefore it ends up being more fragile all this stuff is not type checked so there's this huge debate about oh should you have types should you have tests maybe we should have both because we are Enterprise grade but all the really important really important infrastructure has neither types nor tests in many of these cases and that's not great and worst of all the consequences of this infrastructure screwing up is pretty bad so if you have your application everything's Rosy I make a bad application code change I push it web server goes down all application server goes down doesn't matter I fix it back out the chains push again deploy again server comes back up so that may be a problem but usually it's not a terrible problem what's worse is when the deployment and cluster goes down and suddenly it's not so simple as fixing it and redeploying because those things that your me that's meant to help you deploy is all broken and naturally any of your clients trying to reach the front end don't work or if a database automation goes down instead of taking down one database server for a colonel upgrade you take down all your database servers and you find out suddenly you have no more customer data so that's not great and it does happen in reality so a lot of this centers around bash so what about bash well even though we have python Ruby vagrant puppet all these all these other things the core of these systems often revolves around bash grips calling other bash grips and there are many things which are wrong with bash so the syntax is funky then so I don't know which of you know why this is wrong wrong and you know why this is wrong okay so about five about 10% of you know why this is wrong so other 90% will have it blow up everything is global everything's a string and as a result even if statements are hard enough that 90% of this room cannot get them right so if for example I want to run a script on all files with some extension this is from stack Overflow one example solution second example solution Maybe third example solution from stack Overflow so this is the URL if anyone wants to grab for it I'm not that making this up the first one is wrong for subtle reasons that arise from treating paths and and F find output as a string because I'm splitting on new lines but file file names can have new lines in some systems so that's not right the second one has a bunch of weird glyphs that I don't fully understand but it's apparently a good solution so that's the best solution we can find on sack overflow and for the third one this was not on sack overflow but from my experimentation removing these weird these two weird glyph seems to work and I tried it out on Ubuntu I tried it out on OSX and like okay seems to do what I want and it seems to work it's often the most you're ever going to get with bash it's not even like it should work because of certain because that's what the documentation says or it should work because I've used it before but I tried it it seems to work I don't know so why do people use bash can we use something else as a replacement for bash so in order to do in order to try this let's think of simple use case I want to list things in the folder I'm in I want to look at my current G directory I to make a folder put a file inside and delete the folder so this is completely synthetic but it's a kind of indication of things that you do at the command line so let's try it out over here I have three shells top left is Bash bottom left is Python and on the right side is Scala let me just make this smaller so I'm in a demo repo if on the list files in batch I just LS I can get Branch to check out what branch I'm in um I want to make a folder I can do that file exist okay let's remove the folder first and let's make the folder again um folders there Echo hello put it in folder file.txt and I can list folder see it's there and I can cat folder file.txt and it worked let's try doing that in Python from the python shell because python is a scripting language and I'm going to do some scripting now so import OS os. list dots is how you do this in Python why did that just turn up I don't want that okay so okay so that works um I need to call G so I guess I need to import subprocess and sub process. check call PR bracket code get code close uh Branch so that's okay it's not great um what else do I want to do I wanted to um make a folder with a file so I os. M there folder um I guess already exist so let's say RM RF the folder here and let's say I make it here so now it works um I can list it again you can see the folder now exists and what else can I was I meant to do I meant to make a file so I open folder SL file.txt dot I think I need to say it's in right mode do WR hello and if I read it you see the data is there and if I remove it it blows up so I actually need import UIL shil do RM Tre folder and now if I uh list there it's gone so that was kind of funky so let's try doing this in Scala so import Java IO file new file. list nope wrong so okay it's the folder is gone I need to um new well what was I meant to do I was meant to list well I probably listed so import do process and then seek G branch. bank so this not that much worse in Python um new file folder make there so that seems to have worked um and then I need to write to a file how do you write to a file in Scala how to write to file in SC so I I I am doing this but I I would bet money that most of you would do the same thing so this is how we reite to fence SC this one of the menu ways so let's stick that in um folder file.txt got get bites okay so new file folder. list um okay it seems to have worked how do I delete a folder in Scala so a new file folder. delete and then and list and it's still there okay so how do I delete a delete folder in in Scala I mean spell in Google should know no Google think it's inell i n so I need to delete the directory recursively I could use Scala I Scala IO but apparently that's dead for the last four years so let's not use that here's something that's not dead it's simple delete file and I want to delete that and if I new file dot do list it's gone okay so that probably answers the question of why do people use bash can we use something else and the answer to that question is probably no you cannot use something else so even like bash is okay python is somewhat clunkier doing it in Scala is a huge pain so why is that bash is better because you takes 24 characters rather than 280 characters to delete the folder um even in Python you're looking at a 2X or a 6X increase in verocity and that's too much because we not typing things I really only care about these important bits and the rest is just trash and when you're doing it interactively how much you type really does matter go away you manage software updates Okay so bash is better because there's less syntactic ceremony bash is better because the common operations are short I just type LS instead of new file dot do list bash is better because of few keystrokes bash is better because the commands do what I want and removing a folder with a crack Flags kills the folder regardless which is usually what I want I almost never want it to kill the folder oops but there's something inside I shall not kill it and not tell you and I think the last two are pretty important important so amonite Ops is a library that is meant to provide these things these features in Scala the interesting thing is that none of these features are theoretically interesting none of them should require PhD thesis to implement I can make my commands take few your Cs and I can make them do what I want so I try to do that so mnet Ops the goals are basically have a library which does more less what I want as a goal say less than two no more than twox is veros is Bash so python is 2 to 6X Java is maybe 10x Scala same same as Java let's aim as 2X and let's say that it should be safer than working with python or java

iio or nio because if it's more dangerous and as dangerous as bash what's the point of this as non- goals when I'm not aiming for monatic Pure dependently type safety here so if you guys have a problem that you're writing too much haskill that's great this is not the library for you this is meant to replace bash and similarly is not looking for reactive manifestor creditation everything is synchronous everything is blocking everything is heavyweight and it's not aiming for a 50-year Enterprise maintainability so Mite Ops lets you write code like this so instead of G status I have percentage as a way of shelling out so percentage gate and I have to quote the status because it's a literal so here I use a symbol to save some save one character instead of RM folder file my file. jpeg I RM bang folder my file. jpeg with some quotes slightly more of a bothast but overall not overwhelmingly so and I can show it to you in action so if I import amonite doops import implicit working directory so I you knows what directory I'm in I can shout out to git I can do git status git's kind of dirty get Branch LS um make their like CWD folder LS so it's kind of hard to read but somewhere in here you'll see demo SL folder has been created and I can RM cw/ folder and it's gone so that's the taste of what Amite Ops gives you um it gives you short commands that mirror what bash does after single import these commands do what you want there's no fussing about is is it a removing of folders removing of f removing a fold of children it removes it like what you told it to remove it it removes whatever it can find there at least or at least it tries to and there's no ambiguity in passing Arguments for example whether or not CWD has a space the current working direct of space in it should not randomly cause it to think it's two sets of arguments instead of one and because all of this is in normal Scala it gets passed and interpreted correctly regardless of how the arguments look so one interesting thing about Aon knite Ops is that all your commands return structured data so for example LS of your current working directory returns a sequence of paths which you can iterate over using for each and the paths themselves are structured data so you can use they actually have attributes like the extension is an attribute you don't need don't need to munch with reg Xs and other things you just say path of X check if it's Temp and do things to it so bash supports piping and if you've already there's no reason reason why Amite Ops should not support piping as well if the goal is to make something that similar to bash so that you have some hope of replacing it so Amite Ops provides a bunch of basically aliasas for methods on common types for example they bunch of traversible aliases that look like pipes and basically do traversible things you have for this forward pipe they call often call It On Any Which calls a basically reverses it call a function and you have bang which just calls a function and saves you fors and does nothing else so this will not pass your fifth year Enterprise accredit accreditation test but it does pass to make things short so that people don't mind typing at the at the rapple test and that's the only purpose that these things solve so putting it together mni Ops gives you concise file system operations so listing is literally six characters or seven characters of the space rather than a whole bunch you get structured and concise path operations which I didn't really go in detail but you combine paths using slash and symbols and strings for the path segments rather than trying to splice strings together using plus so you don't need to worry about oh when I combine it do I need to have a add a slash in between the two paths or not you just use slash and it combines in the correct way and it gives you pipes as alist for call collection methods and that's where things get interesting for example they want to find the total size in bites all of in characters of all the Scala files I think this is in bytes the total size of in bytes of all the Scala files I simply list everything find filter the this is Al of filter so filter the ones with extension Scala map them to the size and Summit and we can see it in action here so I am in this folder let's look at a more interesting folder so let's do a recursive listing of the current working directory so that gives a bunch more stuff filter x equals scalar pipe do size sum so this is something that would take minutes to write if you are doing it using normal Scala file system apis and maybe that's fine if you're committing code to some long-term project but if I just want to get this answer and get on with my life minutes is unacceptable especially when the alter is I can do it in 5 seconds or 10 seconds so further examples I want the recursive line count of every Javascript file within this folder in bash it looks like this with ammonite it looks like this one line about half again as many characters I'm not going to try to pretend to tell you that this is so much strictly better than this but it's not that much worse and it's actually something that you can acceptably type at the command line like like I just did over here right in front of you without looking up any documentation so other examples I want only the dot files in the current folder in bash it's like that in ammonite Ops looks like that and we can try it so if I if I list for example my home folder there a bunch of files there I want to filter the files whose last path segment first character equals Dot and I can print line because the repple sucks at echoing things and here you have it this is is this kind of interactive experience that ammonite provides that bash provides that is traditionally been lacking in many of these non- bash languages and it's why people keep using bash even though people acknowledge it is not that great a language so but again half as many as characters have again as many characters which maybe is acceptable last example I want the top seven files in the current in the current working directory so find ls- s- nrk 7 pipe into head 7 the Amite version this recursive the current working directory maap to a toule of the size and it's and and the original file path sought by the negative of first element in t a size take seven so again about half again is veros so that's not great but arguably easier to read so who here knows what dashn rk7 does two people great three people four people okay so who here knows what sort bu does and take so you're paying about 50% verbosity in exchange for uh 75x increase in people who can understand your code which I arguably is an acceptable tradeoff what uh I could have done s by dot size that's very clever I did not think about that so that this this does work so um what does M9 Ops gives you it gives you easy convenient file Sy op in Scala that almost as concise as bash not quite in bash this is LS with no with nothing else but you know 2x the goal was 2x not 1X but it's definitely less with typing in java. iio ja Java niio or or even doing this in Python it's much less typing and the even though it's less typing the data model is much better so in amonite Ops paths are not strings like multiple strings can convert convert to the same path appending strings as paths usually gives you invalid paths in different ways and with Amite Ops you do not need to think about this at all and in fact many invalid combinations of paths for example appending con catting two absolute paths together is that are statically prohibited by the compiler furthermore results from commands aren't strings either I get a sequence of paths I don't get some bite stream I don't need to split on new lines and worry about file names which contain new lines I just take the structured data and can map that map filter do whatever I want with it so this begs the question can we use M9 Ops plus the sky repple as a default shell so let's try doing some real work with this well some fake real work so I already have M9 Ops listed here I can say get diff um so there's a bunch of stuff um I can get commit D am or Das a let's say and then you blue this manually I need to say blah so sub process subprocessing out to open vim and other things from your rapple is very convenient as you can see I can open git I can open Vim um so get status let's say I want to get push so it kind of works it's it's not great because like for example this is bloody hard to read if any of you have are trying to read this it's not great and if I do if if I make a mistake for example I try to open up SBT like why wouldn't you open up SBT and you open up SBT and you decide like well after doing what do you do with sdt maybe you're doing a compile and I get bored and I go kill it and I I get kicked out of my scalar shell and that's rather unfortunate so in general it's not really good enough like it's kind of close but it's not there you can't really use it without constantly being kicked out to bash and constantly being puzzling over weird weird outputs being echoed by the Scala repple so the echo out put un readable control C kills everything so if you have a runaway code you want to interrupt it either you wait to Infinity for the code to finish because you know the because it's t in complete so you don't even know it's ever going to finish or you kill it and lose all your work neither of which are good Solutions you can't sub process out with barking jline I didn't demonstrate this but you could take my word for it and there a bunch of other problems that need to be fixed so one interesting thing about this is that none of these problems are really problems with Mite Ops they're mostly repple problems fact they're all repple problems so what if we wrote Our Own repple so writing repple turns out to actually not be that difficult and like people have written like simp like simply sca.com you can go and try Scala online you have learned trc.com writing a repple is much easier than writing any of these websites you don't need to F to HTML and web servers you just have all the have the compiler and feed code into it so Amite rapple is an attempt to do that so as a goal you should not need to quit your Scala repple you should not need to quit it to open Vim you should not need to quit it because you need to update your dependencies you should not need to quit it because you B the repple because you did something weird how often do you restart your bash reppel because you screw up bash the answer is not very often so why should you need to do this your scalar rapple so M9 Ops is Standalone has both Standalone and distributions you can add to any existing SBT project to make a SBT like test console use Amite rapple and we can try it so so let's just curl it right here let's just so that's going to take a while so I'm just going to use one I downloaded earlier which is in my home folder so it takes a moment to load and here we have Amite repple and you can see the first few compilers are slow because scy compiler is very slow and Amite repple does not fix that but immediately you can see that well the Mite repple pretty prints things much nicer than the scalar reppel does and to show that out even further if I do something like this which is purely just to make a whole bunch of a big data structure um to show to show on screen you can see the mnet reple actually it actually puts in effort to make the thing nice for you to look at so this is not going to be surprising for anyone who's ever used for example I python raer which does something similar so for in Python if I say x for X in range 0 to 10 does it like that if I do 0 to 100 it spaces out nicely like a reasonable person would not like the scalar repple does um so for example I could do list the current working directory and Stat all the files you get reasonable output if it's too long gets it gets um wrapped if I do a full stat which has even more data all of it basically gets wrapped and I can show the full thing because got truncated um there's a bunch of nice features that Mite rle provides that the default scal rle doesn't so you have good pre printing you have nice syntax highlighting so for example I don't if if you noticed I don't have actually showed it but if I list like current working directory SL Target slash I don't know I don't know what you get in Target let's target streams you see that code you're entering has syntax highlighting and that's and that's actually very reasonable there's no reason why your code should be black and white when you're working intell your code is not black and white so why should you be black and white working in the command line um it controlly interrupt interruptible so for example if I list. recurse root and I want to print it all go question is if in a normal scal repple i' be waiting here trying to guess how many files I have in my file system to decide whether or not it's worth losing the work I already put into that session whereas the Amite rapple just kill it just like you would in Python um or bash you get some neat tricks that you can live Lo modules from Maven Central so let's say I want to try out Scala Zed I'm decid I'm really into this monadic higher order type level programming thingy um so I look at Scala Zed here's a website the GitHub is apparently the best tutorial I could find on Scala how to use Scala Zed I load IV Scala Zed and let's just move this here so I don't need you keep flipping back and forth and I can immedately start using scaled in the reple so it's that I do this strange incantation that does funny things with options and it seems to do the right thing which is great um it's not limited to things like Scala for example I want to try let's say I want to try out finagle well finagle is not part of my class path because why should a Repel have finagle as have a web server as part of his class path but could for example like this Lo finagle and then in my case it's cach little bit faster if you guys try it it'll take some while to actually download the jaras and cach it in your IV folder and now I can immedately start doing things so I can do that let's make myself a service in the repple so the MN repel also has nice multi-line editing which is kind of neat so I can for example say I can edit this to say print line hello I am uh W and let say I want to interpolate this properly and you can see like inter even the interpolation is colored correctly finish this off kill it not kill it and then complete it now I have a server paste these in and now I can immediately start going to Local Host 80o and see my finagle code in action so that's kind of cute and this is not something that you'd be able to do normal scalar repple because you spend so much time going back and forth between know build the SBT and your SBT console and your scal rep reple that you'll just forget what the hell you are doing whereas in Amite reppel you pull down the code with one command you immediately start importing and can use a user code right away including pretty big Snippets because multi-line editing makes it feasible to go back and fix things up okay so the rep Amite reple provides a bunch of nice things and it's basically a strictly better scy rapple that there are is possible that cases where it's wor if it is tell me and I'll fix it um but it's not that hard to be better than what's already there so kind of like JavaScript kind of like SC scjs you don't need to be perfect you just need to be less worse and this is I think this is definitely a case where I can it is already less worse than what's already out there it's usable in any SBT project too so if you if you add the previous if you add this command to your build SBT file you can just type test console at SBD test console and your normal SBT console will become an ammonite repple and it's usable Standalone so I already showed you you can download you can download the executable from my GitHub and just run it immediately so this then begs a question can you use Mite Ops plus Mite rapper's default shell so let's try it so over here I have this project called wood JS that's actually not mine it's um some other person on GitHub who put it up for different scal days talk so let's let's go up um so here's my workspace you can see I was playing Pokemon recently um I can go into wjs see what I have I don't actually know what all these changes are so let's get diff um okay so let's say I want to make a change let's first try running it so I'm going to open up SBT cuz the first thing you do in a project well the first reasonable thing you do in a project that you're not familiar with is you see if it actually works at all without any of your changes so I'm spinning up SBT console SBT is being slow as always but here it is and I can say server SL run so wjs is an implementation of the wood collaborate collaborative text editing algorithm so now if I go Local Host 9999 Local Host 9999 I can basically edit text here and it should come to The Other Side by default but it isn't because I screwed up so let's let's kill that and let's clean up some of the come on get out get out okay I'm just going to kill it kill this guy and let's try that again so spin up the rapple um the problem is I CD up sljs so the problem is my git status was dirty so I'm just going to like git reset hard head and then let's try that again um let's just do server run this time since I don't feel like typing it again in the console so hum hum hum spinning up SBT come on SBT you can do it um so this is while while it's speeding it just talk a bit about what this is so this is implementation that has a web server and two web and browser clients Implement implemented in Scala JS um what happened here okay clearly this so let me fix this I'm going to check out this get clean xcf and monite SBT server run see if I can get that working so what it is is it lets you have collaborative text editing in the browser that's implemented using this boot algorithm which is not nothing new it's well known published a few years ago and what's interesting is that it's the same algorithm implemented in both Scala JS as well as Scala jvm so the code is shared so this was done as a proof concept for Scala JS it's clearly not production ready but it should if if SBT does my bidding it should give me a chance to show off how um H how this work how am the ammonite repper will work with Scala JS not with Scala JS with a real Scala project in this case include Scala JS so let's try this again attempt number three please work okay here we are finally worked I guess I finally cleaned out all the bad State I had so this is so now we know this works I can kill it you see interrupting the program doesn't kick me off a scalar rapple um my git status seems to be clean um let's try making some change to this program so let's try changing the place where the the socket the port at which it connects to so I happen to know that the server I think it's server source mean Scala the main do Scala file has important information so let's go to 8888 here you can find it here let just change it to 9999 um okay and you also I also need to change the resources let's list this the resources web socket. JS file so where's my a 8 9999 and then let's say SBT run well that's not right so SBT server run so again this will take a few seconds um so let's wait for it to compile hopefully someday the scal compile get faster for now we can just TW our thumbs as an alternative um address already in Ed oh because my my earlier server was still running so let's change this from let's change this to like 9988 or something let's try that again please connect I swear I didn't have any anything looking at listening a port 9988 earlier yes um so when you have to like kill a program have you gotten that working in uh what do you mean killing a program I noticed that when you need to kill a program drop out of to k um sure so I'll get get in that in a second okay so now we have it working so I'm not trying to Bluff you guys to answer that question um the reason I I to kill Amite to drop out just now is because I had a weird check out of it that had bad code um so after I cleaned up and went to earlier revision that actually worked um it's fine um so in this case killing them once I'm back to normal version of Amite killing program works great so for example I want to do while true print line oh no it's Runway code oh no kill it get back get on with your life so this works um and as you can see making changes and stuff works too you can basically do anything you could do with bash using amonite so for example I could open up a python repple and start printing stuff cuz why not um I could SSH like my web server let's see if this actually works and could do you could do some things here whatever you to do with your normal like SSH work and then you get out back to back here and you can keep going so that's pretty neat and I think that that kind of answers the question in that you even though a scalar repple is not a plausible system shell with some tweaks it kind of is maybe so this is clearly not something that we've that millions of people are using but it's something that I think is plausible and I've use it for hours at a time without getting angry at it so and that's much longer than you can survive it using the norm noral scalar repple or even using a python rapple that you'll get angry within about 2 minutes so you can do real work in the Amite repple without getting getting out of the shell so as another example you can you can open um you can't open like the your home sl. amonite SL predef do Scala file and make changes this for example let's say I don't want the last PA segment to be my prompt I want all of the path segments so I'm just going to say two string and this case because I'm reloading config I actually do need to reload amonite that's actually not different from bash and now I have the full path segments and you could easily put like your bash prompt you could put sorry you could put like your git bran you can put your current timestamp whatever you can change a color so I could say like colors I think it's colors how to do this colors. prompt equals console

red um I'm just going to do prompt dot I'm just going to set it back to the default for now just because it's really long and in general the whole thing is configurable just like a b shell would be you can like tweak with it inside the shell and only rarely do you need to come out so this still work in progress we are still missing things like autocomplete so even though we autocomplete properties and names in scope like I already showed you here um we don't we don't auto complete important things like file system paths which are actually very common in bash like maybe 99% of your data structures are file system paths so they deserve special handling and there are other things which could benefit from autocomplete as well we we would it would be nice to fetch Scala do and show in the terminal I mean intell intell effect is a Scala doc me my crappy python rapper here if I if I import OS I can ask for help on things and it gives me documentation so why shouldn't the scalar reel do this python people have had it for years um and it's is very useful and like Windows support currently it's mostly blocked on Terminal weirdness so the the file system operations already work so if you want to use this an existing like application or Library you can do so but the abite repple as a repple the colors are all ANZ Unix colors which are different from the windows colors and I don't have Windows and I don't I I don't have the drive to go and fix it but hopefully someone can help pick that up so what did we do so we have two libraries one is I basically wrote two libraries one is Mite Ops which is a really nice file system file system Library so I'm using it in a bunch of projects I'm even being paid for one of those projects which is great um and the other one is Amite repel which is a really nice drop in Scala repel which is also great um and together you have something that could conceivably be be your home at the command line so not definitely it's not the next big thing yet but it could be it has a chance whereas the default python or scal reel don't have a chance um we reimplemented much of bash's functionality in Scala so now I can do things like um I can do ls. Rec I want to see let's say I want to filter out only files I want to group by to extension on to map values so it's very easy to do things like that where let's let's show it just so we see the whole thing it's very easy to do things like that where can ad hoc run queries on your file system which in fact are probably too complex for you to come up with to run on Bas without looking for stack Overflow like who in this room would be able to do this without looking at stack Overflow in bash three people so that's the goal of that's kind of the goal of anite rapple it gives you a way to do these things in that's actually it's not even not difficult it's obvious you take you list the things filter out the things that you want group map done so that's what we've done we've Ted scolar synext into a weird bash like form I know not everyone's going to be a fan of this kind of weird piping or this kind of weird exclamation mark and maybe it doesn't have any place in a real Enterprise grade software project but neither that's bash I argue because if if this is sketchy then what is Bash so if we if we accept the premise that having some kind of conciseness is valuable then this is The Logical conclusion that we should make the things that you'd use concise and they kind of behave like pipes anyway so just make them look like pipes um and we've re reimplemented the whole scalar rle to make it work so this is easier easier than it sounds but still was a bunch of work I think there maybe like 3,000 lines right right now for the whole thing which is okay so why did we do it why couldn't we fix bash fix python or fix java. iio or java. niio and I would argue that the reason we can't do these other Alternatives is actually pretty fundamental so if you look at the space of possible system apis you always have this trade-off between danger on one side verocity on the other side and in both cases you probably have ASM tootes so no matter how veros you get it you're not going to get lose all the danger and similarly no matter how dangerous you're willing to go you're still going to need to type something to get the computer to do what you want so there are ASM tootes here it it doesn't go to zero on either end both vertical and horizontal bash I would argue is very near what I call the clips of insanity this is a point where you can subtract characters and a danger shoots up like tremendously and Bash is definitely in that case you misquotes you misquotes around some very aable interpolation and suddenly expands into multiple things and you've suddenly deleted your whole database Fleet um Java on the other hand falls on the other side where it's so verbose it's not even doing anything anymore you're not actually you're not actually gaining anything from the verbosity you're just wasting your time and like Java nio i' would argue that maybe it's better probably is better because it's newer and has more built-in functionality python I would argue is not actually any more dangerous than Java at doing the these things so part of the reason is that Java in the PLS of diminish returns is so verbose that it's not actually gaining anything from verocity but another reason is that both of python and Java basically treat Parts as strings you get a path you don't know what it is you could have random like UPS inside it may be absolute may not be absolute you don't know you may or may not have to call canonicalize or normalize to get to a path that you actually want and when you're working in bash so many of your paths some so many of your data types uh paths that when all they all strings Java's type safety doesn't really get you anything so where does amonite fall on this on this Continuum so I would argue that amonite doesn't fall on this Continuum at all and that's because there are ways to make your code less dangerous while at the same time being less V both or being much less dangerous for equal vity or being much less AB both for same amount of danger and most of you would be familiar with these ways like it's not magic there are better ways of doing things for some subsets of the things that you do like functional programming many many people argue that using map and filter rather than having while loops and for Loops sure it's slower makes more class files but it is more concise while at the same time being easier to reason about and harder to get wrong and therefore less dangerous similarly inferred types give you a large amount of the safety benefits of having a type language without the verbosity because you don't write the types the compiler figures it out so this actually lets you push the whole curve and I would say ammonite fall somewhere here less than twice as a both as bash bash is somewhere here amonites here while still being more types more safe to use and less dangerous than any of the existing existing Alternatives so one cute demonstration is that I have my current working directory I can I can I have like uh I can have sorry the current working directory is absolute path I can have relative paths I can combine relative paths with other relative paths so res 10/ res 10 longer relative path I can combine absolute path absolute path with a relative path I cannot combine an absolute path an absolute path or a relative path with a relative path sorry or a relative path absolute path and why would you ever want to like what meaning does combining users twice give you actually doesn't give you anything um and with ammonite we meant this is some it gives this degree of safety that you don't have in bash python Java iio Java niio or any other um file system API I've seen so there are problems using scal as as all shell so jvm is slow to boot when you have a bunch of class files I know the hello world is maybe 40 milliseconds but once you're loading Scala compiler Scala Library Scala reflect and a bunch of other things usually takes several seconds to boot and this includes all the class loading that's going on and the first compile is slow so it's easy to demonstrate this I go back to bash I open up Mite one two 3 spins up first compile one two 3 4 and then and then it works and of course after that it gets faster but even at its peak if I'm doing this over and over there's still A2 to. 3 second lag which is noticeable especially when python literally takes 4 40 milliseconds to spin up and doing work is more or less instant so that's a pain um bash is python are fast SC slow the Mite repper is kind of fat as an executable so the jar itself is 30 megabytes no jvm plus jvm it's going to be more than 100 um and that's fat for something that doesn't need actually need so much functionality so there is hope because the one benefit of being in a large ecosystem is other people can do work for you and I don't even need to pay them so like Java 9 modules would help cut down the size of a jvm to make bundling smaller which would be great and like Java 9 people have been talking about um about closed World optimization for the modules like do the Doty people have been talking about clal optimization for the modules we should possibly make the modules that we load as part of initial Mite start smaller and thus faster to class load because it doesn't matter how F fast a jit makes it because the first time if you when the first time youve always have to load all this trash and hopefully Doty will make things faster if it makes things slower no one's going to use it so there's only one way it can go so looking at our original application architecture it looks something something like this danger a small island of safety where can reel in your type safe Moon ads but everything else is basically crazy um with scar JS we fixed a bunch of things but there's still this huge spiky underbelly which is possibly problematic the goal of ammonite is to try and solidify that stuff and it's not meant to be typesafe monatic reactive Enterprise worthy but it is meant to be something that you can reasonably rely on and at least you know to to at least parse the way you expect it never mind run and in our case we even have type checking so Mite Ops could possibly satisfy the use of bash and python as a file system manipulation Library while amonite repell could substitute use of bash as a shell for you to live in to do all your work in to open up to to open up SBT to commit St to gate to push things and all those other nice things so and that's how I think it's possible for us to finally move Beyond bash so here's some here's the docs here's snippet if you want to use as library or use a standard executable for the rapple and I'll take questions um so the question is when I'm typing commands sometimes I'm using strings and sometimes I'm using symbols and the answer is they are exactly the same so if you can use a symbol just use and save one character and in this case the assumptions characters are important for for live usage so that's why I do that's why I do this any other questions why do you sometimes prle okay so um let me let me me set let me reset my prompt first actually so prompt because it's really long and hard to use so let set that so the difference is that um the things that you bang are functions that live in scalar land so basically my own implementation of list make there and whatnot and in general the these are aim to behave better for some degree of better so for example RM Works everything rather than just being The Bash RM but for there's always a long tale of things which have not been wrapped like you could wrap them but it's always going to be a lot of things so G SBT actually not have used SBT it's really slow like like Arc for like use fabricate that work and tail that sort of thing so percent is basically a convenient use shell out while the bangs are in built-ins that are presumably useful um and percent so there you don't get any code completion on the percentages not any useful code completion anyway because it's uses Dynamic questions this is yep uh yeah it does so let me show you um uh what what they call it I call it rapple you can see without having syntax I think it's kind of pain Source test resources do that work source test resource Standalone hello do Scala so this is a thing um so let's just cat this to take a look so this is a script that should inally print out hello world um and I should be able to the is not I it should be low. module and then prints it out so there are a bunch of limitations that will eventually be solved like for now a script cannot load other scripts C has implementation problem that we did it wrong and we'll need to fix it um and you can see that the reppel the command line is still slightly buggy um but yeah you can load scripts you can Define scripts and load them and it's it works questions so does your file system like RM and LS fall back to the batch underline or to theer over the jdk library uh those are all rappers over jdk so either iio or nio in theory yes if you're willing to put in a bunch of work but it's not currently decoupled in theory anything can be decoupled recognize environment variabl um it recognize environment variables it does not currently give you a nice syntax to set environment variables for your subprocess calls so that's a limitation that will need to be fixed to make it usable what about sud um I don't think I've exposed sudu yet in theory so I guess the wor wor come to wor you shell out you shell out to bash and pseudo things in here um you can't actually you can't do that H yeah so you can pseudo by shelling out I suppose not quite as satisfying as puding and being able to do like commands like that for example but if you need to you can other questions one back there is that a question that's not a question okay go for it um that's actually perfectly reasonable in many setups but this is meant to work in existing setups not the ideal mono like single process OS any other questions um okay going once going twice going Three Times Gone okay thanks [Applause]