Text By the Bay 2015: Varun Ganapathi, How Terminal makes Machine Learning Fast and Fun
Recording: Text By the Bay 2015: Varun Ganapathi, How Terminal makes Machine Learning Fast and Fun
my name is varun ganapathi i'm the ceo of terminal.com terminal was started after i left google back in 2013 so prior to that i was a phd student at stanford where i did a lot of machine learning and computer vision i started a company which then became part of google at google i did a lot of machine learning and computer vision as well using their infrastructure and uh what i found is i felt that there was a way to make a better infrastructure for doing machine learning in particular a better cloud that was focused on just machine learning and data science so terminal.com is our our start our startup that's trying to make everything a lot easier to do for data scientists so when you go to terminal.com what you see are a bunch of snapshots so you can go to any one of these and these are snapshots that are published by users you can start them up and when you start them up they come up in about 10 seconds usually when you're in when you get to one of them i'll just show you an example of starting one up you just click the button and then you click start and in the background this just happens um i'll just switch over to another view where one's already started up to save time once you start up the terminal you get to a page like this this is an id in the browser on the right side you'll see i have ipython notebook already started up and running inside this terminal what's really cool about terminals as opposed to normal vms you might have experienced in other clouds is that a terminal retains its ramp state when it starts up so what that means is that all processes that we're running when you start up the term when you took a snapshot remain running while you're in the terminal and this gives us a lot of cool capabilities as well um so i'm in this terminal i can you know it's like a normal fully functional terminal you'll see up here um i don't know if it's too small i'll just make it bigger you can see here this is the amount of ram i have on this terminal it's not very much 262 megabytes so let's say my program needs more ram what i can just do is go up here and live update the ram or the disk or any other state about my terminal click save and in the background terminal makes it happen and so when i hit top again you'll see that you have way more ram now it's 3.2 gigs of ram as opposed to 256 megabytes of ram so what was really awesome about that is nothing has changed my computer so i'm just working you know i realize i need to compile something or suddenly i realize i want to do a parallel job inside my ipython notebook while i'm running it i can just do that without having to you know freeze an instance or start up another instance copy all the data over all that stuff i don't have to do that i can just literally just resize my machine while i'm using it and of course you only pay for the time that you're using the bigger instance so you don't have to pay for it when you're not using that then when you're done you can just say okay i want to shrink back to a micro and it just happens in the background as well so basically you have a computer that you can just resize anytime you want its resources grow as you use it and you can just when you're done using it just like your own laptop you can just pause it and so i'll show you that as well when you go to your list of terminals every time you start up a terminal it you have a you know a page here that has a list of all your instances i of course have a lot of them you'll notice some of these say pause next to them so what i can do with the one i was just using is i can just hit pause here whoops i can hit pause here um and it will pause and this pause is not like a shutdown this is actually just it's like standby you just it's like closing your lid and it's pause and then when i click start again it's exactly where i left it so i can do that and when it comes back you'll see that all the all of the things all the process i had running everything is exactly the way it was when i left it i find this to be a much easier way to use a cloud than any other approach so i think i actually pause and resume the wrong one but you get the idea another really cool thing you can do is publish snapshots which is what we encourage people to do because one problem i found in research is that it's not very reproducible in theory code makes it so that you can totally reproduce anyone's research but in practice managing environments is difficult and oftentimes you lose the exact you know the operating system or what libraries i had installed all of that stuff can get lost while you're doing your work so with terminal what you can do is just snapshot your instance while you're running it and everything is exactly the way you left it to demonstrate that i'm going to start up a python server on port 8000 and so it'll start up here i'll pull it up in the browser that's built in um so you'll see that that server is running on this terminal and what i'm gonna do is snapshot this um while this is all you know happening it's it has the actual uh all everything is running um is that easier to see or harder maybe it's harder while it's running and i can basically just give that snapshot to someone else and when they start it up they'll be in exactly the same state so it's starting up text by the bay cool that happened so now the snapshot is published when i start up the snapshot it will be exactly where i left it and so that's the basic paradigm is that we make it possible to essentially very quickly create a url that uniquely identifies the state of your entire instance including its ram and then be able to share that with other people and or just for yourself where you can go back and say okay what was it like right at that moment okay i can go back and compare like what happened what was different what was the same and the snapshot includes all of your data as well so if you have a bunch of data sets loaded into the terminal that comes along with the snapshot and it's copy on right so therefore if you modify the data in one of the snapshots the original data doesn't get affected so i find that to be extremely useful for just keeping track of your work it's sort of turning your entire machine into a document that you can just very quickly start up copies of the internet here is a little not as fast but one more example i wanted to show you another cool feature um is for pair coding terminal is really useful as well so if i pop open uh if i can share a link to my terminal with someone else and we can both be in there at the same time so i'll show a demo of that so i'm creating two windows we're both in there at the same place and we can both just collaborate so say you need help with something from someone who's not even on your coast like someone some far away actually we use this feature even when we're in the same office which is a little weird but you can just simultaneously code in the browser at the same time you know it's simultaneous editing everything works so basically what that means is that you can basically pair code together right in the browser with just sending a link to someone which is really useful the way you control that is by setting permission so you can say okay i want this person with this email address to have access to my ide or you can even do it on just a port so for instance i can say i just want to make this one port available to everyone once you do that now it's on the web anyone can go to it um right so i can show that example um right here so you now have an instant web website um running from your terminal uh shareable from anyone anyone can go to it and or you can share with specific people so that makes it really easy to work together for instance if you're working on the same ipython notebook you can just send a link to your terminal with the ipython notebook server running and just use it terminal has a lot of other cool features but first let's just go back and look at that terminal that we had that i started up to show that it was exactly the same stage so if you remember this one is text by the bay this is the snapshot i just made when i go to it it'll be in exactly the same position that the other one was in so we'll just wait for that to the internet to get us into it so you'll see the server is running and the tabs will reload exactly where you left it you'll see the server is exactly where you left it so if i hit this again um it will just update the same one right here yeah so you'll see that happen so basically it's a very flexible way to bait to start up instances uh do whatever work you want to do in them snapshot them and share them so people have created a lot of really cool ones on terminal this one is a really interesting one it's a vnc client a web-based vnc client so literally this is a browser running inside a browser running inside a terminal you can just you know we can even you know we can even go another level down and go to terminal itself here um oops and so now i'm using a browser that's in the cloud from my browser you know may not be you may not always want that but what's really cool is i can share this link to someone else as well and then now we both have a live session to the same exact web desktop so two people at the same time we can both scroll um we can both use a browser fully with all the graphics and everything right away which is really useful for a lot of people so people have shared a lot of snapshots on terminal for machine learning one really cool one is prediction io i don't know if you've heard of them or if they're giving a talk here they basically made a snapshot that has everything ready to go so you can basically start up a movie recommender on terminal in 15 minutes you just start up their snapshot and it has everything baked into it it has a data set everything's ready and what's really cool is once you've started that up uh it's already a web service that means that you can use it from your other terminals or from any server on the internet so you can basically just spin up a machine learning tool for you running on terminal and hit that from anywhere which is really useful um this video we're going to put on our blog so you can see it they they basically demo this at our meetup a few weeks ago and they keep updating it there are some other more specific experiences we've built so for ipython notebook when you publish a snapshot if you snapshot a terminal that has a notebook running in it we can just go into the terminal um have you guys all used ipython notebook have you heard of it yeah so the advisor the creator of ipython notebook is actually an advisor for terminal so he's helping us make ipython notebook work really well on our website so what's really cool is when you snapchat a terminal with notebook in it it just recapture the entire state of that notebook and just publish it along with it so people can go and look at that and when they start it up they'll be exactly where you left it so i can show i can switch to the notebook one to show that here yeah this is the one with the notebook and what's cool is you'll notice like when i started up the ramp state is present right so that means that every when i press enter right now it's exactly where it left it so let's see we can run all and so on so essentially uh what we're trying to do is make the cloud much more easy to use much more flexible you don't have to be a sysadmin you don't need to be an expert in ec2 or something to use it it's all accessible in the browser if you want to ssh in we provide that as well so there's nothing you can use it just like a normal cloud instance if you want to and finally i'm just going to show a bunch of snapshots that people publish that are machine learning specific so we have an octave ide it's sort of it's basically matlab but it's the open source version you can also install matlab in a terminal yourself if you wanted to if you have a license for it we have an rstudio server terminal which is really nice they have a really good ide themselves and there are a variety of other ones already present so for all of these things you no longer have to spend a lot of time getting it set up you just click start and everything is ready to go um that's actually almost that's pretty much all i wanted to say uh just present how you can use terminal to make machine learning easier to do thanks um do you have any questions oh yeah it's all root you have a root you're it's just linux yeah you have full root access there's nothing weird about it you can app get install whatever you want um you know i can show an example i don't know if this is yum right here or app no it's full state hard drive state everything so the entire machine state is kept so it's just you can treat it just like a regular linux box if you wanted to you don't have to think of it as anything special one small thing stanford class use terminal to do their gpu deep learning on gpus terminal has gpu instances so when you spin one up you can get a gpu right away and then pause it and resume it just like you would with anything else which is really useful all of it yeah that's a great question the credential issue uh one obvious way to do it of course is you can just remove the credentials from it when you're in there another trick you can use is um start up another terminal that just acts as a proxy so it basically is actually the one with the credentials and you just talk to it when you want to get access to data so we've done this um we have another thing called local terminal where we use this feature ourselves where you can download any of these terminals run them on your own computer i forgot to mention that so you can just click download it takes that entire state runs it inside virtualbox on your own computer and you can still use it using the browser or whatever so you can work locally but that's a tangent um what you do with that a second one a second terminal is you just ask it to have s3 generate a temporary url that url will only be valid for a certain amount of time then that url you give to whoever you want to access your data set and then they'll be able to download it uh no we run on ec2 ourselves actually so we um we're gonna launch on more azs so you can pick one that's closest to your data set but one cool thing about running on ec2 with s3 is that the data flow back and forth from s3 is free so that's really nice so essentially you can think of this as just making easy2 like way easier to use and you can just run it um there are also users who took terminal they don't want to run it on our public cloud they run it on their own ec2 and vpc themselves so you can literally just take our software um with all of the container management and everything that we've built into it and run it uh yourself for your for your for your own stuff i just use the word container just uh to clarify these are not the same as docker containers um as you know docker containers don't have ram state they don't have all of these features and they're not um i guess as secure nothing's perfectly secure but these are sort of hardened containers that are designed to be used kind of like entire linux boxes not as just for an app container what so currently the way we handle that is uh the strain way linux does when you run out of ram which is that it will kill processes uh that have too much ram so essentially it'll just start killing things to get itself down to the appropriate amount of ram we could throw an error but so we do that for disc we could also do that for ram um speaking of which uh one this just explains one reason why we we didn't do that but we could maybe put it in some way is we have a thing called idle settings it's basically like power management for your terminals so what you can do is configure a terminal that when the cpu load is less than some amount or when it's been accessed not frequently it should just pause itself or micro itself so it'll just turn into a mini instance so that can be configured to happen automatically so you don't run into this problem where you forgot to turn off your terminal and now you're just losing tons of money because you know you just left it on overnight or something like that it protects you against that um one other point i forgot to mention is an api all of the stuff that's here is accessible through a rest api what's really cool about the rest api we find people using it is that they'll automatically start up a terminal and run a script inside it just for the purpose of like running a job essentially so you can just use terminals as a way of spinning up jobs in the cloud having them run and because they automatically pause they can just you know go to sleep when they're done and you can go check them later or of course you can publish the result of the data set to s3 or something and then kill the terminal off so basically this api uh you can find it on the api docs you can basic we made it really easy you can just try it out right here um it auto-generates the code you need to actually use it so here are some examples you can say what sort of snapshot you want to run that's an id from the snapshots page and that can be your private snapshot you tell it the instance type and then you can say is it temporary or permanent should have auto pause meaning like should it go to sleep automatically when it's not doing anything and then the most useful probably is a startup script so you just give it a script and when it starts up it will just run that script automatically by itself so you can just spin up tons of jobs in the background have them run and get access to them later if you want to any other questions great so if you have sensitive data sets uh we recommend using the private cloud version of terminal where we run it it depends on who is regulating you right so some regulators say that it has to run in for instance uh a hipaa compliant data center right and in that we're using ec2 under the hood so we're we're standards compliant because ec2 is standard is compliant but say it has to be in the government cloud version of ec2 you can just start up your own terminal like cert or entire website basically for your own company just on your own vpc and what's cool about that is terminal will take care of behind the scenes spinning up additional machines for you and killing them off when you're not using them so effectively it's sort of an auto growing and shrinking cluster that you can just run inside your own cloud for yourself for the software for that software uh we've been doing it on a like a pers like a personal basis like people contact us and we talk to them because we offer support along with that to help you actually use it and of course the same thing can be done on your own data center so if you happen to have physical servers i don't know if people do but if you have physical servers you can do the same thing you can just install terminal solution on your cluster locally and then just use all of our features but just in your own cluster so other people have been using that using terminal in that way cool any other questions okay yeah so i'll show you the bass snapshots so we have all the bass normal bass ones available um uh if you go to bass os these are ones published by us these are clean so essentially uh you know official centos 6 ubuntu 1404 debian 7 and if there's one that you want that we don't have we can just create that for you um or you can even create it yourself so you can start from this one and then just install anything you want like get it all set up ready to go i did this for myself for gcc 4.8 and thrift and all this stuff it was a nightmare to install all those so i just exactly so i just snapshotted it and everything was ready i have another one for doing kernel development so what's really cool is you can just create these environments pre-bake them with everything ready to go and then just snapshot it and then this is really useful especially on teams because one person can set up the dev environment and then just give that url to everyone else they can start it up you know get clone the code and then just work and not have to deal with all the library management oh great question uh yeah so there are limit with the gpu ones your your virtualbox has to have access to the gpu um that may not work it depends on your on your local setup is your dev machine linux or not okay so with linux the cool thing is you can run us natively you don't need to run virtualbox you can just run terminal direct natively on linux in which case it will just work what we do yes on the host so you have it installed on the host we mount access to those devices directly into these environments so it's totally native speed there's no uh overhead associated with that so on linux it's really great in particular yep great i'm done okay all right thanks you