Scaled: Cliff Click: Scaling Java and AI with Compiler Power

Scaled: Cliff Click: Scaling Java and AI with Compiler Power

Recording: Scaled: Cliff Click: Scaling Java and AI with Compiler Power

[Music] I think reading your biography and knowing a little bit about you my hypothesis is you are an example of all of this you're a skilled person you scale yourself you've got the PhD at Rice University and you worked on the compiler there and then at some you scale that you have a compiler right and by scaling the compilers killed Java and Java became a scaled language which took over the world and skilled son as a company in the process and then you were at multiple other companies and you applied this technique all right you put the compiler technologists into there of big data and they are that's kind of my my proposal how I see this how you scale yourself you scale called piece of technology which is given a programming language we scale the whole bunch of other things and they do this and most recently you scale other people because you did right and also you taught the workshop at scale by the way which is our conference and you taught I think they're all dozen people which spend the whole day following you into the depths of compiled enthusiasm I've not seen in people doing this so so I would say I think you're very well qualified to kind of inquire on the question of scaling but maybe first you tell you can tell a little bit of your own story in your own words and maybe we can pick up on the kind of questions of scale [Laughter] yeah so so you know a long time ago in a galaxy far far away right so how far back you want to go very young kid very nerdy very insecure very perspective introverted I kind of fell into computers sort of accidentally they were the new thing floating around the block and my parents were helping their kids and I kind of figured them out at a very early age and and sort of never looked back like it was a obvious good film at age safe each I got a Bell Labs lulu teaching computers for the very very young by age 12 I was passed a cast that grave with three days and I was judged too immature and too small go to Facebook mm-hmm although I had passed to eighth grade with three days so I retook the eighth grade every teacher knew me knew I knew the stuff I didn't know what to do with me so my math teacher gave me a vocal coach before I read about Fortran for when I was 12 that's summer I went to the university with my mom as a staff access guys to the infirmary portray an on on mainframe on a Xerox mainframe by 15 I had a RadioShack trash ad I got the byte magazine article on a Pascal compiler I thought this is a cool thing I'll go even with this so I wrote a compiler when I was 15 um these days that's a grad student course and it was just a natural fit wasn't anyone telling me it's all self-taught it's just the right thing to need to do at that time and sort of that theme just carried on [Music] by my my early college years I had taken the fourth programming language made a version which generated machine code directly and also and fourth is a like Java in a sense it has a JIT built into it basically it's a stuck by sluggish stack based language but untyped so think Python with no types but you have a rep all the language is a little odd for a stack we went to your head wrap around it it's a flying that can be made extremely tiny back in the day well it will run on kilobytes well mm-hmm the calculators yes will affect nice try the assault and the the boot roms for some computers and many many many machines the starting boot roms because in a few kilobytes you could have the programming language that let you peek local device drivers and get those off set up before you prove whatever the main OSS though biases have been frequently written interesting whatever as it goes I put an idea around it I put a better kid in it I did a bunch of things to it made different language zalien which is still in use as of a few years ago by a handful of people in different parts of the country so there's a you know 30 plus years long direct usage stop my road going through college I went from cogeneration as a half to formal theory on cogeneration turns into compiler work got my PhD from Rice did a bunch compilers along the way got pulled into Sun it'll make Java go fast mm-hmm because I had written a fast compiler I try did a PhD in fast compilation and couldn't get it my adviser would know so my PhD is in a fun piece of math and the appendix is all about how to make a JIT go fast interesting and the funny piece of math was another one of these things for somebody said you can't do that so I did it mm-hmm or several these things going on coming out of that and you're looking around for a job you know I was talking to people about redoing couple compilers for this 20 C of nodes things I had done it grad school and in all these things sort of the common theme is there's no compromise on my beliefs of what I could do and you know how things would work so I'm a very intuitive coder I understand things that's sort of a route level that I don't see in a lot of other folks and then lets me know what a glance what something will work or not or how to solve a problem but in exchange it's taking a very long time to learn how to put these things into words as well so I could tell you instantly that what you're doing is wrong and it could show you by coding it or write or here's how to solve or whatever but saying it gracefully was learning of a lifetime something as part of the introvert learnings the the part of the no compromise bit was knowing that it could work but the systems are large and complicating it to actually do work was to join the difficulty so always the fundamentals were correct but always the doubles the details in the case of big ass hairy off my skin file oh it's a lot of details going on so the compiler had a lot of details took a long time to get right but when it did other people had trouble getting it right and so hot spot eventually one although it was no means the obvious winner going into it hand-in-hand with that came the learning about parallel industry be computing and memory models and race conditions for which I don't know if I'm considered world expert you talk to different people they'll say yeah you're an expert I definitely have done a bunch of things there that other people have never done or didn't believe could be done hand in hand with that team the ability to have hot spot rod parallel compilation with her all class loading in parallel execution trailers you thought about and get all the bugs out and have it work well reliably and that's part of the scale in the java language was you know the world became multi-core when intel couldn't figure out how to make a faster chip mm-hmm so we got more what to do with them so I figured something out to do with it in terms of their core got some talent from there I don't know lots of different themes I went in hack custom hardware I don't do hardware but I talked to hardware guys directly CPU design Aretas ol sat next to me mmm-hmm we went back and forth of what makes a good Java did project that was another very much of fun exploratory learning thing very much high scale highly parallel your thousand three thousand fours a very loose term memory model so again all about memory models and parallel distributed thinking pivoted over to h2o which was doing is doing machine learning at scale so from parallel to distributed which is even a more difficult version the same thing but also picked up this huge pile of math so I will know no mean claim I'm a math expert but because in the parallel computing expert I've built a platform there that is a decade later still being sold and used widely that does the high speed math faster than the obvious competitor spark I like 10x normal the numbers are currently 10x but last time I checked there was still something close to 10x faster for scanner ordinary differential equation gauss-newton solvers jennarose learner modeling I don't know what the hell all kinds of stuff mm-hmm I kind of watched the rise of GPUs for doing real nuts and help diagnose and design but I didn't implement those algorithms on h-12 but figured out how to do sort of a high threat away parallel version of neural nets and Arno Cadell was really great I did all the implementation details there and I can't sort of praise Him enough I threw in my two bits and said this works and he did all the work questioning I shouldn't claim credit there but I remember that you know you as the CTO yeah you you give talks yeah but what really was great for me because one of the mantras would have in the community that it's easier for software engineer to recover should learning scientists the farmers should learning scientists to become a software engineer because all this complexity right all the conservative systems it takes decades to get right I think it's again at this point and so so the other thing you kind of proved this by standing up and talking about the AI for the software engineers and our audience is mostly software engineers it was jobs on the line to deliver everything they found was promised in terms of III is the performance questions and he said implementation specialist right and so it's one thing to do it in a notebook and another thing to do then distribute the sister so I think that was really kind of what was great about this too is that you guys basically Rivera consol and and you went in and you taught a bunch of software engineers and it's still going very strongly we admit up so I have the basically I think most of the folks go there are social engineers yes yeah yeah so so at age 12 I kind of started scaling the other side I know there's another aspect of me so there was obviously the tech and distributed which superfast key-value store that was cool distributed parallel machine learning very cool paralyzed some algorithm said no one said you could Carol people had said you cannot paralyzed that was cool but I also started working on my people skills and and trying to figure out how to need an engineering team in this sort of the most productive way so both to two people how to do it but making an environment that was safe for everybody who was making it palatable to be there they've been working today do to show your best yes since then I've been at a number of other companies where I have to chemically exercising that half still doing coding still doing distributed scaling computing still doing these things but also teaching people aspects as well as learning them sort of on a continuous basis and for the past two two and a half years now I have keynote a couple times a year on self-awareness for introverts mm-hmm which specifically targeting programmers why programmers stuck-at salary negotiations is the the byline on that one and that's just you me turning my engineering head toward people and emotions you know sort of in a way that it's different from how extroverts view the world and think about things and this just comes about because you know being an introvert I let myself get run over by a lot of folks during my career mm-hmm so so I made some really cool tack and I didn't personally benefit from it other than you know fame but not fortune and by the way if you get a chance to have either but not both unfortunate I held a lot more fun nose is brain helps fortunes a lot more fun no money won't make you happy but you can live very comfortably sadly with money it'll give you a runway to learn the method there's a different things you have to do to learn to be happy and you can be happy without money it's true but like I said it makes life a lot more comfortable specifically if you're raising a family having money helps mmm-hmm you can race a very happy family without any money but having money hops and it gives you an insurance plan should you know crap at the fan kind of thing sure with us we're where we live yes yes well in Silicon Valley you pretty much have to have some money to make it work alright fine so so there's been a lot of self-awareness for myself learning going on and then bringing it from my my core gut and my intuition and then and from my heart to my head and then for my head I can get into words and from words I can turn it into slides and presentations and teaching and I have done this sort of started doing this on small groups individuals and small groups larger groups wrong resonation with the crowd at the GRE which then turn into people who run conferences go to jaikrit they said come give this as a keynote galas a keynote first and their sauce to come give this keynote here and that has been daisy chaining along the subway years so there's a lot of aspects and I guess this goes to where I'm being scaled I I am still a world expert on compilers but also distributed in parallel memory models of memory ordering as part of that I am an expert at machine learning and mutation mm-hmm I won't claim new algorithms per se but I can take an old algorithm make it do new tricks mm-hmm I think I tell you for instance why random forests cannot be run distributed the obvious way where you run a build between every slice I'm at every different note it doesn't work mathematically mm-hmm maybe not for this talk but there's but I know that and that just goes to say I have a competence in ml that's deep go tuna rinse and I'm looking for fraud in stock market and again it's distributed and scared of scaled computing because I'm looking at a billion transactions a day kind of thing and a lot of machine learning and so I have a decent data science aspect to me as well and then there are a lot of people skills that I kept in finding from neroon sick from age 12 forensic and afterwards that turned into I can get the talk on introverts and people will totally resonate and I'll get mobbed afterwards and people will talk to me in this kind of a scene will happen for days and months afterwards I will get one-off individuals hey I was there at this talk he gave it really changed my life there was something there there's a there's a wake up for a large collection of the programming population that I think I can I can approach you know I can teach talk to you in a way that you understand yes you know this is interesting to me I want to pick up on a few themes which you mentioned so the you know one of the topics I want to really explore is how good engineering scales companies and I think been engineer you know right that really truly scaled companies which we respect such as Google Amazon Microsoft Facebook email of okay Tom but you know that they have great open source they have great engineers right and and companies we saw primarily run as businesses such as Yahoo as an entertainment executive set fail and so my as an engineer I love the idea that companies rise and fall ultimately on the basis of technology that have enough chairs right and so what you said it's very interesting because the solution offer second yeah let me claim that you need both sides in equally good portions yes because son had TAC but the management team was not an a-rank player mm-hmm there was some good people in some less good people and that ultimately failed as well yes so to run a technology company it requires both Jeff besos is a is the best of the best for CEO yes right so he may not have the technology skills that I do but there's no question about his business acumen yes that's a crucial piece of it part of that is he helps make the engineering palatable and useful and fun and so you get top top engineering which you also have to add have as you point out for the Yahoo example lack of that turns into you get overrun by better technology down the road at some point so you need both right in but in in the air I have a lot of friends who were at Yahoo analogies Disney different incarnations and pieces and so they were great engineers so what one of the topics actually want to explore I don't want to say that technology is superior to executive because you know that ultimately these are the executives who you know who some companies who buy companies who pay people money so I mean there is no question that the CEO overrules the CTO the question is I think a lot of books which talk about the history of Google Microsoft and Amazon they're written by MBAs and business journalists and they do not look in the tech knowledge and I think that one of the problems I want overcome with this is to get deep insight into basically my premise variable you cannot have a scale technology company well after the great technology it may seem trivial but the the point of technology being great and important and the way it runs is very often superseded by business stories right and so what's really interesting to me is what is the like how a company a skilled company and its executives should properly scale all the best technology what is the the good examples which is the right relationship and obviously the technology very complex it's very easy to just like move little post-its and keep track time like yeah yeah let me let me let me step in here um in all of the cases that I've seen work well here there is a core idea and a core champion mm-hmm the champion makes the idea and the idea is a technology that can change the world and the champion understands that it's changing the world maybe doesn't understand why or how directly but he understands that it is mm-hmm the business person doesn't understand the tech but they see how the technology changes the world and that's the reason they're promoting and pushing it and they let the champion be a champion of the technology right they they that it's the golden goose and you need to have them in the safe place and then you could run with the tap riff that comes this ideology of this this cool awesome type of steams new world that resonates so strongly with other engineers everywhere when they want to pile in hey I want to help do this I would make h2o scale and I made Scala scale I want to make sparks scale whatever there's a cool tech here and we're making it happen we're trying to save the world we see how it is saving the world mm-hmm the actual dotting the lines between saving the world and making money is not there probably not the engineering problem they're learning that piece be handled by somebody else they focus on the tech and they get pumped and they they carry and run wow that goes know you can build you can build a scaled company up to the point where the technology is or is not saving the world anymore yes yes but obviously you need to recognize the the management should recognize this is happening and there is also this stereotype that the Nerds are difficult right so because they don't see the business side right that's all like you've been in leadership positions right so so yeah I'm wondering right when you run teams yeah maybe we can talk a little bit and maybe yeah like what would you see some young guy coming out it was difficult and maybe you would recognize some work of yourself in this guy right and and like he he he does not see yet the implications of the business sometimes you need to cut corners to ship right and stuff like this usually difficult comes with personnel issues not the business not that they can't see the business they can't see the personal impact mm-hm of their behavior on people around them if they can see their personal behavior the busines of the people around them usually they can understand the business implications they have a broader view of the world mm-hmm they understand business implications and you have to explain to them like various cost-benefit trade off always when you're running these companies there is and I have to say just that stopped at a 50-person is the largest company was part of that I personally had a leadership role so at this size companies there's always a lot of risk involved there's a lot of trade-offs you don't clearly see the path to the the business success so you're trying to walk the best probable solution with Watson lots of paths in front of you that continuously for me and as things happen you you make a decision and you change direction make a decision change direction constantly and that's by the way is part of being a scalable person is I handle change very well mm-hmm didn't you speed that way one of the things I bring to the table is that can explain to engineers how that change is coming how to handle that change the change was not bad and what the impacts it means for them to be able to change so a key thing of that is they get passionate they pour their heart and soul into something they build a thing and while they're building it the business changes mm-hmm you have to pivot you have to explain to this person that the theme they poured their money their third their soul energy into their love was good at that time is no longer good this is no reflection on them as a person they are not bad the thing they built was not bad it just doesn't fit the current needs we means right and you have to step them through the laws they're going to go through a grieving process I'm going to give up my baby mm-hmm so you have to help them understand that they're gonna give up their baby they're not bad the baby wasn't bad we couldn't see if we could have seen we never would have steered you this path mm-hmm now things are different we like you to please step over and aim a different direction if there's something you can pull out of your baby and take to the new place that's great hmm but it's not you know let it go I understand this is painful for you I understand that you're gonna go do some grieving process you know go have a beer to memory with your buddies and say you know hit the fan and done and then come back tomorrow and let's talk about where we're headed not where we were headed so the less when like a big project folks worked on for a long time changes does open-source help this mitigate a little bit um so open-source as a as a as a production of a tool or as a consumption so as a consumption it's it's an interesting problem because you pick up somebody else's box and you pick up somebody else's forward progress at the same time if you're in that deep with a particular tool it's very useful mm-hmm if you're producing open source then the the question becomes I'm pivoting away from this who cares yes and if it's open and somebody cares they can pick it up and run with it so that you can feel maybe less bad that it didn't turn into what you wanted um maybe they can take up the slack on it enough that you'll keep this other thing around mm-hmm because it doesn't cost you anything to maintain an extra off side thing that's not your core focus mm-hmm and I see companies do this a lot where they spin off attack that they're no longer part of therefore business focus mm-hmm and there's a community that's been loving it and they pick up and run thing and some of that core tech just you know languishes forgotten or get hard yeah it happens yes I think Apple sells GCC with it's all this editions open-source somewhere you can find it you can build it if you want to yes but I'm but I'm thinking that you know what open source was one of the best things for engineers because their work is not locked in so if somebody let's say does not agree with you okay fine yeah this was this was a big piece of learning for me and let me go down the story here yeah I did some awesome tech at a zoo all they've got walk behind the paywall out us all a lot of it was never brought forth our we tried at some point but it was too difficult to bring it out and about into the mainstream Oracle JVM even as I was you know starting from hotspot the same codebase and doing obviously you know things that a decade later orbitals now to me do to CGC those all has been there ten years ago how fine so so one of the warnings to me was if a company pays for something that's theirs fine but if I'm gonna do a thing that I think is cool and meet and maybe it helps my company I'm going to very carefully delineate when it's open source and when it's funded by the company mm-hmm and never again let it get parked behind a paywall you mean high-scale live and you know mostly non-blocking cached me app is that that thing I had definite push back at Azul systems and I said no sir I'm doing this on my own time you do not get to claim it as Zul property this is Martin mm-hmm and then when I left is all I still had you know ash man and so I've been careful with that mindset going forward mm-hmm h2o I pushed for it to the open source we went back and forth actually between Apache v2 and closed and and a more broader open source but I wanted it open source so that should I leave h2o and eventually I did you know I walked away with the h2o of attack nope and then I used it at no Renzo and I used it in you know personal projects internally for a a number of times so would you advocate for kind of dual development from the start when they start the company you basically say if you're in the space where you expect developer adoption would you say that it's useful to start if it's a dually company funny car they might want if you want developer adoption almost guaranteed have to have an open source piece to it because for developers to buy into your tech and depend their lives on your tack they have to sort of get in bed deep with it they have to understand live your bugs fix your bugs tweak your stuff to fix their problem space it's never a straight yeah mm-hmm this goes straight to work straight to a different thing I have about you know when do I pick up third-party lifts right right I picked them up very carefully because it's almost always cheaper counting the learning time the bug fix time of the other person's stuff the the unknown whatever they pulled in and pulled in and pulled in versus targeted narrow Custom Fit solution it sounds good to just grab and run in my experience this never pays it not never it rarely pays out except for where you have very narrowly targeted libraries that are very well debug simply old been around the block have a narrow focus in life I picked those up mm-hmm broad spectrum things I pick up extremely carefully which was part of the problem was selling h2o it's a broad spectrum tool if you're solving your bumps with it you pick those up carefully because it hasn't been around 20 years right and therefore you can't had the open source to get adoption of all you know gated bricks and spark with the same rod yes and they I mean I witnessed first fans what you mentioned how engineers got excited right like there was a point in time Hadoop was slow and clunky yeah and right in you know like it was not strongly tied because you dumped stuff to disk in the red of the back right and now we have spark where the cluster is a personal computer and if you like program basic again except the whole memory and so and people got excited because in your thing of stuff like that map is there that filter is there right basically you think of a method what it should be called right and it was right there and right and so I think it's hit really hit the nerve and I think I don't know how often how often do you think this kind of happens when the what I call developer champions pick up the technology and then the targets are yes definitely happen h2r is it is it something which is what would you say like this should be the way droppeth engineers or is it is it the fluke like but the the you go get any of these sort of you know engineering software engineering team whatever how to do it and there's a whole lot of people will say you know there's a champion model who there is a champion I think that model really works and by the way it's cold put my extra jacket on yeah so the the reason the champion model works is because there's too many disparate pieces floating around or the technology is complicated always you're working at the edge of what people humans can hold for complexity in their head so somebody has to bring forward the unifying theme and and play what I call the dammit man wrong when the ties are there you can't really tell one tech from another dammit do it this way mm-hmm and it has to be somebody that people admire enough that they'll buy into it and go mm-hmm once you have that model in place then technical fights between engineers are resolved by going to the champion you know sysd i'ma do it this way oh man it helps if he acknowledges yes both sides have merit yes I can imagine doing it both ways I don't have a obvious winning solution but we need to pick one and focus so marching orders I've taken your advice here are your marching orders please go March this one mm-hmm right and you need to acknowledge that the other side has merit this business into you know self-awareness for introverts and the problematic young engineer is not have them not being able to see other sides well or not being able to gracefully state I don't believe your position maybe because they are at a loss words how to explain why they see that has an issue mm-hmm it's fine um it does mean that the more senior people still need to have a trusted person that is the champion that's this is how the project is going to go yes yes and and I think this is it is great like let's say you were an advisor or you're actually involved yeah you can be this tiebreaker easy you know that started will be fortunate but if it's the right guys have about equal seniority and one wants to use this and the other wants to use that that can be a personal conflict right and now it depends on how well people work it out and there are a couple models I've seen happen here I've seen a manager who's clearly not a technical expert bringing tie yes because he didn't know better but he knew the fight was no guy yes okay what then happens is that the losing guy can get pissed exactly and walk off because it was not done for merit but for political reasons to keep things tight and quiet yes to get things quiet because productivity comes for engineers especially when everyone's basically quiet they can't handle the show of emotions well yes so you need to have an on emotionally but not violent but too active environment right people want to be happy yes okay fine not the engineers in general don't necessarily do that well express thrown happiness or sadness or handle their emotions things build till they pop the pop is ugly find it happens okay so the manager is reasonable you saying I'm gonna have to break the tie to get calm but that means he's gonna lose a senior engineer who's pissed right doesn't necessary want to go the public has less people skills on the winning guy because the winning guy got my political support the real issue though is how do you not lose a guy when you're a manager in this position right how you not piss off for the guys you need find a champion that both of these guys admire one of them may become the champion the loser may wander off the remaining guy may become the champion if he picks up even a modicum of people skills mm-hmm but his solution also didn't works people around like trust hey we didn't know how to handle X we went this way that was Joe's solution it worked Joe has something going maybe it's potluck you know maybe it's you know next time doesn't work we get a couple of those under your belt Joe's the champion yes not necessarily the best way to get there actually it works but right but you're there now yes and that sort of you know that's a way to understand the manager side how he picks and why he had to pick you'd rather have that champion come from the get-go usually it's thrown internally usually somebody who stands up and says no no no blah blah blah blah and and they have something sharpened and they're ready to go and they're running with it and as they run with it other people like oh yeah that's our cool idea and they pile in now you have the champion the honest way grew internally and there's no conflict of interest in talking mm-hmm I wonder kind of link the back going back to history right so so can you kind of think how this applies so I'm very curious right Java really took over the world and you had really your fingers on its pulse and success but even not like you said you brought into Sun to hell the scaling problem right right and so how can I talk a little bit about this how did they realize they have a scaling problem how did they fine you how did you get into position you can actually enact actual change and something is really one right word yeah yeah yeah okay so um or so Saul who was very early at Google and it's still very senior person at Google grabbed me because I want to say he saw one of my very early PhD talks but I don't know they saw my appendix is how to go compiler friends mm-hmm and what he saw at Sun was Java the language needed a better compiler and what really had happened was Java was Suns attempt at making a write once run anywhere language so they could keep in the hardware business against Intel and Microsoft when that combo was winning the world and they're like no no we have to not die to this I want to sell smart chips in Solaris not Intel on my pocket fine so let's get people an alternative language where you wrote it it works that'll work on us to fine that was not running fast they got Sun grabbed the self project folks David I know I'm gonna get the wrong and I know no keg chambers they grabbed the the the self folks which were running a startup called anamorphic that was attempting to combine on-the-fly JIT compilation and the nice self environment which was this very cool fun GUI and the nice for a minute so they said pivot over to Java they started doing that there was a bunch of grad students who were sharp guys now very senior but at that time they didn't have any real solid compilers class I can cogeneration like I did when I was a teenager my very early college years they could issue code make it run but didn't have formal compiler skills for doing high quality code generation as a consequence code ball he sucked it was better than an interpreter it was sloppy or says this guy knows his compiler tech inside now let's pull him in and he pulled me and said go make java go fast mm-hmm in parallel so this is a sub me I didn't I came in at a stage there was a job of thing running around but the the golden trinary hadn't hadn't built yet that was Josh Bloch doing the the lips hmm and ugly doing the memory model and me doing the Jets I'll clean it was the the perfect storm of those three events that make Java away you know take off well claps the frequency scaling met parallel which meant memory model and parallel code generated as you parallel programming model whose Java have exceeded not mm-hmm right okay JIT he meant you had this run anywhere thing and you got rid of one of your issues about how you made a binary and had to ship it you deliver it distribute jar files yeah it's not more convenient and then the libraries were like fantastic so there was this perfect trinary I became one without realizing it I had no clue that that was what was going on until until I had hot spot like 1.3 Eve's you know the compiler was working for about a year was pretty buggy but when it worked it worked pretty well bailed out a lot of things but you clearly getting these 10x or not in 10x or not kind of thing was going on you moves the time you had a 10x and you fill in a pothole and it didn't comply uh people were all trying to work out the bugs and how to dodge the pop was what if we're all working progress and I went to Java one to get a talk and it been a while since done a talk but I knew I could do public talks because I'd done a bunch for my pee cheat presentation I'd been a bunch at Sun where I told you can't cheat code on the fly no compiler to do this I was like yes I can of course say this is the no compromise cliff course I can do this I gave a talk I was like to either in 1999 or 2000 Java one and I went to go stand up on stage and I literally couldn't get into the room because the mob of people trying to get in the room exceeded the capacity by more than double and so I was like pushing her way through the crowd and people say hey hey go wait in line you know you can't get in the room on the speaker so you know and that's the point I realized that something had changed in Javas acceptance in the world like people were really in love with it and they wanted it to go fast hmm so I gave that talk son basically rescheduled a following talk emptied the room forced everyone out then let the next set of people in they filled up over capacity like the fire marshal came in and stopped it at 400 people on a 350 person room they had another 400 people on a three from pushing rail and then the end of the week I give it a third time I had a hundred a hundred figures of us thinking I'd show up at yeah and then then we had a - hey after I couldn't you do Q&A these other talks cuz they just like shove everyone back out so the next crowd could get in fine so that's what I knew that something had happened and I didn't quite get it yet but it was important so I got my own self goodness out of this you know my ego was massively stroked Here I am pumped I'm gonna make this thing go so you know I doubled down I blew off Google I would have been employed number 50 cuz or said hey come up with the world well I'll see how that worked out that was the fame versus fortune decision that I had you know internet search he's gonna pay for it in a search so lost that fun yeah my nose of there several times and kind of didn't find the right fit like the the the groups doing languages didn't want me in that was kind of a funny deal because I think I would have displaced whoever was the champion and the new European didn't want that yeah I get it but kind of sucked so then I'm like oh I can go run some sort of email side team or whatever I've been doing tensorflow maybe you know the first love learn all right which is he's a language guy which is are interesting to me okay you're right and so the creator of Swift Ian's desert floor and so I think his whole which is very exciting that now that you have swift bindings for tensorflow suitors obviously better for us happen Python types which actually mean stuff and and and and he can add automatic differentiation to Swift yes all right they have the swift important process but obviously since they're a very strong group so I'm super excited and actually I was going to ask you about it because maybe maybe unless now because this this is a very interesting development which to me super exciting right because what we've seen with the rise of data science is the right of 12-week boot camp or developer who basically pivoted and this is great that how to do that but the folks who come on the data science generally do not have software engineering experience as folks would do decades of this to whom continuous delivery is second nature and things like that and suffering because of bugs right like the normal mode of developers to be depressed and things are not working right and it's not like things are always working and do them more acutely yes right these are generally in symbols and their work this is the the good state right and sometimes so to me it's very interesting that some have this culture of social engineering data science and and and flourish and Firefly it has a different kind of part by the way but maybe I don't know that you want to go through now it stopped - mid-sentence you come back to this one yeah and so so again this is my theory which which I see the software engineer right like I'm a computer science I have a PhD given of science but my favorite algorithm is sort unique take top ten and it's just a question of what you sort on right like it's really in the end it's all that it's it's just that right and so in the questions like you really need to source a lot of stuff and so so so this kind of my my thinking I always look in you know like I asked in the room who runs web-scale system on a notebook on a pattern or book is there anybody is there anybody who runs any real-time API and ipython notebook right so so that was the let's not there are two different malls beckon people hardcore by beckon people and compiler people used to be the hardest score all the back end people and so what we have now I think at switch to filter flow project is where compiler people are brought in to bear on them also huge in Burma Kaka who was the language to a Twitter during Scala he moved to tensorflow team and what they're working on mly our machine learning intermediate representation I think that the immediate goal of that is to basically say because they meet two GPUs GPUs and CPUs it's not really the job of a data scientist to optimize this so they write but now they get deep into machine learning and they take over and tens of hours actually data intensive application built on dataflow so and so they take that and then now built in term in the presentation to to optimize much better than manual right and so so what is very interesting to me now we're at the beginning of the era where we take this fragile ways from the kinds of some of the data scientists and put it in in the land where you can reason about code itself from compiler techniques and obviously from my standpoint lives to safety right like if you do it in in that type safe verifiable provable mode and obviously you cannot entrust your AI data to Python right like this is he can doing that for 20 years but why would you want to end up with the fruit of your hard labor the extraction of behavior customer behavior financial data you deep secrets into a blob of slow goo which and doing it for 20 years there's an obvious answer which is because this other shit's both brand new had has been shown to fail repeatedly and crashes all the time it's very complicated to set up and run with it's not my easy understanding easy manage each control Python open right right right so there's an obvious answer why I'm not claiming that there's not a better way but that there's there is a passion for all compiler engineers that you can solve all problems with compiler technology mm-hmm that's not necessarily true yes but there's a passion that you can yes and it's also the case having done enough data science to recognize that there is a kind of a typing a concept that goes with data science where I have a pile of data but I know things about it that I want to carry forward that knowledge into my next steps and the typing of it would be things like it has a different kind of mean it has a normal distribution I have done sort of imputation of outliers or missing values I have done a bunch of things too it is a certain quality to it I would love to describe that quality in a native way is understandable by compiler hm so when I go to the next step in my my thing I don't pull out an algorithm that will utterly fail if I don't have a normal distribution but I you know I know I have a normal distribution already or I don't know because I'm validating the data and I discover along the way that it has a blend of distributions I cannot then apply certain kinds of technologies afterwards or the math is wrong and I mean you know garbage in garbage out yes there's some compiler tech that makes sense around data science I have the compiler side of it I don't have enough data science other than recognize that I routinely would work through data flows where I knew there were certain properties that it was building on and building and building on and I find missed steps or screwed up or skip steps or didn't quite him the the tech not the math properties correct the following step for garbage that didn't break until five steps later it's a classic debugging problem but on data side of things never stop trace never crash I just got a number out but the number was garbage yes and I had to go backwards why were you garbage well eventually discover this was oh you were garbage you've got garbage and then you rolled it back and oh here I injected some sort of you know a common one in FinTech domains doing any sort of predictive modeling of markets as you bring in history that you're building your model from into your current prediction in a TD way and that's you know gives you this awesome prediction which doesn't work tomorrow because you don't have tomorrow's data to make yes but but it's very easy to accidentally do that mm-hmm okay fine so all just means that there's a there's a obvious crying need for compiler tech and data science so brick this thing it sounds like you can benefit all right in fact there's a lots of it back testing is an example or you can totally benefit from it yeah yes but there are other examples floating around I totally applaud these guys as efforts um but he asked why would I because I've done so for 20 why would I use by their good dunce over 20 years and up until now it's the alternatives have been difficult or garbage wanting things that needs to go did was we only built sort of top-notch algorithms that were hard verified by mathematicians and data scientists that handle all the corner cases they did not fail silently give you crap answer if they were going to fail you've got the polite we're dead now you know this doesn't work mm-hmm and that was one of the big changes that I mean I don't know where SPARC is now but decade ago that we could clearly show the data scientists didn't like to use smart that we talked with des Mikey's party because the algorithms were not the sort of top in one's if they failed they silently gave you garbage instead of reporting back correctly that I'm about to give you garbage mm-hmm-hmm but you guys have great statistical advisor tips Ronny and we totally yes and in house we always kept mathematicians on board full-time whose job it was to validate algorithms and pay attention to them we made the made the engineers like change algorithms painfully in ways they didn't want to because they weren't doing the math right you know the algorithm in theory kind of worked but if you want to get the math right you actually have to do the following thing for the following reasons that are not obvious and they were painful but once you did it then you would at least get the you know the big red flag would come or the algorithms it would say something you know I failed to converge as opposed to I converge to a garbage number right and the thing here you being a mathematician the origin problem helps right because that's the mathematical mentality yeah I know I not had any mathematic training beyond you know what took the galactical engineering degree mmm which I didn't use for 20 years until just now I mean just for h2o and then yeah I spent three years hardcore studying math so what the device will do give to programmer to scale to so so every time I write a piece of code I go backwards I get it more or less working I go backwards and look and say how can I clean this up and tidy yet mm-hmm how do I do tech that illumination on the spot and with that comes this this notion of doing very local only transformations mm-hmm with the goal of reducing the code size and and cleaning up how things are used cuz when you write B's code you don't know worry how you wanna get there but you know you can get there so you're scrambling for you're hunting through some jungle code you got somewhere you never go it in a straight line you always wrote something crafty some dead code you have some false starts you have some left right turns that don't you be there you straighten it out that's tected illumination fine do it even on a very small scale practice it because you can get good at it and get quick at it and when you get quick and good at it the code you write just becomes sharper even moment by moment and every day I do the the practice thing of why all the code I write I tried it what can I do better here how do I shrink it how do i express this and every day I'm looking for what's a new thing I'm going to learn today about the land of coding and it seems kind of trite and it's old hat and I've been practicing it for you know years and it pays off so yes it's trite yes you hear this advice before yes it's there but yes it works burkas practice purchase purchase um not just practice but practice intentionally right so there's there's there's I did it there's I did some practice there's a practice intentionally and the difference is you retrospect on what you did and refine as you go forward you same point I give a talk I give a great public presentation I get best of show words all the time keynotes blah blah blah a month after I gave the talk I go back and replay the talk and I look at it and I look for what I can do better mm-hmm so I'm not just practicing and intersecting on what I did both my code and my talks and all kinds of other interactions and there's an intersection that lets you refine the process so the next process the next time it goes better and better and better and better and better until you look awesome amazing and you're the cooling expert and how did you get there well here's how it was during of a thousand steps but you got to take them yes I'm cured because you said you know you're an introvert right and this sounds like a very introverted process which is kind of like a Rhema Auto regression right like you you cannot do this without external feedback right you have a feedback loop yeah so I wonder how and obviously a lot of people indicate extreme feedback in terms of fair programming and obviously like in our industry it helps to look at smells car your pair programming it does a useful thing yeah so how do you at your level you say you learn new things about coding every day where do they come from who are the people who are from do you read blogs do you participate online discussions what's the best way for you at this point to learn so yeah so so so I crossed the number of thresholds that made it more difficult to learn so when I go to these conferences give talks I rarely sit in on the talks because I'm not learning anything mm-hmm so I have an open mind and a wide reading list that's kind of hit or miss stupidly more questions come around and the basic questions kind of naive but underneath it there's often a thing going on there that I can then drag out and my so many years of coding I send the right guys oh he's falling over because of yada yada and maybe I didn't learn it internally but I learned it in my head mm-hmm and the learning in my head means I could put in my mouth and put my words my slides hey this is a thing that programmers do so so ten years ago I couldn't tell you like I give a talk on on the different modes or paradigms programmers program in in its new code generation its bug-fix saying he has detected elimination a sort of an obvious way to break apart the world and those different modalities should and could and do have different ways that they acting behaved and different expectations come out of them when was that gonna die understand that it took me a long time to bring that to an understanding and then to proceed that convertible lights even silver wasn't so poor the most about myself now that I learned years ago so he's froze by questions from some people right now yeah yeah and I still go find go read look at stuff and get surprises I was like I'm designing a language I went and found this web page that was how to express the following interesting programmer things and every possible language like 30 or 40 languages hmm make a new object have a gift and elves test to find a function and declare type to declare a variable declare the sign of audio blah blah blah blah blah it's fascinating to read things because you can see that history or a people screwed up or try new things and whatever and there's a lot of learning going on there you know forgetting stuff out doing h-12 a lot of where my learning was strictly new math it was math and I was reading papers and I was talking with these profs and they were explained they step you through it except that you didn't learn it but you got feel good if I would work the ball and now it's time to go do it to make it stick yes learning happened there use the cordless right because when you listen to a famous guy like hipster on you feel it understood everything yes because they're nice people yeah yeah but one those who left yes it's like now we've have to call the talk yeah how do you do that exactly no no I do that to other people I stand up say this is how it is it works very nicely and I look and did anyone else do it no mm-hmm like I tried to explain how to do you know the the azores role time TC repeatedly I stood up and said this is exactly how we doing I'm tell you all my secrets mm-hmm it was a decade for Oracle even try okay fine um same thing for all kinds of stuff with compiler tech whatever fine I get it so trips on e stands up and says this is how you do it that's very nice got stephen boy very nice guys explained it was so easy you have to do it doing it was both tough but then I learned my god I drilled into me some amount of new learning about distributions variance and the properties with all kinds of mathematical things there it was a very very useful thing when I go in the FinTech world there's a lot of new learning about about how people feel how the stock market funky marks yes pretty amazing yes because this is David he's coming in from different sausage-making you know tastes delicious but don't look at how it's made it's pretty pretty nasty so I found plenty of fraud in the stock market once you got past how you understand what it actually meant how the market actually works it was it kind of shocking then I did a lot of personal day trading basically week trading whatever you called it for a while that actually was both full of learning for me and you know interesting aspect but didn't want to ask for my life although I clearly could make a living doing that mm-hmm now I'm doing IOT sensors for distributed location since saying all kinds of sensitive stuff mm-hmm but but fog edge past the edge and back you know that the in sensors leave the Internet really and come back and leave and come back and leave and come back and then you want a funnel maybe I'll do something with a and so there's a lot of RF radio rotations going on and a lot of Bluetooth spec and a lot of there's a lot of envelop lying on the day that's coming in as well and so here I'm learning a new a new domain again where we're not coding per se except that I'm learning what it takes to code in these different funny domains but after a bit I can route them into Java during data science and a distributed server for the central piece of Ted I just know very you know that just like here I know what I'm doing other parts there's some store-and-forward funny Network thing where you actually drop no build oh just due to key value store where you can drop nodes and now drop them reliably routinely and pick them up and drop them and pick them about them and do not fail to get out of incoherent right it's a it's a tricky problem and that's sort of where we're at Oh interesting it's you know it's it's it resumes what I've been doing for last year I was a few committee officer for us at IDT Alliance which is industrial blockchain consortium it's it just got merged into industrial internet consortium so they worked with a private blockchains for cars like BMW and Mercedes and Porsche and Jaguar and Sue rent-a-car from Barcelona to Berlin bicycle charging painful electricity yeah so there are very legitimate use cases so my goal was to find very legitimate engineering heavy non crypto application of Longinus the distributed system but I also saw the 4-h so this is very interesting to me so I talked to Brian control a few weeks ago and and we talked about the age and he basically said like the age is empty not because lack of trying but because it's so hard so all right we don't see you know leaders lead immersion on the edge so my kind of metaphor is just like dough fell over the edge right like that yeah interestingly different problems there I wonder what you're taking this right so this is is this basically a distributive system and some people say you forget can eventual consistency forget it people not to venture the system that's never gonna happen just let go all right instead of sitting atomic clocks on top of towers and making it all sink differently this is clip saying you can't do this but I flips like our question here so I think I think I really am close to exact consistency for things that are at the edge with or when they're connected mm-hmm when they're disconnected they're clearly they're clearly getting scalar by the moment it may be a date yeah yeah totally Yeah right when they reconnect there's a period where they're still admitting that they're disconnected because they haven't yet been brought up to spec and then they come up to speed and now I can be exact as the time flight distance to them back and I can give you a trade-off that says you shoes the amount of lazy you want to be versus exact with a dial number except that if you choose to be exact the late T will instead give you a point where you don't know so late exactly will be you are currently exact or you're currently you don't know because I'm trying to achieve exactness mmm-hmm when they achieve exact us I'll let you know and that time span will be the distance and time from the edge to the center or back mm-hmm okay so so I can give you an exact and I could give you a lazy and I gave you a dial a knob and that's probably the limits of the laws of physics but I can totally get you exact fairly reasonably when the device is brought in to some reasonably close or connectivity status mmm-hmm okay maybe that's good enough so I'm curious how do you see this evolve it so do we really change the nature of computing so you know currently the models a data center right and so again Brian control just launched a company called oxide computer where they build data centers for everybody in the general purpose way not customers Google but obviously I think it's a great idea right because you should push all the per source finding all open compute open firmware and engineer hardware software so I think is that what you guys down at the zoo so I'm actually very curious about that as well alright so so there is a question of scale which kind of measures together software hardware but at the age it's all dispersed and and I wonder what you take on this right so you have a data center which optimized by essentially using software and hardware and like now everything becomes software respect risk 5 in open source architecture right so like software is basically taking over hardware that's what's happening in the data center okay you clearly have hardware and you have a base layer that's right exactly it's exactly so so how how did you this evolving datacenter versus the age I mean is the edge just like the data source is not going nowhere like like I mean no this clearly it's not going anywhere that's right yeah it's cost effective to have centralized heavy duty computing with high power consumption high cooling cost and there's just like a bunch of eyeballs I mean you can have like a massive I felt like device I'm talking to edge I'm talking about thingies that are that are meant to be disconnected routinely mmm-hmm so they are battery-powered very low power consumption very limited radio distance band things in limited limited bandwidth when they're connected unless they can touch something other than beacon so is there any computing company there are there's computing there so you can push a model down as a simple model down into them and then can they can raise a fly so I can totally take the beacons out into the field as they come and go and come and go learn from them mmm-hmm build a model distribute the model back out to the beacon and the beacon can raise a flag if you pass if you exceed thresholds hissing but he's still disconnected so at the point that he raises a flag probably the primary response you want to do is get connected mm-hmm and do something more intelligent now but the flag is there to say it's time to do something okay and the first do is get connected and then the next do maybe replace the failing bearing on your you know your gas turbine and for the seein the town and some backwoods India is failing and in a month from now you're gonna be out of power and GE needs to send an engineer over with a new bearing mmm-hmm and the same theme runs all over the place for these kinds of devices so you some basics the needs and then the updates they're limited understanding yeah they have a limited understanding based on their local sensors so you know shock and motion temperature humidity these things integrated over time because they have you know a megabyte or couple Meg's moon so they can hold history in their hand so they can integrate over time and and they hold the history so that we get connected I'll tell you history as well do you see them becoming iPhones because some school of age right everybody means different things say it like this is all I fault everything is gonna have five varieties the answer so these things keep keeps shrinking and moving further out if there's a hill you know that we're pouring technology to top and it goes out okay here there's data centers and high power consumption here's iPhones or phones on there they're shredded and they mostly connect and occasionally disconnected kind of things and they have a bunch of commute technologies around them and then there's edgy things like I'm carrying my backpack which are you know this big and intended to be off the edge and on and off and on and you might imagine things that are even further disconnected like a NFC tag which is a tamper resistant tag that's just not going to ever be connected except when you go through a point of sale and they're gonna look for the rip and the tag that says it was a tampered with right mm-hmm okay now run the same technology curve ten years out the data center is even spicier and more consumption for more for what for blah blah blah our cellphones between ever more intelligent maybe they shrink and we're only wear them in here whatever I don't care things that have cellphone power go to my right thing and now it has cellphone strength maybe I can friend put a cell tower connectivity in this thing it has no power connectivity I can't all right I eat too much consumption maybe maybe a decade from now I can't I don't know right that's the size of my you know my fingernail my pinky or half again that I'm dusting around in places that is disconnected and reconnected distant again right so I think that the the push will keep happening and the curb will keep moving out we keep pouring tech onto the mountaintop and like porcupine molasses its snoozing out into the field wider wider we'll see we'll see so kind of the end of the segment I want to kind of go back right for there and so go back like if you can so this 15 year old boy pool yeah decided to the right Pascal compiler which is very interesting to me right because I actually one of my first language is also for sure because I grew up in USSR which was behind the u.s. I look at Portland for but also have some strange Soviet version a Fortran code Fortran gather'd EGR developed by German which is like Fortran 77 66 plus and therefore 277 so which was exciting to me and then I got my first you know true bypass call a matrix sei was a game team I'm a sex computer but you did not occur to me to do a Pascal compiler I was interested in Nicolas overloading that have this modules issue we can write a disc and the disc was the 800 megabytes floppies and so I could actually connect two of them so I was kind of more in storage and how right but I'm curious so it's like I wrote on a Radio Shack trash ad with a cassette tape that's right this is level 2 basic on a 4k machine there was the first passage with Pascal Rho dot P code and put the P code in RAM and then I swapped the basic program out and left the pea coat and ran home and the second basic program I loaded in took P code and emitted machine code which I think I didn't either spit to desk as a file or disk cuz I tape yes or or execute in place right and my only real programmer right or bunch of little toy ones was a breakout game it had sound and paddle motion and had the classic breakout thing on a Radio Shack crash 80 graphics written in Pascal interesting so so I'm curious right so let's say like yeah we want to help folks kill themselves and their company so I'll I got the physics I'm interested in starting condition right so yes this 15 year old boy and you decide to write a compiler and you basically stick with it for your whole life yeah right and you're like you can with different skills but you you were fortunate you recognized you've got this passion yeah right and you ski in the you basically a lit the whole life right yeah so how do we kind of enable more people like let's say I'm an engineer and there's so much stuff going on right so I'm kind of gravitating towards taking some stuff and my bosses tell me to do some other things right but I I I'm kind of you know okay I want to do my thing so I'm curious like you were fortunate but you also worked hard to get yourself to the places companies projects where you're your core skills right I didn't have that that lineup that I had the paying job that didn't align with what I wanted to go do so this is this is the hole it's your passion there's no compromises mm-hmm so you have to figure out how to make it work but you can't lose your passion or you just as a person you'll hate yourself so you just have to keep following your passion in there so how do you make that work yeah we're a couple ways and and where you end up going depends on your life situation though I kind of stopped here one is to say I have my after work hours and this is what I'm doing when I'm off the job and I very clearly make a distinction this is off company time boss it's not on this piece of work it's my attack yep and I am doing it for me yep right and instead of going to see movie or going out drinking party on Saturday night I am doing my passion right then any artists worth their salt will totally agree and know this is what they're doing for you just the opposite of what the Silicon Valley emphasis if you do them in our employment on our own machine so I think normally now you have to actually change the culture because they don't say on your time they say if you do one company prop it's like it's a fuzzy area now they don't own everything oh yeah right so so the first thing you do is you understand that most employment contracts that you sign are completely illegal in a variety of fronts mm-hmm um there are some company secrets that you're not allowed to walk off with including customer contacts and the like fine but anything that says that you're my personal slave and I own everything you do is like Yeah right okay fine so don't sign those yep and upfront say I'm not going to sign this button I can sign a nearby contract I'm just gonna redline a few things and you NASA buy and it's okay and if you're you know if you can't make that happen because you're too desperate for this job but the coming won't do it and you're not just literally starving don't sign it this isn't not a good place to go work yeah okay if the company most and will just do it you know if your junior guy you need that first job you might have to figure out how to make sure that the thing you're doing for your passion is kept remotely mm-hmm it's not too hard to make it clearly I only logged in on my own machine my home time I only walked into don't do it at work don't work or see loud don't you do it some amazon don't ya don't touch any of that stuff personal machine my own personal Wi-Fi connects to you know do my laptop I push to github unfortunately github account right don't touch anything on anywhere in the company nothin and that'll that's clean yeah I wonder what's your take on this because normally when you sign I PE contracts there is a list of Secrets which you already know you should put in assumption right everything you put there is yours yeah if you forgot something right how did you of this um I put the big ones down right away and I mentioned that there's lots of things that I know of already mm-hm that that I still didn't mention everything and also those contracts also say stuff like if you learned this from some other place then it's the burden of the company to to show that you stolen yes okay so if it comes down to a legal read like like the worst thing happens here is that you piss off somebody who's not very bright about what these contracts mean anyways to threaten you you can take these things and more or less throw them and you know that you get a warning from somebody throw them attractive mhm if it really comes down to a legitimate fight besides getting obviously real legal advice um 99% of things is to say is company secrets you stole you can show we're well known publicly beforehand and you just hit Google once and you got it you turn around and say here dude I got this from this place right you know take your shovin right yep if you walk off with company code and somebody pattern matches their code to your code you're an idiot and a dishonest one hit the hide but if you write an interesting algorithm um and you go and rewrite it somewhere else there's going to be similarities mm-hmm um but that's still available algorithms are not better still practice still legit use yeah yeah right interesting so and finally I think my last question will be very simple so scale is a phenomena basically which is the holy grail around here like everybody you want the thing which makes it Google which makes you cliff right like which makes you something so you know you want to be sparked you want to take over you know minds of thousands of programmers so they contribute and yeah so when you know when we see these things we know rust a lot of passion I grant immunity a fond of what rust is trying to accomplish right I hoped in the best and I'm looking hard at what they're doing you know so so we know these projects insult like there's a flow as an example right like it's some some how would one there was a bunch of other stuff and spark knows something called Scooby we don't remember that but the weather actual question we have this big data framework Suginami so do you think its nature inertia is this yeah are you lucky I mean will you borrow this is Google just a bunch of like super scalable people or is this something we come to convey thoughts raising four kids and I raised four as well there was very clearly some some nature and you know there's some virtual chair so yeah so there's some work in life as well I you know I think a spark had the right idea at the right time yeah they had a champion he was pushing through a better way to talk about data flows and open data science just flows the data yes and and that just resonated with people that was like a really cool concept and that lets you sort of work with data flows which immediately lends itself to data science and in a ways that were just not possible before I think that was so you know that was a Mattias just did did a cool thing Google came up with Internet at the time the internet was was busting and that was just this ultimately powerful thing they have built an amazing company and they had built some really cool tech afterwards yep but their Keystone their cornerstone is still the surface yes right in the implementation of it because they builded an imitation part of the implementation strengthen is out the hardware engineering guys could go crazy do their thing and the engineering guys had the passion of scale mm-hmm right and there so they nailed it yes and now they're looking for I have scale everything's you know I have a hammer everything's a nail right they're looking for everything nail with scale mm-hmm and they're finding things so so there's a there's a cool thing there although I still thinking they haven't found the next thing to replace internet search from their business point of view they have definitely built really awesome tools that they're doing really cool things with you know now I can find all the cats on the Internet yes separate from the dogs yes so so instead of just to expand a bit so you basically kind of the super city or and what advice I think as engineers were kind of understand what engineers need if you're advising a CEO what would you tell them if they want to grow the culture scale is they want to have the company scale through technology what should they care about it's a different thing and this is a position I haven't been in so the first thing you have to do is they have to get the business right so mostly I've been in small companies with this CEO has had trouble getting the business model right mm-hmm and you know at age two I'll clean that I saw the problem but I didn't have an answer for my CV ah he and that was part of the reason that we weren't necessarily the right fit at some point at nur ins ik I'll clean we have the right market fit but it was a distributed team and I couldn't rein in a crazy CEO who's doing the wrong thing I have not been in a company where the technology in the market fit had happed has happened and we're now growing gangbusters and what do we do mm-hmm Saul and Jemma I was not a CCO not right all right it was not in that position rather certain but yeah like if you go back all right like you know what's the gray oh yeah I'm is like right yeah right what's the example from this interaction what's the optimal kind of interaction of business and technology where you grow through technology so so it's still case where you have to keep the engineers happy and rolling forward and enthused and passionate some of that is you explain to the engine and you can totally explain engineers what they do and how it impacts business mm-hmm because this then in turn empowers them to make decisions on different tech they're gonna chase to better fit the business you're looking at mm-hmm right because as a business guy you might see the opportunity if only we could solve this problem but not understand that it could be solved or that there is a better way to solve it mm-hmm when the engineer guy just doesn't see the business side piece of it there are some cross communication that I routinely see not happening that I try to make happen at every company run so that the the engineers get both the risks they're taking at being out of startup and the right theme to go solve for when you're trying to make the technology work I'm still looking for the right fit for them both while I'm in the CTO role and I mad too bad CEOs and a bad business model not work out I guess too bad business models with okay people and too bad CEOs and a couple personal things I did where I completely broke even didn't wanna do that for a living but it was working I didn't wanna do it for later we'll see what this next one goes so how much do you think this year should learn about the technology for I feel free if suddenly we start crying for this no no I just start up the CEO should be stronger mm-hmm he needs to understand the limitations and strengths because he's going to be selling the stock he he has to come across as a sensible reasonable about it mm-hm to whoever he goes to sales guys you can start with guys who don't know the difference between a compiler and an antigravity belt and they'll sell every possible thing that you can't make but as soon as you know they get the words right in the jargon dialed in they'll be close enough you need to get the next conversation and the next conversation is you drag in somebody who's technically expert you have to really make sure that we know and they want to know that somebody believes right yeah here the sales guy is typically really looking to bring a connection in if you'd like to have a guy who's also an expert and could sell that's a very rare skill set but you can get in there that are good enough expert same as a CEO need to be good enough expert so they don't make a fool of themselves see who needs me better because he said really know this that this product will change the world yes he has to believe it will change the world yes right the sales guy only has to believe it will change this company who's talking to you all right all right so it's good it's good to kind of understand the technology how we can scale through it if you want to build a scalable company if you were easier see or you should see that if you want to grow most technologies like the VC still want you know and that's cool great I get it most these technologies I see these companies and startups going after are clearly not gonna change the world they have an obvious cap so you can pivot the harder thing is usually building the underlying base tech which has lots of applications over what the company taglines has a human right and wind and where do you pivot and you might be planning on pivoting all along the tagline it's just a red flag maybe but I see a lot of these companies maybe built some cool tech get it things rolling they they've learned they needed heaven and knowing when and how to pivot is a key one H so we pivoted um you know all right after I'm saying almost a year nine months to to machine learning yes from a distributed key-value store a better key value store yeah which was like great great pivot oh yeah turn yeah yeah right um but totally that was like a you know an eye-opener for me like gosh we can do this and it makes sense and has to happen next time around next time next time every company went afterwards keep looking and then that you built some cool tech but the tagline the company may not be it so you need to pivot and that's what the advice I give the CEOs when is it time to pivot because the business isn't there for the tack but the text hard to replicate hardening yet can we take it to somewhere else where we can grow better with it mm-hmm so let's bring this back to scale and I think it's a good wrap okay well thank you so much yeah I really appreciate it yeah fine and we'll see how you scale in the next iteration yeah we're in front of City College Susan skills all the time thank you [Music]