SBTB 2015: Tiho Bajić, The Rise of the Full-stack Scala Employee
Recording: SBTB 2015: Tiho Bajić, The Rise of the Full-stack Scala Employee
morning everyone this is an amazing menu and this is an amazing stage here I feel very very official you're not standing here I feel like with this type of alleble I feel like we're that you know party gathering here and this is some sort of a community meeting so I won't take long but at first before I share a couple couple of our views at nitro around the evolving open-source scholar community I want to just thank everybody for everybody with alexi Jason Jeannie everybody who helped organize this conference it's an amazing amount of effort that goes into this I believe we have more than 80 speakers lined over several days here just coordinating and getting everything done I know it takes a tremendous amount of effort and guys like Jason and Alexei they have full-time jobs and software software startups as well so I know I know this this is really really difficult so thank you lexan just a round of applause for for all the organizers so in April this year that was this medium blog post that that when someone tomorrow and I call it because it caused a lot of controversy and the idea behind this blog post which was which was called the full stack employee post and which by the way that's like an interesting thing I discovered was written by Chris Messina who is the inventor of the hashtag so when you today use hash big Scala you know that the guy could invent it or who started using it first and popularized his name is chris messina so he wrote this blog post about what it needs to be a full stack employee of the South their startup and it was very controversial because it proposes certainly of lifestyle and and you know kind of giving yourself all in but at the same time it was basically saying like there are these few people who who act like we're capable of being polymath and who act like Renaissance men taking they have multiple expertise they have expertise in multiple skills and so he was basically claiming that you know if you do to the evolving kind of measuring between disciplines you need to be a real polymath to get ahead in the software startup Rowland so you need to be a full stack employee and this meant basically you know being capable of doing front and work back and work understanding databases and saying design marketing basically the full gamut and when when we're looking at the world we live in today there was getting a lot more complex right we all we all know this you know we have you know explosion of data happening from everywhere all the connected devices all the streaming is that we have right now the you know incessant need for immediate gratification feedback and constant uptime 100% uptime global distributed systems and we have all of these other trends they're happening the same time where hardware is getting commoditized you know multi-core parallel computing is becoming more and more powerful and more and more prevalent so everybody carries like a supercomputer in their pocket now and so you combine that with what's happening in the open source world and this conference is really interesting for that and and I was just astonished last night when I was looking up this day that this is a report that came out recently black that open software survey for 2015 and just to reel off a couple of things here basically seventy percent of the company is run an open source four percent of the companies say that they participate in open source projects and they say that more than half of the workforce is working on open source now this was a this is a survey that ran out and you can actually go and check it out here how they how they went about getting this data this is this is really interesting because the trend is accelerating and you know open software is eating the world I just open source is eating the world and so when you combine those two things the evolving complexities and the open software that is eating the world you get this amazing proliferation of specializations right but they you need currently in a software startup right and depends what kind of a star that PR but you might be looking for these type of rolls or people who can fill fill the roles like this like big data architect site reliability engineers UX engineers machine learning scientists you know software-defined anything from infrastructure to networking to data center engineers microservices architects I mean you name it right like it is no longer just about I you know I need a back into your front engineer I'm starting my startup like the world is actually evolving really really fast and changing and so then these two kind of concepts feel a bit a bit in juxtaposition feel a bit odd because this guy's claiming you know you need to be a full stack employee you need to be able to get all these things you need to be a renaissance man and then the world's changing so fast and evolving so fast and you know open source software is eating itself a bit as well because it's it's spinning and revving really really quickly so how can we you know live in a world with these two ideas are competing and how do we as software engineers don't simply you know die trying to get and understand everything and so I believe that to actually be a polymath you don't necessarily need to be a polyglot and you don't necessarily need to be versed in too many things anymore it is all it is important of course in software startups that they have specializations that we have people who are capable deeply capable of doing of doing certain things but I believe that and this is a trend that we're observing a nitro is that when we train our engineers in Scala and use scallop our open source ecosystem you get to the point where you start talking about Scylla fundamentals they're actually bridging disciplines and this is really powerful because now you have this common language so that your research engineers like we have some research engineers from Penn State for example this summer who are working on you know tabular form extraction for pdfs reduction productivity company and in teaching one of our students scala and just seeing how he fell in love with full and how he's using the same abstractions that we're doing elsewhere and how that's literally the same thing that we're doing with data ingestion and data data pipeline engineering and so it becomes really powerful because now all of a sudden yeah like we need specialists right like we need people for you know who for example in our company we have an interesting deep learning where we have an interest in you know tablet data extraction or big data processing and pipelining and extracting unstructured data into something we can report upon but now all of a sudden guys are using the same concept and we are actually capable of talking to one another and and what happens is that when you when you learn these concert which might be a bit steep in the beginning to learn you fundamentally get how the world works and so you haven't you having a more more thoughtful conversation as Alexis said so Todd fluence is really key and so this slide I just tried to capture basically a whole bunch of things that we're looking at currently already using and this is a scallop our open open source ecosystem right so everything on this slide I believe is scallop powered and it's open source so you got you got you know all kinds of really interesting projects here that you guys would have known about but for example on the visualization and you have Sparkle which is amazing you have scholars yes I believe there at least two talks to this conference about this and this this is this is something that I definitely want to be asking Martin about the mass marketing anything you got you have web frameworks for huge fans of play you know HTTP integrations we've been you know looking at HTTP and spray DevOps obviously as we deactivate or maybe you know conductor our conductor is going to be open source at some point too will see that data access you got data processing obviously spark and fleeing everybody knows this is the top of town distributed systems and communications between those systems everything from Al Kavanaugh is becoming really interesting and some some things that really just it in at nitro are on machine learning and LLP there are Emily la ml lib libraries Scala and LP great Matt libraries afford for abstractions par and factory and of course there are you know very specific functional programming libraries they focus on abstractions like scalzi scholars and shapeless and so when you look at this this is a lot right now nobody can know all of this but what's really interesting is that the fundamental cons when they are thoughtfully explained they carry through this entire stack and it is really difficult in this you know exploding world and this competing needs on a daily basis to become an expert in this Renaissance man and I kind of don't fully agree and believe in this idea of the full stack employee because you know i think it's a it's a rat base potentially but if we can be more thoughtful and we can think about abstractions that that bridge these multiple disciplines in these multiple expertise is and we can discuss that that folks and conferences or events like this i think i think we can we can make a really really big advancement and so naturally supporting framework foundation as Alexei just announced it this is something we are really interested in because there are very specific open source projects that have taken over the world but there still needs to exist some binding glue some kind of get going stage before you can you can put them together into a single stack and so really passionate about supporting that and talking to all of you and their boot over there about what we need to do and so last year Mario's from theatre had this amazing opening keynote at scala by the bay about server as a function and that is an example of an awesome abstraction that just blew my mind and I've been thinking about ever since and so thank you for being here today please think about more powerful abstractions and how we can all have you can all share in common concepts if you have any questions about what we do we're over there by the booth I hope you enjoy the conference there are some amazing talks and thank you for participating in the open source scallop our community thanks you