sfspark.org: Brad Murdoch Lightning Talk
Recording: sfspark.org: Brad Murdoch Lightning Talk
there are a whole host of mega trends that are driving the world to change the way that they do business today digital transformation mobile business the move to the clouds the Internet of Things big data enterprises that are looking to leverage those mega trends for their business are realizing that the architectures that they've been running their businesses on for the past 15 years primarily at the back end Java EE and dotnet are just not going to be able to support this next generation of applications that they have to build in order to be able to take advantage of all these mega tracks and so this is some from some reports from Gartner the major enterprises are realizing that something's got to change the traditional architectures are obsolete Gartner is predicting that only 35% of new applications will actually deployed an application servers by 2019 simple bit of math means 65% are going to be doing something different all right what does that different look like well we believe fundamentally that it needs to be something that is essentially a cloud native design whether you're deploying in the cloud private cloud or in your own data centers the design of the new modern generation of applications needs to be something that is reactive so what do we mean by reactive so light band was founded by Martin radowski who's the the author of the Scala programming language and Jonas bionaire who wrote the akka distributed computing framework and Jonas has been working on what we know we call micro services based systems for many years long before the term existed but what we were doing was struggling to come up with a way to be able to engage different people from across the industry in terms of how do you design effectively completely quad Reddy applications and so this is the concept of reactive so the reactive manifesto was published in 2013 we've got more than 18,000 signatories to it and so it's basically a way for people to discuss how you design applications for this new world so the concept is that your systems need to be responsive under any set of circumstances and by responsive I don't just mean in terms of responsive to users but responsive to API requests responsive to any kind of remote requests and in all circumstances meaning you need to be responsive under extreme load you suddenly get a spike in traffic for whatever reason your e-commerce site and your marketing department drops a sale suddenly your traffic goes to Black Friday levels your system still needs to be responsive so it must be elastic it needs to be responsive under cases of failure for any of you that are familiar with the British Airways disaster their system was not reactive somebody killed the plug on their servers and everything went down right well their applications were not resilient so especially in the distributed computing world where you've got more points of failure than ever before you need to be designing applications to be resilient and embracing the fact that things can fail and your systems need to be able to auto detect that or not to recover from that and fundamentally we believe that the architecture underneath that needs to be message driven be able to be concurrent and be able to be asynchronous non-blocking in order to be able to support resilience and the elasticity so this is primarily an AI ml meter right so what does all that got to do with artificial intelligence and machine learning well there's another really important trend here which is all these enterprises that are modernizing are not doing it for the good of their health they're not doing it for to keep their IT guys employed they're doing it for really good business reasons I'm one of the single biggest most important business reason is they want to be able to leverage their data in order to make better decisions closer to real-time so this is where there's been a massive investment in the past five to seven years and data science right why many of you in this room probably have jobs right and it's a fantastic capability to be able to take all of this data and be able to do really sophisticated build really sophisticated data pipelines trains and models come up with some really sophisticated answers to tough questions now what happens when you need to be able to come up with those answers to the tough questions a thousand times a second and that you're going to run your business on this and your whole business is dependent on systems that can process this well the answer is you need to build fast data systems the concept of big data is great but you now need to operationalize it by building streaming systems that can do the same type of machine learning models and turn it into something that you can actually run you run your business on and so we see this convergence of the world of modern applications microservices design as you get increasingly dependent on put on having systems that are going you're going to be able to run your business on you need to build them in a reactive way as I was talking about and then as we see this increasing trend from going from data at rest to data in motion from batch to streaming you've got the the we're really the sweet spot is now for cognitive fast data systems so taking all of the goodness of your data science and being able to leverage it into a running system that you can run your business on and you really need to think about that in the same set of principles that you do for reactive systems because now you're talking about streams of data that never end a batch job you put it you put it in you get a result out you're done but now you're talking about infinite streams of data that never ever stop so the systems that you run that on they better be reactive so this is a big area of investment for life and for for IBM for many of the organizations here we're working on something that we call our fast data platform it's built on top of our reactive platform which includes akka at the core and then we're adding in the other capabilities that Neil was talking about and your reference with respect to what the Kanna common pattern that people are talking about of the smack stack so this fast data platform is something that allows you to be able to start building streaming applications without having to worry about all of the intricacies of what is a really complex set of systems underneath the covers so you can plug and play the various machine learning libraries into this but you've got all of the core that you need in order to be able to ingest data process data and actually have a full system of micro services for your business logic okay it's not a full smacks back because Mac stands for spark Misaka Cassandra Kafka we call the C choose your own persistence because many people don't need to use Cassandra don't want to use Cassandra might use HDFS might use db2 might use dashdb might use any number of other persistent stores but everything else here is provided for you in a fully integrated supported package so if you are looking to take the investment that your company is made in your data science and you're looking to operationalize it so that you can run your business please find me or mark who's my boss our CEO after the meeting and we'd love to talk to you some more and make sure you take one of these books because these were written by our vice president of fast data engineering dr. Damon plur and we'd love to talk to you all right thank you [Applause] you [Music]