scala.bythebay.io: Modern Software Architectures and Data Pipelines Panel
Recording: scala.bythebay.io: Modern Software Architectures and Data Pipelines Panel
So we had a great second day, it was a long day, right? And kind of that's the tradition. We start with a keynote and Julie started us off, which sounds like ages ago, right? And then we fork on three tracks and then we kind of come together again at the panel. So today we have a panel on data pipelines and modern architectures. And we have one substitution, obviously we don't have Reynold, we have Tim. Reynold didn't feel well and he said Tim. And also he mentioned that Tim is a team lead on the two projects which were presented recently, which I think is kind of really cool. We can hear it from the horse's mouth. So I think it's a net win for us
So let me kind of set the context a little bit in the game plan. We'll basically run for about an hour. We'll start with introductions. We have hand-to-hand mics so the panelists will have to work as a team and pass the mics. We'll basically ask a few questions and we encourage panelists to have conversations between each other. And then about halfway we'll open the floor to audience questions. So if you will have a question, you can line up there after. So that's about..
Oh, Calvin, please take a seat here. Everybody is in assembly. So if you have a question, then line up next to this mic in front of the banner. And we'll basically invite you to ask a question. If you ask a question, introduce yourself. Ask a question and any of us can answer it and any of us can kind of follow up. So that's pretty simple. So let me kind of start with this context
I was asking this question a lot from our meetups of Scala, of Spark, reactive systems, ML. It seems like we have, for each component, we have a strong open source project beating corresponding kind of closed source project to the point where we don't even have a closed source project anymore or we don't remember what it was. So we have Spark, we have Kafka, we used to have Tipco and Devoo and all these proprietary things. Now everybody seems to be using Kafka for MessageBus. We have Aka for the API. There is even no such a thing as closed source API, if there ever was one. Right, and obviously Spark is eating into Oracle's niche and MemSQL is doing the same thing. But kind of an interesting kind of closed open source combination
But we don't have what we used to call the SMAC stack, which is basically an end-to-end stack. API, MessageBus, Compute Persistence and some kind of deployment platform. And that actually is a big problem for a lot of open source people. For instance, deep learning for JGuys, they consult on open source implementation of deep learning on the JVM. They need to come into a company and consume all their data and do some deep learning, but they are not equipped to set up the whole SMAC stack. They don't want to be the providers of Aka wisdom and DevOps expertise. So they want to be very focused. However, they cannot do this easily
So they are forced to kind of do a whole combinations and partnerships. So basically, we're at the stage where we have pieces of this stack developed. So we have Spark Cassandra connector, which is very good. We have obviously Kafka feed in Spark, which basically everybody is using. So it's either Spark or Kafka people have to do this and Spark people take care of this. But we don't have the full thing. And obviously, every vendor will tell you, we'll do everything for you. Come to us, we'll support it
But they're not focused on the whole stack. An open source community as a whole doesn't have it. So that's one question I want to ask everybody. Why is that? Can we get the reference stack agreed upon in some shape or form and then it will kind of lift all the boats? So, but essentially the topic here is, you know, not just one project, but the combination of projects. What are you using? What works? And what are the best practices? So with that context, I'll just let everybody introduce themselves. Hello, everybody. My name is Timothy Hunter and I'm replacing Reynolds Shin, the chief architect of Spark, who wished he would be here, but he's a bit sick tonight, unfortunately. So I'm working, I'm an engineer working at Databricks on Apache Spark and mostly MLE, so the machine learning component of Spark
And mostly I'm working on all the projects around it, which is the graphs or graph frames, the Python integration, the sums, one of the possible TensorFlow integrations, and also some pieces of MLE itself. And I'm very excited to be here. Hello, everybody. My name is Rajesh Mupala. I'm from Index. I'm a co-founder and director of engineering there. So at Index, you know, we're trying to build the Google of products. What do I mean? if you look at, you know, Facebook and LinkedIn, they have an index of people
You know, if you look at Google, you know, they have index of locations. And we are trying to build an index of products. I mean, it's an ambitious product. And right now, in terms of scale, we have close to a billion products with us, a catalog of billion products. We crawl close to 20 million URLs per day and about five terabytes of data we consume. And I'll probably, you know, quickly in two minutes talk about our data pipelines. So we have three data pipelines and one data platform that allows users to create their own data pipelines. So the first one is ingestion
So basically, you know, we have to crawl data. I mean, we get data through crawling as well as through feeds. So we use Akka cluster and Akka streaming there. And then this data is ingested into Kafka. The next step is actually, you know, parsing these HTML pages where we use a parser farm, which is on Mesos and Marathon. And then we have, I mean, these two are ingestion pipelines, real-time pipelines. And then we have a batch pipeline, which actually is, you know, series of ML algorithms, which include, you know, denormalization of the data clustering and classification of the data. And this is something that runs on about 100 terabytes of data
We do a lot of reprocessing because, you know, algorithms and data changes a lot. And the final one is, you know, we actually index this data. We also, you know, learn from these data pipelines experiences and build a platform that enables our end users and our developers to actually take, you know, take data sets out of these data pipelines, you know, do transmissions of it, transmissions on it, and then take it out. So that is built on top of Spark. So I can actually talk about, you know, what are the lessons learned and experiences building these pipelines. So having a reference architecture or a single architecture would have really helped, you know, we actually have a mishmash of technologies right now. So the other thing that I can talk about is, so my background before index is in build pipelines, continuous delivery and all that stuff. So I actually use a lot of the concepts from building build pipelines into data pipelines that we built at index
So things like, you know, failing fast, traceability, you know, reproducibility, you know, those are all first class concepts, you know, that we've been able to build. Yeah. Hey, I'm Helena Edelson. I'm a contributor to ACA, an ACA cluster, as a committer for the Spark-A-Sandra connector. I'm a committer for PhiloDB, and I work at a large, very secretive company based here in the Bay. I have it. Thank you. It's my turn
My name is Nikita Shamganov. I'm the CTO and co-founder of MemSQL. is a scalable database and a data warehouse focused solely on real-time analytics. In fact, Gartner recently named this the number one operational data warehouse. What really is, we're defining real-time analytics as the ability to basically subscribe to real-time data streams. So, for example, if you have Kafka at arbitrary scale, you can say, well, I want to stream that data into this table inside our database, and we'll be providing a very, very large scale for this type of operation. We have Bay Area customers that are streaming 75 terabytes of data a day from Kafka into MemSQL and performing immediate data warehousing type queries against the system. There are a lot of other factors into building real-time data pipelines and landing this data somewhere, either on HDFS, in the database such as MemSQL, but the big deal is how can you react and analyze this data quickly, and this is something that we solve really, really well
Prior to that, I was at Facebook and prior to that at Microsoft SQL Server, so I know a thing or two about system development. Hey, everyone. My name is Kelvin. I'm the top contributor as well as the release manager for the Alexio open source project. Alexio is a virtual distributed storage system, which, as Alexi mentioned, could be one of the integral parts of kind of this reference architecture that we may want to build up with different open source technologies. I'm currently working at Alexio, where I'm lucky enough to find a place to basically pay me to work on this project, so I'm very happy about that. Yeah, hi there. I'm Chris Fregley
I started this company, Pipeline.io, about six months ago, focused on really pretty much this exact problem. Our sort of specialty is not just the training pipeline, but also the serving or the scoring or the predicting part of the pipeline as well. So we don't stop at the Jupyter notebook. We actually have a way from a Jupyter notebook to deploy on Kubernetes and scale it out using Netflix open source. I used to work for Netflix and Databricks as well, too, actually. So, yeah, like combining Spark and SensorFlow and Kafka and all that good stuff that I'm sure we're going to talk about a lot. Thanks, guys. So I'll probably ask the first question
So what are the key questions in integrating this pipeline? So what are the typical setups you see? I mean, you come from different backgrounds. So Rajesh is kind of assembling it himself. And Chris has his own open source, full open source version. And Tim and Calvin come from, and Nikita, from kind of individual components, but you cannot survive on your own. So how do you deal with this integration? What issues come up? What things are easy? What things are hard? So I think from Alexio's introduction, he said that Alexio is actually just a component. It's not really an end to end solution. And I think Alexio's main goal is to really help these end to end solutions work together. So one of the issues Nikita mentioned was there's a lot of applications out there which may be asking for memory
And that's going to be a tough problem. How do you allocate all of those storage resources correctly? And one of the problems that Alexio tries to solve is being able to intelligently kind of make your memory available to the applications that need it. So instead of having, for example, a Spark job saying, I need this much memory, it's now kind of a memory that's served through Alexio. So that if another job happens to use similar data sets, then it will also be able to enjoy kind of the in-memory computation benefits. And at the same time, Alexio also kind of looks at the other side of the problem, which is where all of your data is coming from. So a lot of the times people have different storage systems. For example, you have your HDFS clusters. They also have things like S3 or even kind of internal legacy systems
And you would like to be able to do your queries, your analytics on all of these things. You want to be able to run pipelines on all of these things. And that's also another place where Alexio kind of helps out in being able to provide a unified interface for any applications that want to access data through the storage. So Alexio kind of helps on both ends of making applications work together better as well as making data available to applications. And I think this will be a very critical part to kind of this end-to-end solution that we're kind of aiming for as a final goal. Yeah. So to answer your question, I don't know what parts of the ecosystem helped us in integration, right? I think one thing is definitely the storage part. We use HDFS and S3 and the fact that Hadoop's maturity meant that these interfaces were pretty strong and these APIs were strong
So we could just use either Spark or Hadoop and it just worked. The second thing that helped was strong schemas. Our early days, early into big data world, we used CSV and JSON and they were horror stories where we actually started writing nulls into our database. But, you know, once we started using Thrift and Avro, it meant that, you know, it's easy to, you know, communicate across these different systems. You know, you have strong guarantees around, I mean, the data will not be on serialization, deserialization will be all taken care of. Yeah. So I think approaching the problem of how to compose data pipelines, it's really interesting because I found with many different domains at many different kinds of companies and different data sizes, there isn't really one solution. I mean, I think Scala is one of the best ways to go
But oftentimes you can't, I'll hear many people say, what about this technology or this? And it's very often the case that it's apples and oranges. It's not at all, you know, like pick a spark to do everything or, you know, Flink can take care of everything for you. I mean, it's very much a situation of look carefully and discuss your use cases, really investigate, you know, what is your streaming? Is it actually like primarily time series? What kind of data stores are you going to need? What are your access patterns going to be? Are you going to, and you probably would want, you know, separate analytics clusters for storage versus, you know, time series rights? You know, it becomes this really interesting problem of composition. And I think it's always very specific, you know, there isn't a blanket answer. And so composing these systems is really interesting, particularly when you get into situations of extreme scale, where often what you'll find is most technologies, even those that are marketed to handle scale, will actually melt down, because not at the fault of the people producing those technologies. I think it's simply a problem of when you're a framework developer for the open source or anything, you simply cannot replicate what companies see in production, the scale that they see in production when they're testing. Yeah, so I think that there are two aspects to this question. The first one is as a vendor, and the other one is as a developer
And as a vendor, of course, you would like to support everything you can. And as a developer, I think you have actually an incentive to try to focus just on your technology and try to improve and make it more modular. So that's why I think at Databricks, since we had a lot of the Spark open source development, and at the same time we tried to build a platform, we have been focusing mostly on providing some very good connectors with the other technologies, and on Spark itself, instead of providing a big bundle solution. And I think this goes along with what Nikita and Elena have been saying, in that it is already very hard to get one piece of this technological stack to work properly at scale. And also people have very different needs, so you will never be able to address it with one single solution. So this is why being able to import your data, being able to transform your data, and being able to exchange this data between the different platforms is something that we find absolutely critical. Thank you. So maybe I'll follow up
Since you've been behind this new releases from Spark on machine learning with TensorFlow and GPUs, and Chris I know is working on a TensorFlow course for Riley, maybe we can talk about how data pipelines support machine learning. I think there is a huge interest in AI everywhere, and basically I think what they call AI now is everything which can make decisions, and if the NL statement is now AI too, because it can look at the Boolean condition. So essentially everything we used to know as a machine learning data science becomes AI, but in reality it means there is a certain set of frameworks, TensorFlow is number one. Right, which basically people want to support, but it's an emerging area. So I wonder, what's your experiences, what does it mean to do machine learning in production, what are the actual use cases with customers, and how the data pipelines you are building are supporting these use cases? So at this point, especially around the Bay Area, it's very clear that machine learning is becoming a key part of every data pipeline, and also every business in an enterprise. After that, what machine learning means depends a lot on the experience of the people. And we see really a very broad spectrum of definition for that. Mostly when you talk to large established business, when they mean machine learning, they mean linear regression, and techniques that essentially were invented 20 years ago
And why that? Simply because these are the techniques that have been around for a while, and that are the most understood, and also they are the most easy to debug when you have a problem in the production pipeline. And so what we see really is this gradual adoption of more complex techniques. And the frameworks for doing these more complex techniques are usually spearheaded by a very large company that have the means to dedicate a whole team of data scientists, data engineers, PhDs to build these extremely complex systems. And this is why now our new technologies like TensorFlow start to come in for the masses and start to be deployed in some specific problems. And it's definitely still now an open question to see how we can integrate pipeline that we're designed for doing mostly data processing with pipeline that are meant to be doing some more data intelligence and more reactive systems. So in this area, I really see TensorFlow and Spark as being complementary, in which you can do a lot of the heavy lifting and the data processing with Spark to transform your data into a form that is more amenable for complex AI algorithms. And then TensorFlow can peek over for what it does best, which is really solving some critical AI problems. And of course, each of these frameworks come with different mindsets and different requirements
Spark was really designed for large scale commodity clusters, while TensorFlow started really more to work on specialized hardware like GPU, for example. And how to combine it now I think is going to be something that we will see a lot in the next following years. Thank you. I have a bit of an opinion and I'll be pretty quick. So, yeah, so certainly TensorFlow is emerging and even though we're providing database technology, we see that our customers use it in combo with NemSQL. And one of the typical ways to think about it, there is, you know, there is data engineering, where you're just making the data, preparing the data, doing feature engineering, understanding how, that's where actually engineering pipelines come handy, where you're acquiring data from, you know, Kafka and whatnot, you dropping it into HDFS or any MemSQL, any other storage system, use Spark to understand which features you want to extract from there and perform machine learning either in Spark or somewhere else. So, then there are data, there's like kind of true data science. Assuming data is all nice and clean and you trust that data, which is super important, every data scientist is extremely obsessed about the quality of the data that is there in the system
Then you can perform different learning algorithms and you certainly start, we see more and more TensorFlow. From there, there's inference or scoring or, you know, whatever you name it, evaluating the model. And where the world is going, we're going to see a lot more scoring than we're going to see learning. And the reason to that is, soon enough, your coffee mug will, you know, evaluate in your machine learning model to understand, you know, is it cool or do we want to heat it up maybe, to your personal preference. Like that, this is the world that we're going to be living in. Certainly driverless cars are driving adoption there as well. And so there, it's important to score machine learning models at the edge. Basically, the moment you pick them off data from Kafka, you already know if you have models handy, you want to apply those models and understand, you know, inferred properties of that
And we certainly see that one of the cool things that we do, for example, is we help people do directional drilling, where they drill through the ground and sensor data is coming back, flows in Kafka, and they apply machine learning models from that sensor data to direct the drilling, which I find really, really cool. And that's also an indication that, you know, the world is moving towards a place where it's going to be a lot more inference for machine learning models as compared to what we see today. Yeah, so I mean, this is the big focus for the pipeline IO project. And things like code generation, things like that, right? So we have one demo that shows PMML taking SparkML, converting it to PMML, and then pushing that out to a live running cluster and then scoring that way. But if you think of a world where we could take the SparkML model or the TensorFlow model and then generate C code or code that's directly native for the hardware. So starting to look into some of those things. I didn't get a chance to comment on the pipeline one. Yes, I think in terms of like monitoring the pipeline, that's huge
Yeah, that's one thing. Like that's a big thing I picked up at like Netflix was, yes, just having metrics at, right, like all the different stages, have sort of insight, right, like into these pipelines. Don't just treat them like just sort of an offline batch thing that, right, yeah, these should be continuous pipelines that you're constantly monitoring. Get yourself a good scheduler, right? Like, yes, there's Airflow, which is pretty much what everyone's using these days, which is super powerful. So just, yeah, and so get like Elasticsearch set up, get, you know, Logstash, get like Fluentd, have all processes writing into one central place, right? So you can, you probably don't think that you need the data right now, but, yeah, that stuff comes in handy and it's difficult to go back and then retrofit, right? But, yeah, so the scoring thing is pretty interesting. I've been kind of toying with, you know, doing things on like Android and, you know, cell phones and kind of having, yeah, especially with like TensorFlow. There's a really good demo. It's called TensorFlow Mobile for, or it's TensorFlow for Mobile Poets
Yeah, have you guys seen this out there? Pete Worden. So Google purchased Pete Worden's company because he was focused on, or like AI on small devices, right? And this is the exact space that like Google was trying to get into. So, yeah, check out Pete Worden's blog. And he has one called TensorFlow for Poets. And then he has TensorFlow for like Mobile Poets that actually, you build an iOS app that scores with, yeah, like TensorFlow. Yeah, just going back to the pipeline monitoring part, I think, you know, we've done some interesting stuff at Indics where the first thing is, I mean, we actually have a dashboard which our CEO can also see and understand, you know, where, I mean, what is the status of the pipelines? You know, we have Slack alerts where, let's say, you know, jobs, let's say bad jobs take longer than what they're expected. We also make sure that for the real-time pipelines, you know, we look at something like, you know, Kafka offsets to make sure that the event time and the processing time, I mean, there is, the skew doesn't go too much. And then we look into that
And then we also have auto scaling enabled, you know, in cases when, let's say if the lags are pretty high, you know, we make sure that the cluster auto scales and reduces that lag. Thank you. So, I'll ask one last question for me, maybe if you guys have questions, please come up to the mic and then we'll fill your questions. So, kind of my short version of the question is very simple, is Google going to kill everybody? And the longer version is, right, so we basically are building this Mac stack and the different components, right, we have Kafka, we have Luxio, we have Spark, different pieces. What I recently noticed, that a lot of people just start, basically, throw everything into Google, run BigQuery and, you know, build a dashboard. And all the SEO, you know, likes the dashboards, they work, right? So, basically, it's kind of, right, I started thinking, this is really interesting, right, because essentially, I mean, there is a lot of guys who cannot be in the cloud. But it becomes extremely easy. So, we have, you know, the smack stack or pancake stack as a sequence of different pieces, right? But if you look at the whole thing, actually, a lot of people now say, well, we are building a database
So, Smite is a startup building trust. They basically decided to build a bunch of counters and from these counters, you know, they determine abusive behavior or fraud. And eventually, they realize, we are building a database. The database has certain features. It has to be, in addition to being an asset or some kind of that, they need to be, you know, it needs to be resilient and distributed because that's the requirement, right? So, but you still can think about it as a big database. And if you look at FileODB, eventually, you put together different pieces and in the end, you get a database which has certain features. So, we are all building a gigantic database of different set of features. And in that case, Google has the bigger database there is, right? So, how do you see kind of cloud-based platforms kind of threatening open source components combined into a data pipeline? So, I think in this situation, there's never going to be one big data store that would solve all the problems
It's always the case, I've found, that you need several different data stores. Each one solves different problems or they're optimized for certain situations. Primarily, they can do other things too, but they're optimized for certain things. And there's always so many data flows and different directions and things that all the data pathways have to do in a system. And there are so many different users. There's the data scientists. The data has to go to different clusters for ML or it has to go through analytics pipelines for many different teams. And it has to usually be stored in many different places in many different types of storage
And like columnar, press storage, and for fast, like ad hoc querying later or some kind of cold storage for backup of raw data. I mean, it's never one thing that I think would solve it. I mean, it's not a big deal. The world is much bigger than BigQuery. And BigQuery is good for running a query against data stored inside Google with a latency of 4 to 16 seconds. Not everybody is happy with the latency like this. There are plenty of scenarios that the latency requirements are much stronger. There are plenty of scenarios where you need to run transactions against your system
There are plenty of scenarios where you are not okay with 40 seconds delay for data coming into BigQuery from their streaming solutions. And those things may or may not improve with that. What we do know for sure that we live in the world where data processing becomes richer every year. A couple of years ago, there was no TensorFlow. There was no rise of deep learning. Feature engineering was not as important. And those things evolve together with the desire of storing more and more data. So that, I think, is an exciting thing
And Google is not going to kill infrastructure companies either. Yeah, to echo what Helena and Nikita both said, kind of one situation that you might find yourself in, or maybe not yourself, but a company may find itself in, it sees that there's something like BigQuery. There's something like Redshift, there's something like Redshift, and one business unit or kind of one team says, wow, that looks really great. I'm going to use that and it's going to solve all my problems. At Alexio, I've heard that story. And so, of course, one issue I've seen kind of people then come back to us again. It's like, well, my other team also found another kind of a cloud provider and they happened to pick a different solution. And now kind of our data is siloed in these two different things
And at the end of the day, companies always end up wanting to get more out of their data. Like, kind of the big data explosion is all about people trying to derive as much as they can from all of the data they've been gathering. And then they come up with the problem, how do we kind of consolidate all of these disparate pieces that we've kind of been working on isolated? Because at the time it sounded like a great idea because I'm only solving this one problem. But then when you wanted to expand it and kind of relate your data between different groups in your company, then you kind of need to start brewing your own solution or maybe scrap everything and go back to one centralized store. But then, you know, a few years down the line, you'll have the same problem. So that's kind of where I see really this open source stack of companies having the flexibility to be able to really be in control of what their data pipelines as well as their data analytics look like is going to kind of win out against these brewed solutions provided by cloud providers, which may work out really well out of the box. But at the end of the day, it's not something companies will find themselves happy with in the long term. Yeah, I mean, I agree to what you guys said
I mean, you know, just talking about personal experience, you know, we are on AWS. We have 1,000 boxes on AWS. And, you know, when they're all, I mean, AWS has come in. I mean, they've given us free credits for Redshift and Kinesis. But, you know, we would prefer HBase and Kafka. Just because, I mean, when there are open source alternatives, I mean, we would prefer those because, you know, it's easy for us to open the source and, you know, figure out when things are going wrong. So we definitely don't want to get logged into these technologies at least. Yeah, so I totally agree with what everybody has said
And I think also one thing I would like to expand on is it's also a matter of a bit of perspective. When you use, essentially, every business tries to solve a particular problem. And all the solutions we're talking about are tools to solve this problem. And when you look at, and so then you may decide to choose a service or an open source solution or a combination in between. And there are different trade-offs when you consider each of them. Because each of them has really two sides. The developer side, where you try to, where you actually help build this platform, and then the user side. And usually when you're working with a large closed source system, then you can only contribute really through one side
And actually just to have a quick understanding, who here contributes to an open source project? Yeah. So this just shows, and probably the reason why you are doing that is because you found the limitation in this open source project. And you were willing to step in and fix it. And being able to have this flexibility for projects, even if they are huge, even if they're really complex, like Spark or Kafka, means that you can really influence the project, not only in the direction it is going, but also in how you can adapt it for your needs inside your company. And being able to do that is really one of the strengths of the open source. And also this is one of the reasons why we see people also adopting Spark for their use cases. Because they know they have the ability to change it. Even if Databricks offers it as a service, they know they always have the ability to adapt it to future needs or current needs for which we'd not provide a good solution
Thank you. Yeah, so I have one comment on that. I mean, I think I'm more fearful. Or, yeah, what keeps me up is it's services like the Google Cloud ML or the Azure ML. Because I think they're doing a pretty good job. They know their audience. Like we're all open source people, you know, so we can sort of geek out over these kind of things. But there's a lot more of these enterprise people than there are us, right? And, yes, one of my favorite projects, recent projects, is Project NiFi, right? The Apache NiFi project, Niagara Files, which got picked up by Hortonworks, I think, now kind of runs that project
But it's kind of the best combination. It's using open source, or like beneath the covers. And, you know, it like relies on Jetty and, you know, things like this. And there's some like messaging going on in there too. But it's this really slick UI that you could just wire up really easily. And you have just enough insight into what's going on behind the scenes. And you can crack the code open, that kind of thing. But, yeah, I mean things, yes, I remember being at Databricks actually when, I think we were actually at the Amazon Summit
I think it was, we had a booth there. And it was the summit, I think it was two or three years ago, when they announced the Amazon service for like machine learning service, right? Where you could just kind of wire it up and, you know, do your hyperparameter tuning and things like that. And like we were shitting our pants at the booth, right? Because we were like, holy shit, this is exactly the kind of space. And, you know, this is the kind of functionality that we're building for customers. And then these cloud providers are now building it. So, I think it's not open source, you know, you can't crack open the Amazon ML service, you know? Yeah, which is a huge downside. But I think the problems are all pretty well understood, right? Like we understand machine learning. I don't think we really need to get in, you know, to start cracking code open
And I think the cloud providers will have failed if they can't really meet those sort of higher level needs. Thank you. I think we have audience questions. So, let's invite the first question. Please introduce yourself and ask a question. Hi, my name is Matt. Several of you mentioned TensorFlow as a deep learning library. Amazon recently announced that they're going to support another library called the MXNet
And they describe TensorFlow being limited in licensing. And I'd like to see your comments about if there is any demand in our companies for use of MXNet. And what kind of limitations do you see in TensorFlow there could be? They mentioned if you develop at scale for special chips, that TensorFlow is not free. So, definitely there is a lot of deep learning frameworks around there. I think I can't, I could count them on one hand. Now I think it's on two hands. So, of course, after the question is which one you want to pick, because each of them actually has some advantages in one specific area. And so, for now at Airwix, we have been focusing on TensorFlow because this is the one that provides the best engineering frameworks around it
Simply, also simply this is the easiest to install and to start with for most of the customers. So, as far as licensing is concerned, I'm not aware of these limitations. I thought the whole code was Apache 2. But I can imagine why they would want to put some stricter limitations around developing your specific chip for that. They mentioned mostly on GPUs, type of high-end GPUs that would not be allowed to use them. Well, for the TPUs maybe, for their tensor processing units maybe, is that what they were? No, no, no. They're saying that if you need to use high-end GPUs, the code is not open. From NVIDIA or from somebody else? Yeah
From NVIDIA or from somebody else? Yeah. So, most of these libraries also rely on fundamental blocks provided by GPU providers. And actually, right now, it's not plural. It's really singular. It's only NVIDIA. And there is indeed some strong legal limitations into how you can embed the code of NVIDIA. And if you don't use this GPU code, then you are usually having a pretty bad performance. So, usually the limitations come from there
Yeah, so Alex Smola just gave a talk yesterday at the ML Comp. I think, oh, yes, I'm actually wearing the sticker here. He was comparing the performance of the MXNet to TensorFlow. And, yeah, so he was showing, right, that like MXNet had a slight advantage there. But I don't remember anything, yeah, because he was talking about TensorFlow. But, yeah, he's the guy from, right, like the Amazon machine learning group. Yeah. I mean, there's GPU version of TensorFlow now that works with, you know, CU DNN and, yeah, the NVIDIA library
So, yeah, I'm not aware of that. Just my two cents. If the concern is licensing, then devil is in the details. So, you know, sit down with your lawyer and find out. Clearly it's free for running TensorFlow on NVIDIA. And it has a good license. And then whatever beyond that is really details. As for performance, and we touched performance a little bit
We're in talks with NVIDIA for, you know, some of the deep learning stuff, what people do at the edge, as well as just we constantly exploring how we can make our database even more performant. They were saying that for driverless economy, people not always default to TensorFlow, for example, and they use more performant frameworks. They mentioned CAFE. And then, like, you know, Tim said, there's a lot of them coming up. And different ones will have an edge for a particular use case. So, tables in the details. I just wanted to address that I don't know how mature that ML technology is at this point. And in some situations, you know, you really want something that's robust and mature for the data that you're going to be getting out of it
And plus, I think that the total cost of ownership, you know, sometimes when you have just vast amounts of data, it becomes not very cost effective to do that in the cloud and you need to find another way. So, I would say no on that too. Thank you. We'll have our next question. Please introduce yourself. Hi, this is Greg. A question about when you're doing machine learning scoring, that's fairly steady state and it's low variance. When you have to stop and retrain models every once in a while, particularly if you have, like, one model per person, you know, for a whole company
And you're doing monitoring for something like fraud detection or insider threat detection, account takeover. So, then all of a sudden you're going to get spikes in having to retrain your models. So, talking about your different stacks, what do you think are good architecture designs? I have had my approach. I'd like to hear what your approach is. So, just for dealing with the stacks for the retraining. So, a number of our customers do that and they actually use Spark for training and they use data stored in M-SQL and these are the two systems. Oh, and then there's also data incoming from Kafka. In general, what I would start thinking about is how long does it take to make a change in the system? So, if there's a data scientist that there's some sort of iteration where you go from, you know, future engineering all the way to deploying this model
And what I've seen work really, really well in different organizations is to focusing on that latency of making changes. And that latency comes from multiple places. The ability to take a model and deploy it somewhere. The number of stacks that you're using. If your data scientists are in R and your serving layer does not support R, I'm sure your supports R, then you're in trouble because you need to provide some sort of mapping there. So, lining those things up and like squashing, trying to merge different stacks would certainly help with that latency. Now, to directly answer your question, what do we see our customers do specifically in banking? They store data in M-SQL, there's Spark sitting there, and then when they need to retrain the model, they grab data, create that, you know, data frame, grab it from M-SQL. We give them, you know, cluster to cluster data transfer, which can run in massive throughputs
And so, once that is enabled, then they can retrain in Spark and deploy the model very, very quickly. So, do you support R models beyond what's in PMML? So, what we do is we actually allow you to deploy a snippet of your own code. And we support PMML, we support R models, we support TensorFlow models by doing that. But enabling basically SIFT data, which is in the database, through a custom piece of code, either at ingest or when it's already started the database. You know, at scale, obviously, because everything is at scale. So, one thing I'd like to add, because we see definitely the same problem and also the same solutions. One thing I'd like to add is that a lot of our customers have been trying to have a lot of individual models for each of their customers. And usually, they don't have enough data for each of the customers to be able to extract a lot of value from that
They realize that either their model is overfitting the data they have or that the model is actually too close to a baseline to really have a serious improvement over having a one model for group or cluster of customers for that. So, I would say here really definitely your mileage may vary. But we found that having the data scientist training a few models, a few simpler models on the larger data sets usually is more cost effective. That's who is a customer, right? The customer is an individual that you might not have enough data if the customer is a company. Yeah. Just a quick comment. If the customer is an individual and you're in the organization, you might not have enough data to train the model. If the customer is a group of people or an organization, then it's a different story
Thank you. Any other questions? Well, if not, then let's thank the panelists. And next up is happy hour. Thank you.