Scale By The Bay 2019: Jacques Nadeau Interview
Recording: Scale By The Bay 2019: Jacques Nadeau Interview
[Music] I am my name is Jacques Nadeau and the CTO and co-founder of trivia [Music] this is my first time at scale by the bay I've heard about through a bunch of friends and decided that it would be an interesting opportunity to talk [Music] well I think one of my favorite parts and is is talking about how Arabic came about so I work with a bunch of people on a project called Apache arrow part of the Apache foundation and one of the interesting things is that arrow sort of straddles two worlds it's a transport transport format that's really good for processing or its processing format that's really good for transport and it was designed specifically to straddle those worlds because we've recognized the need to reduce the amount of serialization deserialization overhead that you had when moving between different systems and so we basically said you know what these two things actually need to be coupled in a way and we need to start with an opinion that those things should actually be thought about at the same time and I think it's a really interesting different way of looking at things I think that people who are building things for transport generally are not thinking about processing and thing people who are working on things for processing and generally thinking about things were transport and so I really like to share that with people because I think it's an interesting way that this has come about I think it's one of the reasons that it's been successful to this [Music] yeah so so arrow is a score to what what Dromio does so Jeremy it was a datalink engine we're focused on allowing people to analyze and process data explore data provides a data processing data query engine as well as a semantic layer for your data whether it's in a on-prem data Lake or in a cloud you like we do a lot of stuff in the cloud and core to that is highly efficient processing right especially in the cloud the focus is on efficiency and reducing costs and so arrow is a core part of how we process data it's kind of the linchpin it's a kind of our secret sauce for how we can process data very very efficiently and that allows our customers to be able to process more data in their budget whether that's by you know being able to get stuff done faster so they can make conclusions faster or whether that's being able to shrink the envelope of resources that they need to complete a particular task [Music] was actually really excited about a talk I heard just before mine so Thomas I can't remember his last name I think Herzog or something like that from Salesforce I was talking about how they are struggling with the concepts of data steps data sets versus tables and so they're talking about how a lot of people just dumped it into a data Lake whether that's s3 or some kind of custom internal store which i think is what they use and in those situations you've got a bunch of files but applying meaning to those files can be challenging and you basically push that set of decisions to the consumer and so in their journey they started to realize that they need this abstraction of a table on top of those folders of files and so he spoke about that and talked through sort of their journey in that and and they're sort of their ultimate arrival at a new Apache project called Apache iceberg and it was one great to see that they were going through the same experience that many of our other people that we work with are going through and also to see that they came to the same conclusion we did which is is that not only do you need a table format you need a table format that's sitting inside of Apache foundation and that iceberg is the one to pick and so that was a nice talk to hear sort of their journey and also there are acceptance and appreciation for the work that the Netflix guys did to build the iceberg project originally [Music] just continuing to make people more efficient like I see time and time again the customers that we work with are really struggling with data is growing the needs of the data is growing everybody needs more and more analysis to be done but budgets are not limitless and so I think that we've been working a lot on efficiency and we see a lot of ways that we've been able to make people get more bang for their buck and I think that there's some really great things that that we've been working on that will be coming out soon that are just going to increase that efficiency advantage even even further for us and that's a really pleasant thing because it's an easy thing to go in and say hey I'm actually gonna save you money by selling you something and that's a that's a pretty fun thing to do because it makes makes them feel better about what they're doing and you're solving a real problem [Music] well I think that my favorite feature of the conference is the developer focus the engineer focus that a lot of conferences are very much vendor focused not technical problem focused and I think scale by date the beta is a great job of having talks that are very interesting and technical having speakers that that really know the details and focusing on that and sort of I think that's something that a lot of conferences lose sight of [Music]