scale.bythebay.io: Evan Chan Interview
Recording: scale.bythebay.io: Evan Chan Interview
[Music] so my name is Evan Chan I'm a senior software engineer oh you mean like the profession or do you mean like engineering challenges yeah right I think that usual like I'm a I'm a big data engineer so we always have to deal with problems of scale resiliency how to keep things up and running fault tolerance how to make sure things don't go down I think those are like in general the big the big challenges well I would say like you know it's always a there's like several levels you know when you do things that are certain scaled and then you keep growing it like from one machine to a cluster to bigger clusters then you find that at each step things that you thought you know works you know don't work as well or when you go up velocities like in like you know for example even the way that you run certain common software such as like Apache zookeeper for example like makes a huge difference and when you find that yeah when you start interesting a lot of data then you have to always adjust and figure things out yeah I'm not sure there's any magic Tony ah scaling in the future I mean I think there's industry-wide you'll see a lot of like he will always talk about the cloud blog you know so I think we're increasingly moving to an area where you have you know different like you have a more flexible architecture like instead of bundling things together like it's going to be possible to ship part of your functionality as like functions and that run is part one or part of cloud thinking then you can use another part to do like to do storage then you can use something else to to serve those things up so I think you're going to increasingly see this flexibility and be able to plug into stuff and order and some of the things that we've had to think about in the past such as having to integrate you know everything and run things yourself will become less of a problem for a lot of people yeah yeah I think functional has been a huge benefit when especially for us who are building data pipelines I think we think about you know two things well what one thing is like the entire like reactive streams things like stream processing that has been like hugely beneficial like to be able to use an abstraction done lets you manipulate streams like very easily it's like that's something we take advantage of all the time and it gives you back pressure you know we also use the rest of the reactive stack like we use akka that's been a huge in terms of supervision and coordination and being able be able to have flexible topologies so yeah would I think we take advantage of all of it oh I think definitely the people you know you see a lot of you know really good folks from speakers - but attendees just and I think part of it is that this is a smaller conference so it's a lot easier to find other people it's not like there's like 10 tracks and a huge space to cover like it's a very small and intimate and I think that facilitates the interaction