SF Scala: Yao Yue Interview

hello everybody I'm Alexa Kramer of the organizer of scholar here at fallen dB and we follow the panel on infrastructure over there and one of the panel's SEO you hear she's a staff engineer working on on caching and so she leads the polygon on your project so this is very interesting I I'm curious how the kitchens from the metal part how did you get to choose this project so I didn't choose caching I got dropped into caching because that's the one thing that that caused her to be on fire a lot of the times uh-huh and that was only one other person at the time working on caching so I joined the company at the time and they were like okay you go welcome caching that's and that's six years ago yeah so there's something this was like yep so what is it what makes it so fundamental so so caching is a conceptually extremely simple service right you put something in memory on the potential on a different machine you you want it out you may or may not be there so compared to database a lot of the guarantees are just removed you get to basically do a very simple service that's that does not provide too much guarantees the only one it has to deliver as to be fast you have so it's the sort of the ultimate high-performance distributed system that you want to deploy in a data center okay so I'm attracted to cache because I'm interested in performance fundamentally what about things with stale is it the job of cash or is it the job of the whole system as to know what data is in which cache so you're not getting stale data well someone said there you know one of the two hot problems in computer science is cache invalidation so generally the answer to stillness as you do the best you can a lot of the time scalability and throughput and in latency is Trump actually Trump the need for fresh data so people actually would go very far on stillness they would serve still data for a few seconds if that means they can cut a lot of their you know throughput requirement or their agencies so this kind of trade-off happens all the time so I have this interesting personal experience I had interviewed Google and I got the author by basically answer every problem with caching so they throw puzzles with me and I would kind of make up take a pause and then I was realize it's caching and so because the different people loop they probably thought of most pressing problems for them and they're all basically problems so is it as well is kind of caching than one of the major pieces of distributed system well one quote I like to use is like there are three fundamental problems to solve you know happen all computer science problems roll down to these three right hash - and trash and I actually email the author of this quote I said why trash like did you foresee that was in the 90s I was like did you foresee then you know garbage collection is gonna be a thing mm-hmm he's like no it's just it just rhymes I need a third thing so so really it's just hashing cache right so as a concept it's it's it's universal but you know depending on the context when we talk about caching we are talking about specific form of caching and I'm certainly only own a very slice it's very thin slice in the entire spectrum from CDN to CPU cache right now it's it's pretty ubiquitous and so you have this Pelican project and we're not very familiar with it yet so what is the advantage of this what's a good way to start to try it can you tell us a little bit more I'm sure so Pelican I think one way of looking at it as twitter has been using cache caching very heavily for a number of years and at very large scale so it's an answer to all the problems in terms of feature in terms of scalability that I have we have encountered in our production so as we think how we can fix these problems that then that answer to that is basically Pelican so don't think of as as oh I want this whole new thing to replace all the existing caches it's a natural evolution from actually all the things we have been using while we one example is it has a different code name right but if you look at the source code closely 50% of the code in this code base Ashley is borrowed or you know borrowed from heavily tested code bases like Redis memcached when proxy all open-source projects and so we're just trying to architect this critical component in such a way that it overcomes a lot of the scalability problems we saw before and it paves the way for more feature development in the future that hopefully it's very can be carried out in a very productive way okay sure so if you just go to get up you know slash Twitter yes slash Pelican yes it's there so it's a single code base that has no external dependencies the huh because we it turns out having dependencies makes a lot of things hard so we did a lot of work to do away with dependencies uh-huh and it just compiles if you have C make you have GCC or Ceylon you can compile it you can run it I I'm happy for people to you know perform test it compared to Redis or mem cache or whatever and tell me how you feel about nice so thank you much I think it's very exciting you know I would like to have you back at micro services meetup because you know I think a lot of this stuff is you do together with caches so thank you very much looking forward to learning more yeah