Scale By the Bay 2023: Interview with RJ Nowling.
Recording: Scale By the Bay 2023: Interview with RJ Nowling.
so my name is RJ noling I'm actually an associate professor at a small school in uh Milwaukee called the Milwaukee School of Engineering so we have about 3,000 students mostly undergraduate programs with some Master's programs uh our Focus we're Polytechnic so all of our majors are related to engineering and so I'm a co-director of uh several graduate programs in machine learning and I teach courses such as data science but also ml production systems and I had four years of Industry experience as an ml engineer between my PhD and faculty position and I've recently done some Consulting work with Memphis uh who's developing a uh sort of message streaming platform uh data streaming platform so um I I teach AI uh but you know my focus is is especially recently is on the ml production system so when I was uh an engineer in Industry I worked on massive systems that for ad click prediction um with hundreds of millions of of different cookies and so one of the things that you know we realized is that or at my university is this knowledge is kind of specialized and it's you learn it tribally you go met work under someone at a startup or another company doing this and so my focus has been how do I standardize this knowledge and package it in a format that's easily adap uh picked up by other people I think one of the the industry changes I've seen is you know four or five years ago when I was doing this it was mostly batch and so we do a lot of sort of the backend pre-computer a lot of things for our known users and that's very computationally expensive um you know and it it wasn't very amendable to you know figuring out which users had updated data and which users didn't so you ended up recomputing everything it was expensive very complicated pipelines hard to maintain hard to operate hard to keep running um and now I you know I love streaming because it allows us to do things like you know oh we have some history for these you know incoming data for these particular set of users we can update just for those users the information we have and update our key value stores or whatever to use at prediction time so it the pipelines become a lot smaller in some ways because they only need to process the changes and that makes them easier to manage and easier to implement uh makes the resources you know in terms of cloud compute costs and stuff easier to predict um and then the other side of that is is you know we're seeing some of that incremental computation moving from the back end to the real time you know so one of the other industry Trends is you know this was kind of what my talk was about was not just hey let's pre-compute all of this information we need for the users because we don't want to store things when you know only 1% of our users are active users or something like that or 10% that's costly so instead how can we store information such as say you know uh the American Community survey data like incomes and population size and stuff like that for zip codes that can be used across a wide range of objects so it's much smaller much more efficient to to store but then when you do the predictions in real time you can pull those from the key value store and do your join online and so you basically just put everything assemble everything as late as possible in the process to avoid having to pre-compute it which again reduces cost and it also makes you more responsive right you can pull the latest data and and so you can make sure that you know if you have new information about the user that you incorporate that as quickly as possible in terms of some of the talks I'm most interested in there's a talk by Quin here about streaming graph so I had some friends who tried to use graph databases in genomics and it didn't work very well because the data sizes were just m massive and not in terms of like raw gigabytes per se but just a number of very small records with a lot of interconnections and so I'm very interested in how Quin as a streaming graph engine uh allows that incremental comp again that theme of incremental computation allows that incremental computation um because I think it might make it more feasible instead of trying to do giant batch on graph um I was also very interested in the talk on converting a scallet pipeline to rust I don't personally use rust but I have some uh industry folks who I have connections with who do and they're just amazed by like the the operational efficiency of Building Systems in Rust and so I'm kind of curious as to sort of going forward as a technology we know it's great uh how well will programmers and teams be able to adapt to it it's known as kind of a hard language um and so I'm kind of curious as to how that's going to play out so my favorite conferences tend to be conferences that i' would say have like two or 300 people um because you get to actually meet people and talk to them and you don't feel lost in the crowd and scale by the bay despite the name has the appropriate scale of attendees and they're all practitioners right it's not most of the talks here are not necessarily a companying company talking about their product that's an engineer talking about a problem they learn they faced and how they solve that problem and their experiences so as another engineer I can take that and use that now that isn't to say there isn't great representation here of companies and products there are but I love that they're they're talked about in a very practical way of here's as an engineer what you care about here's how it can solve your problems um and so I just really enjoy the conversation and meeting I've been able to run into people that I worked with in Industry you know four or five years ago uh who haven't seen in four or five years and so that's really cool