Transcript: Scale By the Bay 2023: Interview with Nick Hill.

my name is Nick Hill I work for IBM research I work on infrastructure for serving uh large language models so I don't work on the applications as such I work more on the the the actual platform um so there AI I'm supporting all of the other sort of AI applications that are built on top um really looking to maximize the uh use of the the gpus and get as many um requests through as we can for a small footprint so I talked a little bit about this in in the talk I just gave but uh there's a lot of change in this space of uh llm inferencing right now like every other week it seems there's a new project coming out and they're incorporating the techniques of other projects and adding their own and saying they're much faster so um our sort of main I guess ongoing project is to try and keep tabs and all these things and figure out okay which ones can we uh leverage and where do we sort of add our own things in um so yeah really you know looking at uh how do we keep up and um and which which Tech Technologies do we choose out of this set that that keeps changing I really like the the keynote from Chris latner um I think the uh uh the new the work he's doing with the MOJ language is um is really interesting and yeah looking forward to seeing how that that pans out might be something that we can use in our work uh at some point um and uh I'm also interested in yeah other talks on llms there's one about pytorch 2.1 new features tomorrow which we use a lot of those um and actually I uh quite interested in some of the rust talks because I've been using rust a lot more uh lately I'd say it has a different feel to a lot of other conferences it's sort of much more sort of friendly developer focused uh I really like the way it's a mix of different uh subject so like I mentioned rust there's a you know rust representation also distributed systems is something I'm involved in also AI obviously so it's like the combination of these things I think is uh it's really nice to you know be able to see talks on each and somewhere they combined so