LLM Avalanche: Stefan Krawczyk: Hamilton is a Python Micro-Framework for Describing Dataflows
thank you hello um my name's Stefan I'm the CEO of a company called tag works but I'm not going to sell you anything today I'm going to talk about an open source project I created at Stitch fix it's called Hamilton and it's a python micro framework for describing data flows the thing that I kind of want to leave you today is hey if you're building a large language model powder apps I think Hamilton is something you should add to your tool belt to help you build it so to set some context um two things you know getting these llm-powered apps to production I hear is a bit challenging and a bit tricky right uh two things that I kind of here most common is hey observability is tough and then this kind of impacts iteration and debugging where I know people can spend a lot of time a lot of time figuring out what happened just because you know the iteration and changing things in production is difficult my take here is that it's a software engineering problem in that you need to engineer things in a way to provide you the hooks for observability and you get a right code in a certain style that's amenable to make changes quickly confidently so that when you make changes you don't break things easily in short the high level is I think you know Hamilton can help you here especially if you're rolling your own code which I hear most people are doing if they're taking uh their apps to production so so uh why does Hamilton help in this context so most large language model powered apps are largely just a sequence of data flows meaning you have data and you're transforming it um and so you generally have an agent which contains the state of the conversation but then it's taking some sort of actions and what I mean here by sequence of data flows is really these actions so if you want to find articles or summarize articles or something you can model that individually as data flows or directed acyclic graphs if you study computer science this is Hamilton's bread and butter and so what I'm talking about is you should consider using Hamilton to help model the actions what is Hamilton this is a lightning talk so this is going to be brief but Hamilton in a nutshell is a paradigm it's a slightly different way of writing python code you write declarative python functions and they give you a data flow or a dag here I've defined two functions the way to read it is I have the first function uh it defines at embedding the embedding requires a query and outputs a list of floats and then I've defined another function that gives me the nearest neighbor IDs that takes an embedding it takes in a vectodb client and then you know the top K result the top 10 number here with Hamilton what we've actually defined in code with these functions is this following data flow you can kind of see that query is required for embedding since it's declared in the function and then the nearest neighbor ID similarly declares three things and you can see that represented in the function arguments one way to think about Hamilton is that if you were to draw a flowchart of a sequence of computation it is the most direct way that you could write that in code where effectively any node of computation you can write as a function and by the way I didn't hand create this image this comes with Hamilton for free so it's a great way to also bolster your documentation now why should you use Hamilton well I think it will increase your iteration speed give you Clarity a review of code and also increase your testability every function in Hamilton is unit testable and has a great integration testing story but uh more so when you're making changes you don't have to deal with object-oriented hierarchies no need to kind of understand and really dig into something so here I have an example so if I really just want to abstract out using the open AI client I can all I need to do is change the signature of a function it's very clear what I've changed and then structurally you can also see that represented in the graph that's kind of defined the other is around modularity and composability this data flow here we can easily add it to a different workflow pretty easily just by declaring a function that depends on it and so it's very easy here for example to reuse and compose workflows in a modular way so in the stop flow I was getting the nearest neighbor IDs given an embedding and I just connected it to another flow that is getting say the relevant documents connecting it with a tokenizer and chunking it this means you know and so getting to the the iteration aspect it's very easy to switch between multiple implementations you want to switch Vector databases you want to switch between uh you know different llm apis out of the what Hamilton gives you is there can be very surgical about it you can very easily change different node implementations or even you know just cut the data flow and switch that out with a completely different implementation and that's very clear and easy to do uh with how you kind of Define the code so this is a lightning talk so I'm going to try to summarize this and kind of leave you a few takeaways so first is that Hamilton is a way to write python functions in this kind of declarative manner what you get is a data flow and what I'm uh telling you who are saying that it's good for in that context of llms it's a great way to model uh the different actions that you need it to take so this means I think you're gonna you can iterate faster and have a clearer testing story and similarly if you pair this with Git when you change your app you can always go back in time to understand how a particular flow worked out very easily it has a very lightweight dependency footprint runs anywhere that python runs Jupiter airflow fast API Etc uh there's a lot I haven't shown you and it's actually built to be extended I I was on the platform team at Stitch fix so we built this to be you know to make an easier life for a platform team and so uh if you need to add caching which is one of the things you want to do to save costs or if you need to you know emit different kind of observability the framework is extensible and easy for you to add this kind of functionality to it on top of your Hamilton data flow and then lastly Hamilton is not only for llms you can use it to model any data flow so if you're doing traditional data processing or feature engineering or model pipeline uh Hamilton can can be used for that and so why add um you know more tools when you can kind of consolidate on one in short just to summarize um thanks for listening if you want to get started you can scan the QR code that will take you to the GitHub page with loverstar it's on uh your pip install SF Hamilton you can be up and running in less than 15 minutes if you want to go ahead and try Hamilton now you can go to tryhamilton.dev it's running piadite in the browser and otherwise we have a community on slack and if you want to reach out to me or have questions I'm more than happy to chat to you uh and otherwise if you're interested to know what I'm actually building at dagworx feel free to sign up everybody to chat about that as well thanks for listening