Rust AI: Melanie Warrick & Melissa Herrera, Rust Confessional | Durable Agentic Loop on Temporal
Hi. All right. So, we know that you all have deep dark coding secrets that you need to confess. So, if you want to confess those, feel free to scan this QR code and submit them. We're not going to track you at all or anything like that. Um but I'm going to show you what it looks like when I run an example and we'll find out if this works or not. And as it's running, we both work for Temporal and we're going to see how while it's working, uh you can you can actually watch that the agent is doing things, analyzing what I just submitted and thinking about it and saying, "Oh, you know, how bad is this uh automated agent that uh what was it? An automated agent is receiving HTTP 429 response because oh, oh look, some people are submitting stuff. All right, that's awesome
Um we've got another confession and Ferris is judging you. Now, while this is processing, what I'm going to show you is that it's in a holding pattern. And what's lovely about the way Temporal works is that when I I can I can leave it in a holding pattern for results for as long as I want. I can have a long-running workflow and then I can unlock it and it will finish and it'll give me the results of that initial confession that I made and it'll give me my penance. Got to do uh add bounded exponential back off with jitter, retry limits, and tracing for each attempt and test it again against the scripted 429 sequence. Okay, so Temporal and Rust. Rust can fix that. So, I'm Melanie, this is Melissa
Hello. And we're going to get into giving you some Let's start with the basics about agentic loops. >> Yeah, yeah. Uh Thanks for the amazing intro, Melanie. And I hope you all got your confessions in. If not, the QR code is there on the bottom to keep submitting those. But what you just saw was a glimpse of just what Temporal does. The left side, not the right side
The right side is the fun part. But Temporal handles all the hard things for you. Like agents. And I mean, the agent loop itself, we have this simple loop that we're showing here. Could be simple, right? We have the deciding reason. So the agent decides what to do, and then acts on that with maybe tools, maybe some API calls. And then it observes, and then it folds it back into the agentic loop until it reaches that goal or stop condition or even failure. So I mean, everyone here should be familiar with that, yeah? With agentic loop? Raise your hand
Yeah, yeah. There we go. I mean, this is Rust in AI, so I hope so. And what that looks like in Rust, and I'm not going to claim to be a Rust expert, but zooming out at a high level, we see here we have that loop defined. And that reasoning part where we say let decision, we have an await. We decide to act on that. In the Rust code here, it says wait to await and then use some tool. And then we decide the verdict and return that back until it's decided that it's reached its goal
So this is what that loop looks like in Rust. >> Yeah, that's a very basic abstraction when you want to think about agentic loops. So it can choose The loop itself can choose to do a lot of different things, like the this research or verdict or use other tools or use an MCP, even create its own code. There's all kinds of things that a loop can do for you. The thing is, the loop is easy. And we know, as Alexius was saying saying earlier, Uh, there's a lot of things that can go wrong when you put in a loop out into the wild and when you try to make it work in production um and there's things you want to fix. So, a lot of things that are being discussed right now is how to create scaffolding and harnesses and other types of things that will make that loop be able to survive in production. And you hear about memory tools, uh safety, safety's huge right now
Um but in particular, we think a lot about durability. So, that is why of course we we work for Temporal. We're going to tell you more about Temporal, but durability is this whole idea of being able to maintain what's going on in that loop, what state was it in, where did it leave off, what if something happened in your distributed system that required you to have to like recover from. So, what does that look like? >> Yeah. And so, what you just saw in the first 2 minutes that Melanie was demoing was exactly that. It was an agentic loop happening. Um on the left side you saw some things kicking off, right? So, from the right side we had our rest UI earlier. Um that's our client kicking off some type of workflow
Thank you, Melanie. So, um just a refresher on the left side, that's Temporal. That's showing each of the steps that the agent is taking to then um show its uh state of completion. The right side is what you saw as the UI. This could be any application that you're building today for us. Um and so, if you head back to the diagram, thank you. So, um yeah, client kicks off some workflow. So, in this case that's when you submitted uh when a submission comes in for the rest confessional
And it's kicking off the workflow. Inside that workflow is the loop. That's where the loop lives. Um and what we're doing and what Temporal is doing is that um you orchestrate this in this Temporal workflow. And each step uh that that agent is taking is persisted. We track the state and so that every every step that the agent is taking is durable. The whole workflow itself is durable. So, that at any point in that agent's process, um say that a network were to drop, API might be flaky, network partitioning, that's no that's no feat to Temporal because Temporal is managing and ensuring that that whole agentic loop is durable and not losing state in the process
>> And I'm going to mention the fact that the activity is your non-deterministic, like if you're calling the LLM or if you're making any tool calls or if you use any kind of API calls, that would be held within our primitive, that's the activity. While the workflow is really tracking the steps of the loop. And the other thing, Temporal is open source and free to use, so you can experiment with it without having to pay us any money. >> Exactly. And so what this slide showing is the same zoomed out version of the Rust code that you saw the agentic loop, but built with our Temporal Rust SDK. So we do have a Rust SDK, that's why we're here today. And same kind of what Alex was saying is that they were, you know, existing while Temporal doubles built in go and we've invested a lot in our Python community with AI, but now we are also seeing a lot of Rust developers emerge today and hence the creation and our investment into the Rust SDK. So here it's the same loop, but on top of that what Melanie just mentioned were activities
So we have an enum up top that defines what those activities are. So there are several that are happening in the actual app itself that we're showing you, but here it's showing like call model for example, that's non-deterministic. The the research query, that's also non-deterministic. You put those or wrap those I guess into an activity as defined that here at the beginning of the agentic loop. >> Yeah, the CTX activity there for example is what will say capture that in our event loop in the log of that. Okay, so what we want to do now is show show what it's going to look like when we run this and then we do some kind of network partition. So, I'm going to submit this uh seed I'm going to seed some examples. You are still welcome to uh confess as many programming sins as you may have out there
Uh and unfortunately, we don't have the side where it texts you back, but if you want to find out the results, feel free to find us after. So, it's currently running. It's currently trying to decide what kind of uh final ratings and results should it give you. Meanwhile, I'm going to say let's mark let's partition this worker. So, it's going to take this offline. And when it goes offline, it's going into a pause state. Now, I could leave this in this position for as long as I want. Uh granted, you can set up certain timeouts and other things that are going to be pre-built for you, so you don't have to think about it
Um but at the same time, I can leave this workflow I can leave this pause. I can move the workers around. They can get isolated for some reason. Um and then I can also say, "Hey, I'm going to submit something into the system." This means I've just said like release uh that decision. Say it's now it's time for you to actually make the call and and make the determination with the final actual uh send whatchamacallit the the results the judgment is for what you have confessed. Now, you'll see it says the workflow task timed out. It doesn't mean it's died cuz when I bring it back up >> at one system boundary. So, we saw that this workflow, all of these workflows were able to complete whether the system was on online or not
And the reason why is because Temporal service is storing the state. And then when a workflow disconnects like that and reconnects, it will replay. It's not going to redo it. It'll replay to the point it left off and then it'll drain the queue to recover anything that might have been staged and ready to submit. And it'll basically make sure that your state is managed in your system. >> Yeah. Couple things I'll mention is that the Temporal Rust SDK, it basically plugs into your standard Serde ecosystem to handle your workflow data across those boundaries so that that it's already handling the standard input and outputs that you're you know. Um and also Temporal is that the underlying engine is based off of Rust
So, like we have the Rust SDK so you don't have to leave the Rust ecosystem. Um but we are also running off of Rust. Uh and as I mentioned, it's open source, it's out there, you can check it out yourself. Uh the what you just saw is in essence this text UI that we had. Uh we had many different workflows and you can have a like you have hundreds, thousands of workflows running. They're all running in maybe one worker, multiple workers depending on how you split out your your system. And you've got this UI that you're interfacing with. So, basically, the main things we want you to take away >> Yeah
Yeah. Um so, a few takeaways here. Um building you can build your agent loops in Rust. I'm sure some of you are already doing so. Um so, the reason act observe loop. And then as uh Melanie was mentioning, support support of Serde at the boundary. Um secondly, making those existing loops that you have in Rust durable with Temporal. Um we have this concept of durable agents, especially when you're shipping production agents
And we don't want to have to face those issues after the agent has already been shipped into production. Um Melanie, I'll let you >> Yeah, I'll take the last. So, yeah, and we got a Rust SDK. Uh a couple things. You'll see we have GitHub. Um pointed you to the link where the Rust SDK lives. But this last thing I've listed up there is new for us. So, I mentioned about how we're about this durability piece
We also are now with We have an experimental um temporal agent harness. It's newish. It's up on our temporal community uh GitHub. So, you should totally check it out. You should message us and let us know how it's working for you. Um but we're working on standardizing uh some specific functionality around uh harnesses in particular for agents and supporting agents. And I just lost I went to a black screen. That was fascinating
Uh there is another slide, but I think maybe Claude just died on me. Connections. Anyway, um you're welcome to come up and find us afterwards. I'm going to see if this does recover and I can give you my last slide. Ooh, you can see all my notes, too. Perfect. So, yeah, if you wanted to check out the repo that we just showed you, this QR will take you to the demo. And if you want to play around with your own confessional and make it how you want to make it, you do have to get yourself a Twilio number if you want to make it work
Um and yeah, I appreciate y'all checking this out. last but not least actually, I'll do a plug for the fact that tomorrow night uh Melissa's going to be running the durable multimodal AI meetup. So, you should totally come on in and say hello more to us. We would love to see you there. Uh Melissa runs all kinds of uh events around AI in San Francisco. And if you want to speak at one of them, you should also give us a shout. Okay, there's my plugs. Anything else? >> Um no, that's it
Uh thank you again for everyone uh having us here at Avaya Scott for bringing us in. And yeah, if you have any questions, feel free to let us know. THANK YOU. >> [applause] >> ANY QUESTIONS? >> YOU CAN'T HAVE ANY MICS. IT'S ONLY for us. >> [laughter] >> Are there any other questions? I thought it was really good. >> Thank you. >> [laughter] >> We take comments and compliments, too
So, >> take that, yeah. Thank you, everyone.