Jamieson Leibovitch, Uber, on Reliable AI — Interview with Alexy

Jamieson Leibovitch, senior engineer at Uber, on reliable AI at AI By the Bay 2025. He led Query Copilot, Uber's text-to-SQL application for Presto SQL that lets users generate queries and reach insights faster, his first deep dive into AI. Reliable AI means agents that solve problems with high quality and can be trusted to take an action, which comes down to high-quality evaluations and an evaluation pipeline that shows whether each prompt change makes things better or worse. Uber went from no AI to basic tool-calling agents, static multi-agent systems, and now dynamic ones; the next step is task-management-oriented collaboration, unless the models get good enough to need none of it.