Lakeside AI: Bringing Models to Data with Open Lakehouses & Agents) | Starburst , AI By the Bay 2025
Welcome everyone and uh a big thank you for joining us. Um there are lots of open seats. If folks want to come closer if if you are uh um going to struggle to read if you happen to have small fonts please please uh come closer to to the stage. So over the next uh 20 to 25 minutes, my colleagues and I will share a blueprint of Lakeside AI uh that you all can use to bring AI in a governed data rather than ship sensitive data to external models runtimes. Right? But before before we do that, let's let's uh let's introduce um the Starburst team here. >> Um hi, I'm Mana. I'm a software engineer at Starburst. uh been at the company for close to three years now
>> Hi, I'm Martin Traverso. I'm co-founder and CTO of Starburst and also one of the creators of Trino. I've been uh working on that project for the last 13 years uh since I we started working on it at Facebook. >> And I'm Jender. lead uh engineering at uh Starburst and I've had the privilege of uh building data infrastructure and data platform along with Martin at Meta and also at Netflix where we use a lot of opal standards like iceberg and trino. All right. So I assume all of us in this audience are engineers and we have been tasked to build the world um where um LLMs have uh access to all enterprise data they need uh without any privacy risk without any brittle pipelines without moving data especially your crown jewel data into external systems. a world um where anywhere in the company, I literally mean anywhere in the com any anyone in the company, an analyst, a product manager, a support rep can use AI powerfully without learning prompts or tools and a world where AI agent work alongside us uh and and make every employee a 10x uh individual
This is this is the this is the promised land for enterprise AI and this is where we have been striving all of us have been striving to be right now. Uh but but the sad part I think that the sad part is that we're we're not there yet. We're not there yet. Um the sad truth is that most enterprises are not living in that promised land and in in fact most AI initiatives end up today in in in this in this graveyard of enterprise AI and and it's not because the models are not powerful enough. Um it's also not because teams do not have the talent. It's because the architecture itself is flawed. Right? If you if you think about it, the two the AI depends heavily on moving sensitive data to external LLM services, which means you're duplicating your data across vector databases, caching layers, search engines, and you're building a lot of ETL pipelines and you also have your proprietary gateways and then then u you're you're kind of uh copying a lot of data and and as a result uh you now have to worry about governance and governance as you can imagine gets fractured when you have uh so many different systems. Your your pipelines are breaking, your data is drifting
What is happening as a result? Your models are starting to hallucinate. It's it's because they do not have access to the right context. So what's what's uh what's the what's the result? Result is that we all are spending massive amounts of uh resources. We're seeing slow time to value and uh we're also delivering AI outcomes that cannot be trusted. So this is where a lot of AI projects are stalling out. So question for us is that we have seen the two worlds the the world that we aspire to the world we want to go to. How do we how do we bridge the gap between the two worlds? And and and I think this is because of the deep architectural divide that exist and no amount of fine-tuning of these LLMs or model swapping can fix that. We need a new architecture
A new architecture where the data stays inside the lakehouse without copying data where models and agents come to the data through open standards where context is is delivered uh through data products and metadata and where we have a unified governance that manages data and models both in a unified way. This is the foundation of Lakeside AI and this is this is this is how we move from vision to a to a reality. This is the blueprint. Over the next uh uh few minutes, we will actually build this blueprint step by step. This blueprint, this architecture finally connects the promise of AI to the reality enterprise live in today. With that, let me uh bring in Manik to help us with with step one of the blueprint uh which is building the lakehouse layer. Um thanks Judendra. Um so as you can see in the last slide um so we say iceberg lakehouse is the foundation on top of which you can get u the value of AI
So in the next few slides I'll try to show what we can do to build out such an open lakehouse architecture. So so at starburst we believe iceberg is at the forefront of the open lakehousebased architecture. uh we have already helped our customers build, optimize and operate a iceberg based lakehouse also called ice house at scale. Now a typical company where you are probably working at is probably having data fragmented at in multiple storage platforms like operational databases, data warehouses or message cues etc. And um having all of this data ingested into a lakehouse uh can make it the single source of truth for LLMs to use at context. Unfortunately like there are many challenges to building such a lakehouse. Uh first like um you your business data is continuously evolving. So you want your data warehouse to be like as fresh as possible which is very hard to do
Um f then you need to connect to disperate set of data sources for which you are probably writing custom pieces of code uh requiring a breath of knowledge um that makes it very hard and lastly um uh you're like you are probably using some complex set of tools stitching them together to build out these pipelines to ingest data which can be very hard to maintain. Uh and we faced this problem at Starburst when we were handling our business data. And uh we realized that a nice solution would be a managed iceberg based pipeline. And when we started off building this out, we had three main goals to abstract away the complexity uh so that data engineers can focus on business logic, build for scale and number three keep the data as fresh as possible. So today we support streaming and batch injection use cases. For streaming we use uh we we support Kafka API compatible sources and for batch S3 ones. Uh not only do we help with landing the data, we also help with maintaining your data. So iceberg table formats um kind of um have a lot of metadata uh which is needed for query planning and often times that can become a bottleneck
So you need like some sort of metadata expiration and compaction there as well as data compaction. Uh if you have worked with high throughput systems, you have probably heard of small file problems. And all of this gets automatically handled by our platform. And we call this starburus manage table that is continuously getting data in and being maintained a live table. [snorts] So that's good. Um but there are other solutions out there. Why do we use starburst? And uh what I'm going I'm hoping to convince you is that like we face these problems. So we have built it keeping them in mind
So we have a low code no codebased solution to get you up and running as quickly as possible to ingest your data to these lakehouses. Um we have uh like not only built ingestion and maintenance, you can kind of build out your medallion architecture by transforming your data off of these um first ingested tables and um who among have who among us have not faced this problem of like schema changes breaking your whole pipeline. We can handle all of these configuration changes seamlessly across all this pipeline for you. Last next we also built for scale and that's what we have seen in our internal testing and in our customer adoption our systems work at very large scale uh for all your use cases. [clears throat] Finally uh we all know that incon like some small imperfections can creep in unwantedly causing catastrophic changes. For example, you might have a poison pill message in your messaging queue which breaks down the system completely or you probably introduced a configuration change that you were not planning to which makes the data unqueriable and we have uh features in place to help you correct those mistakes. Um for example, if we detect some mistake some error in your record, we notify you uh in a or and dead letter it as well so that you can kind of fix it downstream. Um, additionally we support something called a reset and replay where uh if you have introduced a configuration change that was you can reset to the last known good point and replay all the records to fix the problem
So so far I have primarily focused on structured data uh but I would say iceberg is a good format for uh or definitely becoming one for unstructured and semi-structured as well. uh with its new variant type uh in v3 form spec. Uh not only that, you can also like just storing data may not be enough. You need uh indexes to support some of your rag workflows. So iceberg can already stores stats for data analytics workloads. But you can imagine we can extend it to support uh vector embeddings and full text indexes for your um rag workflows as well. So in the next few slides I just have an example of uh a table that where we wanted to kind of generate embedding for uh uh the transcript which is kind of like a blob text and uh so you can kind of create a extra column here which is an array double that represents a vector embedding today and uh you can generate it by something like a SQL query that we all uh which says like we do an update with a and you can see there is a function called generate embedding which is a function and it lets you generate these embeddings uh for you and lastly once you have this you can use some of our similarity functions to do the rag workflows natively with your lakehouse uh and with that I'll hand it over to Martin to talk more about how to use TRO for your query workloads. >> Thank you my so uh we have now we have all this data in in in the data lake
Uh so the question is what do we do with it? How do we analyze it? How we access it? How we derive insights from it? how how do we join it with operational data sets that may not be on the on the data lake and how do we make it available to AI systems and how do we integrate it into AI workflows. So uh for that that brings us to to the next step which is uh using Trino which is a a pretty powerful federated SQL SQL query engine. So, Trino was born at Meta U 13 years ago to support the massive uh workloads and data sets and analytic requirements that uh engineers and data scientists and data analysts had at the company at the time. Uh we open sourced it. It has been adopted by thousands of companies and organizations uh over the years. Uh it's fast, it's scalable, it's efficient. uh I mean it can run at meta scale, it can run on Netflix scales etc. And one of the unique features that Tino has is that it supports um federation
It's it's integral to uh to score to it core and it it it allows you to run uh analytics over your data lake and at the same time uh bring in data from other data sources and and and basically merge it all to all together and derive insights from that. So we think that TRO is a perfect platform for uh bringing your data lake together with the uh power of LLM and AI agents and AI AI analytic work workloads. So if you so on top of Trino I mean SQL is the core the is the the language that Trino speaks. Uh SQL is how many of you know SQL have uh implemented anything in SQL? Okay. Uh so SQL is like the lingua frana of data analytics. It's supported I mean it's been in existence for 30 40 years. It's been uh is supported by the vast majority of analytics vi tools. Anything that has to do with data understands SQL
So uh it's familiar. It's it's it's it's not the greatest language. I mean, as as someone that has spent years working on on the SQL implementation of TRO, uh, parsing through thousands of pages of SQL specification, I can tell you it's not pretty. It has all these corner cases sometimes makes me want to throw up. Uh, but it's what we have. It's uh, it's what people find familiar. It's what people are used to. So using SQL as the core as a the the layer through which you analyze your data and you integrate uh uh into LMS and you drive your AI workflow seems like the natural conclusion
Um and then of course with Trino and the federation capabilities it presents a a very unique opportunity of act of of using data lake data uh doing analytics over the data lake and running um your your your analytics uh or integrating your analytics with the power of what LMS can do. And how do we do that? So one of the things we we support is a set of functions that allow you to interact with LMS right there and then in your analytics. So you can write SQL queries that um you can process data through an LLM and you can uh you can do some specialized computations. For example, we provide a set of functions that allow you to do text classification, sentiment analysis. And these are things that you would otherwise have to write Python programs or or programs in other languages like to combine your data from the data like read from iceberg and call out to LMS and that's very complicated. So we believe that uh doing this through SQL it makes it much more accessible to to users. And then of course if you if none of those functions uh satisfy your requirements there's always the generic prompt function. You can you can call to an LLM uh com create a prompt from your data provide provide that tom and then get responses back
But what about users that don't know SQL don't understand SQL and that's represent the majority of all the business users uh that we see in our customer base. there's a growing demand uh from our customers to have access to more sophisticated uh smart assistants. So customers want to be able to do all these things with data uh derive insights um build comp more complex workflows and and and and workflows without having to learn a programming language without having to do uh learn SQL. Uh so for example you may have the CEO wants to do wants an anal run an analysis of ask a question today they have to talk to um some employee to go and run a create a report for them they had to wait a few days for that in the best case or you have a product leader trying to understand do churn analysis on customers uh to figure out why are customers abandoning the the product that requires engaging with the data data team and IT and creating all these pipelines and reports and dashboards and it may take a long time. So the time to to that value is is very very um very short very very long. So um so there's this demand for for being able to to bypass all those layers and and be able to do that much much much faster and much more efficiently and increase the productivity of the business users. Um so what we're we're proposing and and this I guess the direction the industry is going uh is creating AI agents that allow you that that perform the job of those smart assistants. So at Starbucks we build a and of course the the the natural interface for the for those agents is through uh language natural language and that's the the LLM revolution over the over the past years have has made that possible is making that a reality
So at Starbucks we built a an agent that allows business users to interact with their data um and this can this makes them a lot more productive. They can they can derive insights. they can um they they can do more complicated analytics that would otherwise require uh very complex implementations and the agents can just by taking using the power of LMS can reason through multiple multiple steps to figure out complex answer to complex questions and and then potentially even feed into downstream downstream uh workflows like after you done your analysis uh create dashboards or or create u some some report that gets refreshed on on an ongoing basis. Now the biggest challenge with um agents and LLMs is that they they are trained on internet data publicly available data. So LMS don't have access to your business specific data your the context of your business the business rules etc. So they tend to produce make a lot of mistakes they tend to uh hallucinate a lot. So we think and we found from from uh experimentation and and things we've tried that providing additional context about your business data your um the structure of your data the relationships uh glossaryy of of what business terms mean for your specific domain is critical to make these elements perform perform well. So at Starburst, we package those uh that business data into what we call data products
So data products are a an abstraction over your data and metadata about that data that describes relationships, describes documents the the the data and the metadata and pro provides examples of of how to use how to access that data and that gives the LMS context to to perform uh uh perform their actions more accurately and pro produce correct results. So finally and we we so the the industry like is a a massive explosion of uh uh solutions and different different things that are happening every single day and in the AI space and and we can't uh we can't assume that our platform is going to be able to do everything. So we believe that opening the platform to other tools and other um uh basically the entire ecosystem to for for business to build their own agents and their own their own tools to adapt them to a specific needs is very very important I think and we think that's going to uh potentially increase productivity by an order of magnitude more um uh by letting businesses uh do with our platform uh what they what they need for for their specific use cases. So we are exposing and and we believe this is an important thing is to expose the capabilities of the platform through MCP so that external tools and agents can uh can use the the the power of the platform in their own agentic workflows in their own uh LM interactions. Uh so with that I'm going to hand it to Jitender who's going to talk about the the next step of uh this architecture. >> Yeah, thanks Martin. So, so far you have tune seen two stages. You have your lakehouse built and and you also have your uh intelligent access layer which has rich semantics that get fed to the the model
So, you can build your agents and open it to uh protocols like MCP. The next critical step is always governance, isn't it? Like models now need same governance and and and rigor that we have applied to the data. So just like data models need access control permissions and policies. So if you look at the lakeside AI architecture right like and and and this is where many projects uh either stall or completely fail is because of lack of governance or companies trying to figure out how to introduce governance. So uh in in in your open lakeside architecture you need model access management which lets you define who can call which model when and for what purpose all inside the same governance framework that we have applied to the data. Uh and then you also need to think about uh model management because it's it's not always one model that fits all. you need to have multiple models get that get configured in your in your uh lakeside AI. So it's it's important to know uh the governance applies uh regardless where the model runs in the cloud in in VPC or a completely airgapped environment
That's the unified governance. So any lakeside AI architecture you have been tasked to build you have to think about both access and management and and that's not it. You also have to think about model monitoring and rate limiting. What basically that means is that once you have registered the models and you've defined who can access each of these models, the next step is to track how these models are actually being used across the business. Right? the the uh information that you need is is the at that fine grain visibility uh usage patterns by model, by user, by team and by endpoint. And this will help you solve a huge problem that every enterprise faces right now because the LLM usage is exploding. Costs are unpredictable. You must have heard horror stories about agents getting in an infinite loop and costing companies tens of thousands of dollars which means you need model management and rate limiting
Uh so you need to know which apps which agents are driving consumption. So that's another key um construct you need in your in your lakeside AI architecture, right? uh and and that's that's that's uh that comes in built in in certain platforms like Starburst. So let's let's close let's bring everything uh together. So step one you build your lakehouse. Step two, you put your intelligence layer and step three, you bring your governance layer. And everything that you have done is in this step-by-step approach is uh you you've taken you've taken open standards and you have assembled your lakeside AI architecture uh to build a trustworthy scalable enterprise AI. And it's not again it's not about just pushing more data to more LLMs. That's not what AI is
A AI has to be scalable and trustworthy and it happens when you invest in building uh using open standards. So what we have shown you is you're reversing that flow. You're bringing models to models and agents to the data inside your govern lakehouse when you combine iceberg and and trino and data products uh which is your uh context layer search SQL functions because you want your data scientist analysts but you also want everyone in the company to be uh interfacing with these agents under a unified governance. you get one architecture, one place to govern, one platform for analytics and AI and suddenly the the land of enterprise AI stops being a concept and it becomes a reality. [snorts] So um this is what we have been working at Starburst for for for many many years as as Martin mentioned Trino was born at at Meta and Iceberg was born at Netflix. So these open standards have completely revolutionized how we build the modern AI and analytics architecture. Now you put MCP into the mix and introduce unified governance for models and data. You have one platform that that helps you address all your AI promises that that your company has tossed you with
So not we don't just stop there. A lot of our users are bringing their own models and also data sources to build these workflows. Now, we we didn't do a demo here. I would encourage you to stop by our our our booth on on the ground floor uh to see a demo of all of these things in in work. Uh and and I I I think the engagement, the discussion doesn't just stop here. We would love to continue to have this conversation with you outside of this room. uh we are growing we are growing aggressely across Bay Area, Boston, Varsa and Bangaluru. So uh we have many roles open
So check it check us out at careers at starburst and then um if uh some of you are interested in taking this for a spin uh go to Starburst Galaxy or just just search for Starburst Galaxy and and uh get a free for life cloud account. And finally, if you sign up for our newsletter and and enter uh you could you can actually get this t-shirt. This t-shirt is also available at at our booth as well. So, and uh thank you so much for engaging with us. Hopefully hopefully you learn the building blocks for an open lakeside AI architecture. Um and I think we have few minutes for a couple of questions, right? So, yes, we have we have few questions. But again, thank you for for participating in this session. Any [applause] questions?