Rust AI: Dale & Dalton Prescott Ng, Extended context for AI agents with pgGraph & pgContext
Everyone, uh so nice meeting all of you. I'm Dale. This is Dalton, and this Damian. So, the two of us are co-founders of Evoqore. Uh and as he said, me and Dalton are identical twins. So, we actually came to SF about just 4 weeks ago. Um we have been building our startup since November of last year. Uh but we decided to YOLO and just come to SF
Um and since then, we've actually grown quite a lot. So, we raised our pre-seed from Founder Sing, uh and we've gained a ton of um traction and working for a lot of startups now. So, our mission at Evoqore is actually very simple. We want to make the world's data AI native. Um because basically the problem that we see in the world right now is that there's more and more data being produced due to AI, but the technology we have just isn't efficient enough to reason over it. So, we're reaching a point where basically there's more data being produced, and our technology is still over here. So, we can't see and understand and reason over that data. Okay
So, I'm going to assume everybody here knows Grapher. Who here doesn't know Grapher? Okay, good. Only a few people. So, just a very simple explanation, Grapher is vectors and graphs. Vectors let you store the uh let's tells you how similar data is, and graphs tell you the relationships between those data. And for you guys, you're all technical, this might be easy to understand. When we actually went to talk to like enterprises and also more engineers out there, the question that we got that people kept asking to us was actually something very funny. >> [snorts] >> So, this was actually the biggest challenge that we faced at the start and that a lot of engineers don't actually understand graphs, why to use them or how to use them
But, to us it seems obvious because we use so much of it. But, graphs are very simple. It is a better way of displaying and understanding your own data when you need to understand the relationships between it. So, for example, when you are asking questions much like the ones that AI agents today do, these are all graph-shaped queries. Hence, why we wanted to have a full graph like database inside Postgres and PG Vector has been around for the longest time. So, we decided to add the missing component. So, PG Graph is a library that we launched a couple of months ago and that we've been working on heavily ever since. It is a full native graph database engine inside Postgres and it's built in Rust using PGRX
So, this is available on GitHub right now. It's Apache 2.0 and we just released our version 1.0. So, these are the kind of questions that AI agents today need to ask when they are looking at data and these are all graph-shaped questions because these are normally a mixture of multiple SQL queries when you're trying to ask it. So, when you try to put these queries onto databases today without using a graph engine or graph database, it tends to be very complex to write these queries. They could be a recursive query that you have to write in SQL or it might be very slow to run. The good thing about Postgres and other relational databases is that data is already arranged as a graph. When you have put in your different tables like users, orders, payments, disputes, and connected them with foreign keys, you've actually built a graph without realizing it. The difference is that SQL doesn't really expose this layer to you very well, and there are some newer updates to Postgres that have tried to add a graph layer, but it's still using the same query planner that Postgres has
So, because we were building our own AI applications and we were building things like company brands and internal AI brands, we realized that the performance of the options today for Postgres the they're really, really bad. So, we needed to find a way to have really good performance, and only way we sought to do that was to put an actual graph engine to Postgres. So, the way that we did it is pretty straightforward, but basically we allow you to build an index that arranges your data like a graph, much like how you build indexes in Postgres. And we have some smart stuff behind the scenes that help to manage these indexes automatically for you. Second, we we do support a subset of Cypher, and we have full GQL support, but we realized that a lot of developers that are using graphs for AI aren't the same engineers that are using graphs today. So, a lot of them actually just want a simpler way to use it. So, we have our own SQL API. It's designed to feel like native SQL
So, in PG graph today, you can do things like traversals, you can find shortest paths between any two objects. You can do neighborhood searches. You can search for nodes. You can do collected component searches. These are all the actions and the the queries that AI agents need to make over database. Now, the sixth one that you see over there is actually the one that made a lot of people we talked to very excited because what we did was we put ability to build these graphs as full multi-tenanted objects inside your Postgres. And also manage them in a way that you can do at scale. So, you can turn some graph graphs off at certain times
You can manage some as hot, some as cold. So, just by putting a proper graph engine in the Postgres, which is based on CSRs, we already achieved a 10 times speed up over doing a normal recursive query. You can see some of the query performances that we benchmark against LDBC. Um we did do a really funny test internally where we put as like a very large portion of the entire blockchain. And we were able to do very deep traversals in real time. So, when I say in real time, I mean under a second. So, the way that we did this was that the first thing we did was we put When you build your graph index, we're using something known as compressed path rows. This basically is where we represent a graph, which is a two-dimensional object, as a single contiguous array
So, you see that we are representing it in these three contiguous arrays, which means that This is why it's so fast because we can load these onto the CPU cache all at once. Now, CSRs do have a big problem, which is that they're immutable. So, what we did is that we have them built as bounded CSRs. So, this means that your graph is composed of many, many small CSRs. Each of them have their own index that tracks the difference. So, we can maintain real-time sync and compatibility. And because I'm Asian and MSG makes everything better, this was something else that we added. Which is called massively scalable graphs
So, this is very specific to the use case of people that are building agentic applications today. Which is that they want to build and have many multi-tenanted graphs within one database. This is something that's very hard to do automatically and at scale today. And also being able to manage those in memory. So, for example, I have a read replica and only keep half of these graphs hot. In this read replica, I keep this other half hot. And of course, this is I have to give props to the library that made it all possible, which is PGRX. The entire project is written in Rust except for some Python code for some benchmark scripts that we had running
And this PGRX is really good if you guys haven't tried it. It gives you a lot of access to the to You can use a lot of Rust within even with Postgres limitations. So, it's very easy to try. We have a quick start script. This has been on GitHub for a while now. Um and on our website, we also have a demo that you can run in the browser. So, the other stuff that we have been working on just to very quickly share. This was a library we just launched, also built in Rust on PGRX, called PG Context
This is our replacement for PG Vector. It already We already achieved a 5.3 times speed up over. But, more importantly, we brought a lot of the features that developers want to have from children into P into Postgres. So, this is designed for for multi-million vector scale. And also, you can do things like hot metadata filtering. You don't need to pre-build indexes. You can filter by vector search together with a hybrid search together with certain columns as your metadata you want to filter, and it will merge it all into one query and we rank it for you. So, that is what we need to complete the puzzle for us, which is that we wanted to deliver full graph right without leaving Postgres
And that was Now, we just launched our hosted version of the platform. Uh currently, everything is free. We are basically just testing to see how it scales. It is a fusion of PG graph and PG context, and then a Kubernetes layer that we built to scale it. >> Yeah. And we like to call this context window that can become the size of your database and infinite context window for your AI agent. >> So, everything inside of the PG context and everything in PG context and the PG graph is fully open source. It is Apache 2.0, so break it, start it, ship it
Uh and you can try our hosted polygraph.com completely free right now. Thank you, everyone. >> [applause] >> Yeah. Question. >> Any questions? >> Hello. Um I'm very excited to see follow the single parents here um and also doing graphs. Very cool. I kind of wanted to ask because you mentioned like CSR
I forgot what it stands for, but there's sparse somewhere in there. And I kind of want to ask like a big challenge with graphs is usually like one particular like helpful kind of thing. >> One particular what? >> Oh, like you know when you have hot nodes like there's one central node and there's a lot of things going >> Oh, like a super node, you mean? >> Yeah, like a super node. Like Justin Bieber knows a lot. >> Yeah, I know what you're talking about. >> yeah. So, how do you guys handle this exactly? Or is it >> Okay, so super nodes is a big problem in graphs, but we have a easy and and a more useful solution to this problem. So, the easy solution is if you don't want to use if you just want to use PG graph, then you can set a from degree limit
So, if you hit a super node like Justin Bieber, um it'll cap out after the limit that you set. So, that's that's the easy solution that we have built in. Now, I did have a long discussion with team about how to solve this. So, the problem with super nodes is you need to have an idea of the direction you're going. The good thing is that when we put in a graph frag layer, we can use vectors to do that. So, when you combine when we combine PG graph with PG context, we can re-rank the all the relationships attached to a super node using the the vectors that are on those layers and then use that to traverse past a super node. >> So, it kind of just breaking the edges based off of which is the most important path out essentially? Or is it? >> Correct. >> Okay
Oh, interesting question. Thank you. >> Questions? >> So, if you're not PG native and you have other databases, can you still use the PG graph and bring in data? >> Okay, so we are working on adding support for other databases now and we're also we also have a version that we can run in our Kubernetes where you don't need to have the Postgres layer itself. We can connect to any database now as a read replica. The The main use case of this would of course be if you have a database that you want to have a lot of in and out transactions and you also want to have a agent connected to that life context. >> So, you said you use foreign key for edges. >> Yeah. >> But, foreign key is one-dimensional
>> Oh, okay. So, So, the foreign key denotes that there's a relationship there. But, PG graph does is it creates a virtual table where you can set the relationship levels. If not, it will use a default one which is like table from table X to table Y. Then, you can change it to something else. >> Yeah, but right. Two nodes can have multiple edges between them with different contexts. >> Sorry, what do you mean? Like >> We can work in the same company, live in the same city
This is all type of >> Yeah, correct. >> So, you you do that with Postgres? >> Okay. So, if you either you add relationships for data that doesn't exist or we can use your existing foreign keys to auto build graph. So, if in the case like what you described, if you have with this like if you you can very easily model this in Postgres by just having like one table of nodes and one table of relationships and then just a recursive foreign key. >> Mhm.