Agents By the Bay: Nikita Rokotyan, Cosmograph

Agents By the Bay: Nikita Rokotyan, Cosmograph

Recording: Agents By the Bay: Nikita Rokotyan, Cosmograph

Another excellence. All right. Perfect. Okay. Nikita Rokotyan. Yes. Hello everybody. I'm Alexey >> I'm Alexey Kruber of the incoming architect at Neo4j

We're on location at Pi Bay the 10th anniversary edition in San Francisco with Nikita Rokotyan, award-winning data visualization engineer and co-creator of Cosmograph. He's going to talk about it in a few minutes. So we're going to not going to hold him long, but tell us first what is the Cosmograph? Yeah, so Cosmograph is the fastest graph visualization library for the web and for Python notebooks. So if you ever deal with a large network graphs then it's you know it's tricky to visualize those. So with Cosmograph you can go like beyond 1 million nodes right on your laptop. You don't need any server side back end. It stays right on your machine. So yeah, this is Cosmograph

So you showed me on your phone some beautiful uh visualizations, things rotating. Is this a web page? So is it running in the in the Safari on on So it is the technology itself is a web technology. >> Yes. Like any widget in in Python right now. Yes. So we run a WebGL Mhm. force graph simulation Mhm. that does two things

Mhm. First, it calculates the layout for the graph. Mhm. And then it renders it. Yes. Everything happens locally in the browser. Mhm. And you know when you work with notebooks, we say Python, but essentially it's a browser technology

Right. >> So yeah, it's a it's a browser framework. Okay. I mean I am a back end guy, so a lot of this kind of is not something I know by default and I think a lot of people now just fiddle around strings and come up with results. So so thank you for explaining this, right? Web browser technology is already like a platform. Right? A lot of script kids don't know like what's beneath it. Like the the fact that the browser runs a computer is rocket science. All right, like and how an OS works is even harder

So I'm just exaggerating a bit, but like WebGL I heard it. I didn't work with it. So tell us what WebGL is. So WebGL originally is a graphics framework. Yes. >> It has been created I don't know, for games, for 3D graphics in the browser. >> Right. But then some people came up with this idea that you can actually pretend that textures are your data matrices Mhm

and you can use shaders. So shader is a program that is being executed on GPU. Yes. And you can do computations. Yes. So what we do with Cosmograph, we pretend that our data are textures and we we run all the computer computations using shaders Yes. with the GPU. Yes

It's a GPU computation technology. Yes. But for the majority of the world it is just like a graphing framework. Right. And there is a new framework called WebGPU which has this specific new type of shaders called compute shaders which you can use to actually do like machine learning, AI, all these kind of things, but it's still very new technology. So in Cosmograph we rely on WebGL because it's supported by everything. Right. Again, sorry for newbie questions, right? Like as an engineer I want to dig deeper

So browser, right? Like normally it renders HTML. Like right like how does the browser know that your computer has a GPU and on iPhone it has its own weird little GPU on uh some Apple silicon and like maybe on your Mac or PC it has something else. How does the browser get a hold of the GPU to run WebGL? Well, it's very simple. You have an HTML, you have this call what's called canvas element. >> Yep. Yep. And then with this canvas you just ask like, "Hey, give me this so-called context." And you you say WebGL. Yes

And if it gives you the context, then you have WebGL. >> have it. Yes. Okay. The rest of it you can go deeper like there are some extensions. We rely on not all of the GPUs work and like the nature of these computations really depends on the hardware. Right. But put it simply, it's just you know, browser knows it

Right. So is this portable enough? So you write it once Mhm. for WebGL and it knows when it's running on the iPhone Exactly. >> on the PC it does the right thing. Yeah. Okay. So so your code does not change. So Cosmograph is a library which is written in what language? JavaScript >> JavaScript, yeah

and shaders. The shaders is its own language. You like vanilla JavaScript or TypeScript? TypeScript, of course. Okay, nice. That's good. I love TypeScript. Great. So so now uh you said uh you you you ran visualization studio, you want to know what it is

How did you come to do this? Like why pick visualization out of all things you can do with computers? Well, it was my passion since like my very early days. I was always interested in two things. One is science Mhm. and another thing is design. Yes. >> So I was studying physics at the same time working as a graphic designer, a rare combination. Mhm. >> I did PhD in physics

But due to my passion to design, I was experimenting with tools like creative coding. If you ever heard of Processing Processing, that's right. There's this framework, yep. Exactly. I was doing all these kinds of things. Yes. And I was trying to find a niche where I could do both science and design, creative things. Mhm

And data visualization became this thing where I I can I can do both and I'm really grateful for that. So I can do scientific um things. I can do creative things. I can make information beautiful Mhm. but still useful. Right. No, this is great because you know, I personally found that having even basic visualization changes things dramatically, right? So I work for Neo4j which is a graph company. We've been around for a long time, so we have a lot of different tools

And one of them is the desktop. So if you download free Neo4j edition which is actually fully featured. It's the same as the enterprise except for there's no like multi-user permissions or clusters. But it's a fully featured version. It comes with what we call now explorer. And so you can load whatever data it is in the graph, immediately see a graph. The fact that people see a graph dramatically changes the kind of dynamics because now they can click on nodes, see what what's inside the node. And even the basic operations you can do already changes people's attitudes

Right? They believe there is data >> Like it's not like you just throw gobs of tables at them. All right, they don't feel It's a great communication tool. It's a great communication tool. Right. So can you tell us a little bit about like what visualizations you did before? What was the most interesting? What most kind of inspiring things you like the best? Um well, I love visualizations and I used to run a data viz studio where we created all sorts of projects, not just the visual side of things, but also the data science part, the back end part. We were building these data tools that you run in the browser. Right. And I think the tools I like the most, those are the tools where you need to build some kind of a simulation

The thing we also did for Cosmograph. Okay. >> I really like working with this large scale large scale high performance point cloud visualizations where we have physics, you have dynamics because it is interesting Mhm. to work with those kind of things. They're challenging because you need to apply your scientific skills, engineering skills but they also visually very beautiful. Mhm. And that keeps you motivated to keep working on those. So we had a couple projects that won a few awards in the data visualization space

And uh yeah. Interesting. So again like as a total kind of newbie, I heard that people use D3 Mhm. which is like a library. And so whenever I hear like people want graphs, they say we like can we use D3? But apparently it's hard. Like you need to learn how to work it. All right, like I think any library with a lot of features becomes complex and people are lazy. So I wonder uh first of all, why not like like use D3? Like why do we need a new library? Mhm

And second you know uh kind of how do you envision using it? Like do people use it by kind of coding against an API? What's your kind of plan for to make it available to users? Well, first D3, I love D3. I think this is the framework that created the whole industry of modern data visualization. So we cannot underestimate D3. Right. It has a few problems. The biggest problem is performance. It's SVG based. Okay

So this graphics is just slow. It cannot render large amounts of data. Mhm. So this was the key thing why with Cosmograph we chose WebGL. Mhm. I have another data viz framework called Unovis Mhm. which is built on top of D3 and all of these classical charts. Open source tool, so feel free to check it out, but for this specific problem you when you really want to go like beyond hundreds of thousands of nodes you need to go lower level

You need to work with the GPU um to do things quickly. I mean like to run code that runs fast. Mhm. Um and answering your second question Cosmograph, it's it's many things. It's one name, but it is many things. >> Mhm. One, it is a web tool. You can just like go to the website and you know, just drag and drop your CSV Right

>> and have your graph visualized within seconds. Mhm. Then it's also a Python widget. Okay. >> if you're a data scientist, you can go to your Python notebook get your data and visualize in Cosmograph right there. Mhm, nice. But it's also a TypeScript library. So, if you're building a product that needs large-scale graph visualizations, then you can also use Cosmograph as a TypeScript JavaScript library React library Mhm

to do that. Mhm. And the core of the library called Cosmos GL is open source, it's under an Open JS Foundation. Mhm. Same foundation that hosts huge projects like Node.js, Express, >> Okay. Electron. Okay. So, it's available for everybody

Nice. Nice. So, when you say Python widget, how does that work? So, you're in Python notebook, you're using Python. Yeah. You get visualization, how does it hook up to TypeScript? Well, we use AnyWidget. AnyWidget is this framework that kind of like does a lot of these things for you. Mhm. Basically, you have your JavaScript code

Mhm. Then you have your Python code. And then, so there is a server, and you send your data. So, first you get your data from the Python side. Mhm. You encode this data. Mhm. You send it to the browser through the network

Okay. And then in the browser, you like you get the data, you initialize everything, and you render everything. Okay. Yeah, and it goes both ways as well. Okay, interesting. Uh so, that means probably that will be the bottleneck. Like, if you want to like large data sets, can they go through the same way? Yeah. >> It is It is a bottleneck, it's not huge bottleneck

I mean, you you can render still graphs that can like be hundreds of thousands of nodes, several millions of links. Mhm. And it maybe is going to take you, I don't know, 10 seconds, 15 seconds. Nothing. >> It's a small bottleneck, but it's nothing because if you if you try to use any other tool, it's going to take you, I don't know, hours. >> All right, all right. So, this is all going through the browser. Again, as a newbie, can this TypeScript just read directly from a database, or this is browser as firewall and like it's sandboxed, so you cannot directly load from a database? You can

It's an untested feature though. It's something that we have built, never really tested. But I'm glad you mentioned that because we use DuckDB in the browser. So, we use this WASM version of DuckDB. >> And the reason for that, the first version of Cosmograph didn't use it. Yes. And we were always going like out of memory. Yes

So, then we found that you can use DuckDB, you can like efficiently use the memory, you can use WASM. So, now whenever you send your data, it's been uploaded into the in-memory DuckDB, and we use that for analytics like for a very fast cross-filtering, it's it works magically. I'm happy to hear this because like small world, I just talked to Guido. He developed Python at CWI, Centrum Wiskunde & Informatica in Amsterdam. So, DuckDB was also created later, which are both very practical and efficient things. And Hannes Mühleisen talked at Amsterdam PyData a few weeks ago, I've seen his talk. And DuckDB is a mother duck is a sponsor for Yeah, by the way, conference which is coming mid-November, and you guys should attend it. So, this is all great

And so, this is actually I think the use case they emphasize that they're basically exactly the same. You have WASM DuckDB, and you have server, and they can talk to each other, and that's how you connect applications to the mothership. So, this is awesome. I love it. Uh now, again, newbie question. So, like long time ago when I did my PhD, I used R, and ggplot2 was amazing. Like, so you know, this was amazing. Like, so there is like a whole kind of grammar for visualization for science this people use

And so, do you have the same kind of language people can use to visualize things in Cosmograph, or is it just pure TypeScript or like like if you want to create multiple objects and you want to reuse visualization or compose them, is do you have some kind of language or you do you just program it in the in TypeScript? >> Well, since it's it's kind of a single use case just like graphs or point clouds, Mhm. then there is no language. You just you have this widget, and you have properties. Right. You can say like here is your points. Yes. >> Here is your links. Yes

Uh get color from this column. Yes. Get size from this column, etc. etc. etc. So, you kind of like describe your configuration. >> Okay. You can add a timeline, you can add a few things

So, maybe you can you can say there is like some kind of a language, but it's this it's really small. But do you then do things to it? Like, you turn it, you rotate it, you zoom in it, you want to look inside of it. What do you What if you want to superimpose more? Like, multiple graphs, can you do that? What if you want to like one take one graph impose it on the other graph, and like visualize them both together? >> Well, if you do TypeScript, then you can do all this with our library. >> Right. And color one graph one color, and TypeScript the other color. Yeah. For example, yeah. Yeah, yeah, yeah

So, with with the Python widget, you can maybe you can just, you know, uh merge your data sets and just give the give different colors. Give different colors. But still render it as a single graph. Right. I mean, what what really struck me, and I talked to Thor earlier, this is very unusual startup you're thinking of product because you're doing things for humans. Nobody cares about humans anymore. Everybody cares about the AI scale, right? You want to feed enormous machines gobs of data, right? Gazillion billion bytes of data. You want them to figure out the patterns

You want them to show you beautiful things because like you're like passive consumers, just sit back and like are amazed. So, but you really want humans to look at it, you want them to turn it around. So, I'm thinking essentially if this product is for end user, that's why I'm asking for the language or some kind of interaction because eventually like I want to do a lot of things. So, we, you know, we talked with Thor like if this is used for exploratory data analysis in multi-dimensional space, effectively you it like it like what if like an LLM discovers some kind of basis, you know, orthonormal basis in this space, decompose in different dimensions, you're in finance space. Like, you want to see like I want to study this financial data, you know, is it like highly volatile low like or not? Is it, you know, is this thing going or not? Is this Right, like you there is multiple dimensions that people know about their data sets. They may want to like change visualization. So, I'm just thinking uh right, like because like you already showed this thing rotates. Like, people inherently want to do stuff with like if this thing rotates, I want to rotate it manually

I want to rotate it slowly. I want to rotate it counterclockwise. I want to now zoom in. And once I zoom in, I want to rotate it again maybe around different direction. I want to like take a look around things, right? So, like do you envision this? If this is like full interactive environment where I kind of kind of I'm exploring my graph, Mhm. right? I I I just think that humans will do operations in using this visualization they will want to reuse later. Like, zoom in on this big blob, turn it around several times. Like, maybe I want to script it

I'm just making things up as I go. Like, is this something people do? Like, do you observe users do this, and they ask you like, I now want to be able to do this kind of rotation or transformation, maybe I want to flip it. Like, is this something you envision kind of how this thing develops? >> We actually had a few requests to do all like all of this transformations possible. But let me answer your first question about the like LLMs doing things. >> Yes. I feel like this is exactly why people might need a tool like Cosmograph Mhm. is because machines they can mess up. Right

And you need to have an auditing tool Right. to make sure they did not mess up. Right. And at the same time, when you let's say you're training your model, and before you train your model, you need to see your data. You cannot do it blindly. Yes. >> So, this is where Cosmograph can be really helpful. And what we observe right now, people coming with very different problems

It really surprised me because when we first built it, we thought it's going to be like a tool for fighting fraud. Mhm. Like, financial transactions. This was the initial We love this use case. New project customers love it and use it a lot. >> But then a lot of people came from biotechnology. Mhm. And they started building all these protein networks and everything

And like people from Stanford, Harvard they like starting writing and asking things and saying that like they cannot imagine their work without Cosmograph anymore. And I was like, wow, something we didn't expect at all. So, um we do realize that different users have different problems, and they might want to have different user experience for that. Mhm. We are only in the beginning of of this Cosmograph journey. So, we don't know what we're going to build next. Mhm. Right now, you can open your graph, you can zoom in, zoom out, search for things, Yeah

cross filters, there is a timeline. So, you can easily identify anomalies, let's say find a needle in a haystack, like do this kind of exploration. Yeah. And like maybe like extract a subset of your data and do something with it later with a different tool. Mhm. Or you can also share your graph with somebody else because this is one of the problems Mhm. we saw that many people have. They work on something, and they want to show it to others

And there are no tools for that. Right. >> So, with Cosmograph, you can just go, upload your data, share. Yeah. Just works. Very cool. So, thank you for these insights. I know you have a talk to to give

So, appreciate it, and hopefully we'll see and follow your progress in visualization, get to play with this. It's great to have a new kid in town, you know, new visualization tool to show graphs in. Sure. Thanks you so much. Thank you. All right. Yeah. Spasiba

Spasiba shoya. Tak, shas ya zakrivayu lavku. Prekrasna.