Chief Scientist: Slava Tikhonov, Head of AI at CODATA, Paris

Chief Scientist: Slava Tikhonov, Head of AI at CODATA, Paris

Recording: Chief Scientist: Slava Tikhonov, Head of AI at CODATA, Paris

Hello everybody. I'm Alexey Khrobostov, the founder of the Committee Research Center for Reliable AI at Northeastern University and head of community at Lake Sale. And here I am in Paris at the International Science Council and CODATA office. And with me we have Slava Tikhonov who's the head of AI CODATA. Welcome Slava. Thank you. >> Uh explain to us what the Council and CODATA are. Yeah, so we are non-profit organization and we're advising to political bodies like like governments, presidents, uh uh organizations like like World Health Organization, United Nations

We're advising on on policies on data and artificial intelligence and other stuff. And this is supported by the government of France. Yeah, that's right. So we are here also Yes. >> in France and yeah. But you operate globally, so you you operate all across Europe and and around the world. That's right. Yeah, so yeah, we are based in Paris and uh uh our global activities include America Latin America and Australia and other continents like like we have a lot of activities in in Africa as well

Mhm. And yeah, we are basically like like global player advising on global policies. Great. And before that you were researcher at the Royal Dutch Academy and you were one of the authors of Dataverse and Crossref and Semantic Crossref and CDIF. So we're going to discuss all of this. We uh see it as a semantic layer for AI. Uh is AI navigation as you you call it. That's right

So it's it's yeah, it's a bit more complicated because what we are doing now, so first we created navigation layer which is Crossref for machine learning. Mhm. Uh standard originally initiated by Google. And now we are building extension and this extension includes also cross domain interoperability framework, which is what we use for semantics. So, basically you can you can see Cross Sound as a navigation device in your car. So, if you want to to drive somewhere, you have to select a city and type the street name and after building number. And we do exactly the same for AI. So, AI knows where to find resources

Mhm. And as soon as you arrive, uh you have to ask what is available at home. So, this is about relationships and uh cross domain interoperability framework CDIF allows to describe all all these relationships and describe as uh defined as variables. So, very detailed variable layer including units of measure and uh uh properties, hierarchies, and all related information. And on top of that, we also have actionable policies. So, we use we are using all standard called ODRL, Open Digital Rights Language. Mhm. And basically we can we can put all stuff together with a decentralized identifiers, DIDs

And they also have uh verified credentials. So, we can see from whom uh is coming every kind of beautiful information. And also we have provenance, we have uh basically logs and all this information is completely transparent. So, we can we can build um responsible AI applications. And they can basically be used for any kind of research activities and they can support any kind of uh um well, any kind of activity related to AI, basically. Well, this is great. So, this this a lot of information. So, let me unpack it a little bit and kind of zoom in

So, first of all, we are talking about data quality. And a lot of folks who do agentic AI, uh you know, I'm in Bay Area and we have agentic frameworks, you know, five meetups a day. Most of them talking about agents interoperability, they talk about MCP and tool calling, so they talk about the functional side of agents. Where do agents go? What do they do? And it's taken for granted that the data is there. It's an elephant in the room because if the data is not good, if the data is of questionable provenance, and agents just go and find it without checking, uh they will produce garbage. So, uh we focus basically here on the data quality. So, all of the uh croissant, semantic croissant, uh you know, DID and DRL, all these technologies come together to first of all uh annotate data sets, so we know exactly what is the meaning of each column. And CDIF allows you to say if it's a number, what is it? Is it a Kelvin? Is it Celsius? Who produced the number, right? Yeah

Did it randomly appear? Can I trust this number? You need to know which lab produced the number. You can just take numbers for granted. Uh if it's financial data, they should come from trusted exchange if you're going to act on this data. So, so we're basically talking about data quality, and I think Slava is one of the few people who actually consistently allows kind of to build this whole navigation layer for data, right? Because once you have the data, then, you know, then agents can find the right data, but not Nobody really focuses on making sure the data is correct. So, this is huge. And both DID and DRL are W3C standards, right? So, they exist for many years. DID is implemented uh by Microsoft Azure. You can just go and, you know, and use DID primitives as Hyperledger

You can get DIDs on any Hyperledger compatible uh network. So, uh so if this is established under this is not something new, this is basically and uh semantic croissant has been developed by Slava and other folks. So, this is, you know, set of technologies which which is trusted, which is working, right? So, in in all of this exists. This is not a white paper. Right. >> [laughter] >> No, no, no. That's that's really true. So, uh what we do now, uh you know, everybody is moving uh forward with Agent AI, and we are basically aligned trying to align our infrastructure with modern direction and what we see now happening in Silicon Valley

So, people are building applications on cloud on Codex. So, they're using different tools, different uh services, different applications. And at the same time, we don't see responsible AI. So, basically, it's not reproducible. Mhm. >> you know what you're building is based on a completely wrong code or wrong data. And we're trying to bring this layer of trust. We're building that by basically communicating to people responsible for standards

And they're working together to define to make definition for for every variable. Mhm. So, to to to make actually this definition available for every model. So, not only for Google, for Anthropic, for So, it should should be like layer of trust, which is distributed and you can connect, you can get the definition of variables and this is how you can connect to to like global infrastructure and reduce randomness. Right. Right. So, uh basically, this technology allows to uh to avoid non-determinism. And so, like let's zoom in on this a bit, right? So, So, one of the things which I was really impressed by that you do is you basically wrap, you know, a prompt in in a DID because a DID is an ID which can have an XML payload

And so, you can attach prompts and skills. And if you want to, you can encrypt them and then you can let middleware like a llama decrypt them, right? So, it's easy to build the infrastructure where and because a DID is a global URI, then you know, middleware can actually find it, right? And if there's a ODRL and public infrastructure, it can actually decrypt it as needed and execute it. So, that's what you've done with a llama. And then you should the highest speed up because you basically you're imposing determinism by doing this. Can you explain a little bit how that happens? >> Yeah, so what we're doing we're trying to kind of freeze vectors, they're called tensors, inside of LLM models and when reducing ground truth. So basically you have a pair question answer and it's getting digital signature and it's been also signed digitally. So you have owner and you have person, you have human who is behind. Right

So you know who is responsible, right? >> Right. And as soon as you have this key and the value pair it will become fully deterministic. So as soon as you ask this question and it will always be the same the same answer. And basically that means that you don't have to to do compute. Like for example, if you're using local AI local inference engine like Llama VLM. So it runs with speed, let's say 50 tokens. Mhm. Which is because they have to do compute

So you're asking a question, there's a very complicated process behind and it's basically slow, right? And what we do we're kind of pre-caching action, but this is not real cash. So we have the whole vector. And it's also it's it's not cash in traditional way. So we can also define threshold. So like if question, let's say 95% is already been answered. Mhm. So we we can also take like similarity measure. Mhm

So you don't have to to compute again, so you can just pick up all these precomputed vectors already and use these to answer question. And only compute like 5% Mhm. what we don't know. So it's also very significant speed up the process. So instead of using like time waiting for for let's say 10 seconds for answer, it will be probably one. Right. Because because it's already got kind of pre-cached. Right

And what what what is really important? So, as soon as it's been computed and human agrees to put a signature, to put its name So, you can also distribute this knowledge. So, you you can just export this the whole tensor. It's around well, in general, 50 kilobytes file. And you can package and you can transmit to another inference engine. You can just run it, ask the same question, and it will be the same speed and the same answer. Mhm. So, you you're basically turning F and you're converting from non-deterministic answering machine to fully deterministic where F is predefined. It's it's coming from source that you can trust

And this is really important. And I mean, for cross-domain interoperability framework where we're supposed to deliver some solution that will allow to convert like to reuse data in different domains. Let's say you're doing something in physics and you want to just to repackage the same variable in different standard and provide to another domain so they can also connect and they can use that. It should be possible because this is kind of universal distribution mechanism. Mhm. And for AI, I think it's it's this is what exactly what we need right now because LLMs are getting crazy with all these uh uh randomness and we can't just rely because we can't reproduce results. >> Right. It's Reproducibility is the hallmark of science

Non-reproducible things are not science. And Yeah. You're basically bringing this back into the domain of science, which is incidentally the mission of Coda. And right, which is great, but it's also required by any business. So, I I want to emphasize a very important point here. So, all this is happening on the CPU. Right? The the cached values are being obtained from the middleware, right? Yeah. And so, you you you you demonstrate a thousand a million tokens per second because you're effectively in in you're avoiding uh you know, GPU whenever possible

So, that means enormous savings on compute. Yes. It means moving back to CPU infrastructure and and and right for reliable, kind of, you know, reproducible data. So, anything we should need to be reproduced will be reproduced from CPU. It It was It was before computed on GPU once, which is very, very important given that shortage of compute and waste of energy and money. That's completely correct. And what what is actually happening right now? So, we started to to to produce CD variables and we do it once. Mhm

And we're getting registry of variables and after we can just distribute them without running computer again. Right. So, like like you said, we're moving world from GPU computations, which is very expensive. >> Right. >> Requires a lot of resources and it's it's long process. Right. >> We're We're moving to completely deterministic model where everybody can can just download updates from us. Mhm

Run on own computer. Doesn't matter. It can be a really primitive computer. It It's actually because it's kind of precomputed, you can also use really small models. Right. Which you can you can run on your mobile phone, right? You don't need like powerful computers to to do that anymore. So, basically, AI is becoming really democratic and everybody, I mean, every researcher at every person on the earth can can can just install it and run, right? Right. And the same goes for agents

As soon as we have this ground truth, which is distributed and, you know, we we also have complete provenance information. We know our region. We know who's responsible for that. Mhm. We we can just connect plug in all these agents and they can do reliable job. And we can also establish a reliable baselines, which is extremely important. Everybody's talking about baselines, how we are going to test what what agents are producing, if they're working together. Right

>> So, we can also, like like put the ID the centralized identifiers on every agent produce verified credentials and we can see what every agent is basically contributing. Right. And if something goes wrong, you can go to a logbook and you can find you can actually found who from agents uh not not getting enough data or it's getting wrong information and from which source. Right. Which is completely different >> the bad agents can be fired automatically if they're not producing good answers. >> I really like the idea also not not well, not not technically not fire them but but basically they should be gone with suspension. So >> That's right. We can kind of ask Archived

Archived. Archived. So we can also ask AI like re-implement. So this agent has already collected a lot of knowledge. So now we can package knowledge. We can also see we can check what is valid and what what is not valid anymore. Right. >> repackage and run new agent basically like having knowledge from previous one

>> is Europe again not just fire agents. We have to upskill them and give them a new jobs so they can disconnect they can actually >> Yeah. And >> [laughter] >> Right. So I want to focus also I think this is super interesting. So you talk about drift, right? So when you say basically not determinism, it also means that if something does change, then you can measure the difference. You can detect it. You call it basically a drift. And and I think one way it can be implemented is European IDs not only the questions but also the answers

Yeah. So now you have a question and answer pair which is timestamped and versioned and we know exactly which version of the agent, which version of the LLM, which version of the whole infrastructure >> Which version of human because Which version of human? I mean hopefully I mean we cannot measure but we know the human identity of the user, right? So so so now we have basically complete kind of traceable provenance of the whole tool chain that produced a Q&A. So talk a little bit about how do you detect drift? What if the LLM has changed, Right? And how do you see what has changed? Yeah, so drift is a very well-known problem in knowledge engineering and knowledge organization. So, knowledge is completely So, it's not static. It's It's moving. It's moving in time and space. So, let's say if you want to understand how people I used to think about like like nuclear physics 100 years ago and you want to to to get opinion of Albert Einstein. Mhm

So, you'll get that from the his book, right? So, this knowledge we we can just just capture. We we can package and we can put it also in LLM and we can digitally sign it. So, it knows that this is not up-to-date knowledge. It's It's like from 100 years old. Right. And by using that approach, we we can also put like like different times timestamps and find different books and we can see how knowledge was developed in time. Which is kind of unique because I don't think someone has created something like that before and now we have really unique opportunity basically to see all this drift how knowledge is developed in time and how we can reuse it, you know? Like like So, how how to track this delta? How So, let's like look technically. So, let's say let's just start with Newton, right? So, Newton, you know, published Principia Mathematica and put them, you know, you can scan it

It's a beautiful I have like the reproduction of the original edition in Latin. Beautiful beautiful, you know, volume. So, you you scan it, you put it in LLM. Now you got, you know, Einstein. So, here's a different views of physics. So, how would you get both answers about something and and compare? So, this is where agents actually doing job. So, basically we're asking different agents. So, one let's say one agent is already responsible for named entity recognition

So, it's basically trying to to understand where person is being mentioned what kind of dates available and what kind of organizations and uh you know, all these all these entities, right? And basically, it's kind of foundation of knowledge graph. So, that's something we can we can already capture. Right. And we can transmit this information to another agent. Right. >> Which is kind of linking all this information to available knowledge. Right. And knowledge should be installed in knowledge base

So, it's it's not uh this is uh something different from people did uh with LLMs. >> Mhm. So, like standard approach, let's capture all knowledge. Uh Uh we'll put it together. We'll build new model. Right. Train LLMs. Uh train LLM, yes

Right. So, instead, uh what we can do we can just put uh identifier on every claim. Right. >> And uh it will become traceable. So, you will see from which source it's coming. Right. >> And who actually created this knowledge and uh what what kind of uh attitude it has. So, all this kind of information is traceable, right? >> So, we have a knowledge graph next to LLM

Yeah. And for reasoning and so, by the way, uh I must mention that one of my professors at University of Pennsylvania and originally from University of Edinburgh, and he returned there, Professor Mark Steedman, one of the world's top experts on knowledge graphs. He basically was advocating that architecture for many years, and he and his students published a paper proving that LLM is not capable of reasoning or in uh entailment. Right? So, so a knowledge graph is something that you should be doing. So, you're basically doing the same architecture. So, you're extracting knowledge from LLM. You and and you stamp the claims, you know, the questions, right? So, now you know that LLM one believes. Yeah

And and you put it in the knowledge graph, right? So, >> That's right. Yeah. So, you have like a concept in physics, right? Like, what is force? And you put it like LLM, you know, Gemma 4 believes that Newton describes force as this, and you put it in the knowledge graph, right? >> Yeah. And the force and the reference will be to Newton. And now you question something about force in Einstein. Are you creating different node or you comparing the first node with that? Like Talk me a little bit through how do you see that drift, right? Like how do we know what Einstein said about force which is not changing there. So so this is exactly where archive is very useful because basically if you have all this knowledge in knowledge graph it's it's archive. Yes

>> So digital archive. Yes. And instead of like packaging all stuff in a model you can also query it, right? Yes. Using modern languages like Sparkle and we have another agent and it knows how to operate Sparkle. It was trained on that. Right. So as soon as >> Once it's already in the knowledge graph you can walk the cipher or whatever you want. Yeah

Yeah so so basically you're asking question it it does this analysis. It recognizing topic it it's doing an entity recognition and it's querying graph to see what is what is available. Yes. Of course like like part of information uh um it could be kind of contradiction also you know it's it's basically extracting all facts Mhm. and this is where we we have to rely on on LM models and on knowledge which which is already been been packaged inside of LM because it should know how to distinguish all these relevant facts from irrelevant facts. Uh but at the same time you know if you have new knowledge coming in you can also get it from knowledge graph like to confirm some some claims to find evidence. Mhm. So this is much more powerful model because it's very very expensive to train new model

Yes. So instead of like like packaging everything in in one big file you're basically connecting to distributed system and this distributed system is live. It's like every second new knowledge is being like like captured it uh processed and stored and it's available. Mhm. As soon as you are asking question like like what is what is kind of new information you you have right now, right? >> And uh so um you know we we can also think like about knowledge is moving in time and space. So, like in different countries the same fact can can be understood differently. Right. So, now you have perfect opportunity just to ask how it's been understood in the US

Right. And in China and Europe. Right. Like and you can see from different sides of the same. >> Right. And if you'll ask LM right now, like go to ChatGPT probably probably you will get on only approximation, right? Right. I mean actually it's funny because if you ask, you know, uh models from different countries about political sensitive things in this country, they still give you political filtered answers. Yeah

So, some of the answers will be actually incorrect, right? Or or massaged uh according to what's what's going on. But I think it just points to bigger problem that LLM is an average answer as of today, right? So, it's LLM is actually not versioned. Uh the current architecture of an LLM doesn't let you say, "Okay, what did you answer yesterday? What would you have answered to me Yeah. uh before the date X when some event happened?" Mhm. LLM just is not made currently at all, right? And knowledge graphs, you know, you can I mean it's interesting that knowledge graphs are also inherently not versioned. It's very difficult. So, people have to manually version them. So, if you want to send, you know, implement temporal change in a knowledge graph, >> Mhm

you have to think of this yourself. You have to attach timestamps to every update, create basically like lists of values Right. >> But but you can do this, right? Like this is this is a and also it hints at the fact that we need the semantic layer uh separate, but we need it for multiple reasons. We need it for caching and performance. We also need it for the archival purposes, Yeah. for detecting drift, right? So, Query Graph kind of is this, right? Like it's a set of technologies uh but so, by the way, everything we described is on the website querygraph.ai and Slava's posts uh are there and some kind of general information, links to the open source. We because all of this is actually built. Right? This is not a white paper

Built in open, yes. Built and in capital And ready to use. in capitalism. So, and then maybe I'll just wrap up with So, we are here right after the Go See in Paris conference. Go See was the global open source AI event basically put together by multiple companies around the world held in Station F, the iconic AI capital of Europe in Paris. And so, we just seen presentations from some of the leading companies and model makers. And we talked about semantic layer to many of them. There is very positive response

Maybe we can talk a little bit about folks who do, you know, open compute and Beijing Academy of Artificial Intelligence and what is the synergy between open compute and kind of semantic layer? Yeah, I was I was very impressed by a level of presentations at Go See and it's unbelievable and I'm also quite impressed what China is doing because it's supporting open source, open knowledge and basically what I learned now there is a new project called FAI OS and it's basically they're building an operating system where all knowledge is free and kind of distributed. Yes. So, from one side they have support of different chips. So, for example, it's not only NVIDIA, you can also run your your software your models on Chinese chips, on Korean chips, like basically they have I think 30 different chips already supported. That also means that you can connect your own laptop. Mhm. So, it doesn't matter if it's Dell or if it's um someone else. Mhm

So, HP, um Apple, so they don't care. So, so basically you you can just run the whole operating system and uh you can get access to shared resources. So, you can also kind of create computational resource which is enormous and you can do it for free basically because all people already have these laptops and it's kind of, you know, creating shared memory and shared storage and you can just use it to do like serious stuff and you don't you don't even have to to fund the creation of new data center which is like big deal right now you know and everybody's running out of out of the compute so now let's say if some research institution wants to do like serious compute so they can combine they can install this software they can combine all the sources together instead of one institution and do that stuff which is unbelievable you know. >> Yes yes so it's open compute for open science and you know open source AI basically and I think the semantic layer we're talking about can be extremely useful because people should be able to find the data sets they need and they should be trusting and because it's led by scientists obviously it will be very useful for scientists around the world to to be to and scientists are usually the ones resource constrained right they cannot pay for huge data centers so I think that's that's really good spirit so that you know community. Yeah I would say well what is also very important and we also contributing to this development is support of multiple languages so as soon as you have AI which you can trust so you can also get translation of the same terms and same vocabularies in different languages so you know that temperature is defined like that in Chinese in French and other languages and you can reuse that so it's basically always the same definition and it knows what kind of units of measure it can control that if something goes wrong it can understand that temperature is measured from this limit to this minimum to max and it can immediately like this policy like you can define also digital policy it can immediately like alarm that something is wrong and as soon as you have this enormous infrastructure like open compute deployed everywhere in the world you can just send signals and this is how I think like like world will be moving in the future is global collaboration. Right. Uh okay, and so maybe finally I just want to kind of propose that this whole setup can be uh put inside of a company. Right, so if you have a corporation, you have different data sources, different departments

You need, you know, first of all to make sure that all the data inside the company is in trusted format. It's It's basically explorable for the agents which are running business inside this company. That the computation is efficient. So everything we described can be basically an enterprise data platform. For AI, right? And so it is you know, the AI navigation will happen inside of this company. And and you know, ODRL level will be basically used to precisely define which employee, which department have access to which data. With the data in this case encapsulated as the whole business knowledge of a company, right? Like these are the problems and skills which will never be shared outside because they basically express what the company is doing, how it's making money, how it's keeping secrets, how it's doing innovation, right? So So if it's done in business context, right? You You You You need to basically have very tight control, right? Because you you talk about value. And here you can help impose it on agents automatically

Yeah. Uh the IDs let you trace. So if something goes wrong, right? Uh performance drops, uh profits drop, something happens, breach of you know, privacy, PII. You can always track which agent access what, which prompts was used, which answer was received. And the drift which talked about is very important because it can help business understand if the performance is changing, why, you know, my uh e-commerce model is not performing as before because, you know, something has changed in the underlying LLM has been used. And all of this can be used with the open source LLM run on premise. So all of this can run inside of data center, right? And so if you suddenly let's say try even without internet connection without internet so it may maybe factor automation, right? And and so if something went wrong because for instance a new open source model was used.