SBTB 2023: Alexy Khrabrov, Open-Source AI: Community is the Way.

SBTB 2023: Alexy Khrabrov, Open-Source AI: Community is the Way.

Recording: SBTB 2023: Alexy Khrabrov, Open-Source AI: Community is the Way.

[Music] get my own conference I usually just organize it but I you know there is this first time for everything um and that kind of Suits my recent career pavot where I finally joined the speaker circuit myself uh so uh I'm a open source science director at IBM research you've seen my colleagues Anthony Dean and Carl just before me speak about different aspects of uh open source for science that we do at IBM research I've also been recently elected as the first chair of the new J F Commons which is uh a part of vinks foundation technically it's a committee of lfi and data Foundation uh but it's probably big enough so we called it the commons and I think it's poised to grow um and obviously this is the Linux Foundation answer to the question of what AI should be Linux Foundation is firmly in the camp that it should be open source I also have this icon here llm Avalanche who knows what it is who has been at llm Avalanche a few folks so LM Avalanche is uh started as a Meetup that we put together in June uh week before data brick Summit a lot of folks were in town uh and uh it just you know it's just thought like you know I also run and started B area AI which is the oldest AI metup in B area and so we thought okay let's kind of do that before and what happened is in um in two weeks we had uh as you can see on the right we have you know thousands of people in downtown San Francisco Isco we had all the leaders of L Foundation we had multiple you know senior Executives from all the area companies attending this uh because we managed to do we call it the shortest the deepest uh technical conference on LM at the time it was four hours 40 speakers three panels three tracks and a thousand attendees so as you can see this is a lot of activities we do with open source science uh with my colleague Tim bonman who is also here in the audience uh and this is you know a social technical activity as you can see uh we attend different meups and generally connect open source developers with Scientists to accelerate science solve hard problems of humanity uh and generally Advance you know the field of uh open source AI so that's another example at open source science we have an initi initiative called map of Science and so what we want to do we want to take all of Open Source ever used for Science and we want to find it we want to map it to actual areas of science where it is actually cited and advancing science so we want to find all the scientific papers which side software in general we want to identify which of this software is open source so through our collaboration in open source science interest group on map of science we were I think one of the motivations for ch work initiative to run a three-day Workshop endend of October with 60 people from around the world leading universities leading labs and so here is my colleague uh Peter star who is the lead of deep search about which you just heard from Carl uh he used it to very quickly identify GitHub links in archive.org but we have enormous amount of research coming out of this right and a lot of this stuff is related to J fi usage and science so um so kind of you know if you step back a little bit right uh why do we need community and obviously we're here at a community event and a lot of folks actually are very happy that you know after a long time we came back we have a tradition of community so can anybody say like why do you guys come here what do you enjoy about a conference like this anybody diversity diversity that's a good one meeting people right learning learning new things right yeah so a lot of this is all correct right but you know I think it's another like we all could have stayed home and just read a bunch of blogs right A lot of people do that and we can we did this before we'll do it after still we gather together in the room and we you know meet a whole bunch of new and old friends at the same time so when you know before I came to America right it was 30 years ago uh I found this book called America is a civilization by the author Marx larer so who heard about this book who read it so this book is actually surprisingly not popular anymore but it's been a bestseller for about 30 years I think it was published in 1955 and it's been a bestseller so highly recommend it so basically you know I learned everything I need to know about America from that book when I came here I hit the ground running there was no surprises for me and it's still keeping kind of keep keeps validating So Max L basically kind of distilled the interesting features about American culture and I think it applies to software culture very much so he said that you know in in American culture the ultimate truth is revealed in a public setting so if you if you watch a movie about court drama right like Oppenheimer an open in open Heimer right the truth kind of this the scenario I hope I'm not spoiling for anyone it's like there is a coure drama there's an interrogation it's a public setting there's a public record if you like enormous amount of movies have like the final scene happens in a church if it's a kind of a love movie right there is like a waiting so uh humans gather together and they establish truth collectively right and it's especially true about things which are not certain uh so obviously like Al is an extremely uncertain area everybody Miss we don't know how they really work the smartest people they are very willing right to take kind of leaps of imagination in an example I've seen at last new rips uh in New Orleans uh who's been at the last new rips in December so you know Jeff Hinton got his life Achievement Award there and uh right so like the the the the huge Hall was full I think there are like tens of thousands of people right like the the the room stretches into the Horizon and so everybody packs like the front rolls are completely packed you cannot find any seat everybody wants to see Jeff Hinton unfortunately Jeff Hinton comes over the internet and everybody is deflated of course like you didn't need to try to fight for the seat in the front row and uh Jeff Hinton comes in and he basically says guys I don't think the brain works like neural networks like computer networks I don't think back propagation actually is what we experience so I P hands forth a new framework from you know forward forward propagation and so he came kind of uh through this this uh proposal where basically we need a lot of positive and negative examples so and the idea is that that explains sleep because sleep is the phase where we train with negative examples so all the nightmares you experience is actually negative examples of something you shouldn't see right like the brain is trying to generate impossible situations so and what's what was most interesting to me at this point that it was like a collective kind of gasp right everybody was willing to suspend disbelief and all the work which we performed with back propagation was suddenly in question right regardless of 10 years of enormous development right multi-billion investment everybody for a minute given Jeff Hinton who is the Pioneer and the founder of the field going out and saying maybe this is all wrong right like everybody was willing to think maybe it is right because this is not math this is experimental science so it really struck me at this point right this is why people go to new rips this is why again in December tens of thousands of people will go to New Orleans because we establish the truth under a certainty in a collective setting and also by kind of looking at what other people are doing you've seen about 60 talks on this conference a lot of them are about applications of AI we kind of we don't know exactly where this is going but we see a bunch of smart people trying different things so you kind of see the envelope of future right and so this is I think the key value here right you cannot determine by sitting at home and reading 10 you know 10,000 blogs right you will not see immediately which of them are promising because a lot of them sound very reasonable maybe there are people you don't know but they're from legitimate companies and it's very hard unless you actually see them speak if you see people speak live you quickly form your own judgment is this person trustworthy do they really do Advanced work right or they just like repeating some Buzz Words which is nobody this conference does so this is really good uh so this is kind of what I think very important um proposition uh I want to make first of all we can establish truth only as a collective group of humans under uncertainty second there are a bunch of claims in AI about performance about trust transparency ethical responsible Ai and so forth right and all these claims are made by various groups usually they're made by government committees companies and so forth and if you look closely uh very often these are not legitimate bodies to make these judgments a company cannot say that their product is trustworthy because a company has a commercial interest right and uh a single company is not it can never be an Arbiter of trust performance ethical Ai and responsibility no matter how good it is so we need to establish a process by which we adjudicate the claims of Truth responsibility something being ethical or not because ethical AI is determined by a set of questions and tests and very often adversarial tests right which can only be solved in the collective setting so we need a community we need to define a community which can adjudicate the claims of trust performance and ethical eyi um so now I want to kind of uh do a little bit of a review of the some things which we not talking about uh often right I've not seen the kind of next year of llm so we've see a lot of people building charts right you can chart with different things you can chart with your documents you can like the the whole rag area you can chart with your company documents which is fun uh one thing I've never seen yet and I want to ask anybody did you see a talk about llms where somebody presented a working production solution deployed where they replace a substantial part of their business with an llm such as a call center or a customer service ticketing Department did you see did anybody see a talk like this probably not if you've seen a talk like this let me know right but uh I think we have we basically see a big uh a really big gap here right so we're kind of the early adopters of Technology we play with things a lot of things are now exploratory a lot of lolm applications are basically our attempts to figure out what I'm going to do with them and a lot of players in this area are startups right so imagine a typical startup we have several present and they're great startups but if you want to replace a customer service in a bank right you need to come to a bank and basically say give me your Oracle database which keeps financial data give me your Salesforce CRM give me your logs from your call center and the probability that the bank will give it to a startup is very very low right it's probably not going to have immediately if at all so um if you're a company like open AI right again like you need to convince somebody to give you the most intimate data your your financial data your customer data and until the the Enterprise is assured nothing bad is going to happen they're not going to do it so we we have everybody in holding pattern right now we do not see actual Enterprise deployments and I think players like IBM and Samsung and big companies and Hitachi and so forth they're probably best positioned because they already have the trust of the customers right it takes really enormous investment in in the relationship with the customers to earn their trust we do not have that yet uh in the llm business uh and so we need to see this transition from exploratory to production and that will not happen I think in the current setup very easily because the trust is the elephant in the room you will not gain the trust of businesses to deploy it uh and I kind of posit one of the Reas reasons for this what's actually going to happen there is a huge change waiting us right because if you look at who who who heard the phrase digital transformation who engaged who performed a digital transformation some of you guys went into some company and said like we're going to do digital transformation on you right so it's kind of a little bit P it's kind of sounds like something from early 10,000 2000s so I propos the term digital transformation 2.0 which will actually be the true digital transformation digital transformation did not yet happen what we've seen was a very kind of nessing thing it was a it wasn't something in its infancy because digital transformation was actually something about document flow so I was the chief scientist at the company called Nitra which was a public company in Australia until it was acquired Again by a private group so my chief scientist uh handle actually corresponds to the actual last you know job I had at that company so Nitro was you know and is a great provider of PDF digital transformation you can go and you can replace a bunch of manual things with a little bit automated things you can use word and PDF and so forth so that was a digital transformation you you have a contract flow you negotiate a contract you see the changes very quickly you use traditional LP kind of identify forums and Fields and so forth and leases and you can speed up that right so your worker your lowlevel clerk will be empowered right by this so but if you look at the automobile people thought that we're going to have a electric horses now we have electric clerks right they are kind of and with the lens the current proposals let's replace the current lowlevel clerk with an llm and maybe attached to a clerk so we'll now have a flying horse it's still a horse right the question really what's going to happen we do not need whole SS of this company business we do not need a bunch of middle managers because what middle managers do they take the strategy from the leadership of the company then translate it to the stable of clerks implementing this right and so currently if you look at the flow of information through a company there's a bunch of emails there's a bunch of slack like people are talking people spend enormous times in meetings they're basically forming like this human Network which is extremely suboptimal there are people who spend their whole lives in meetings like the human routers they collect certain inputs and produce certain outputs and this is very important right because a company as a conservative company cannot kind of go randomly and do things you know at random so they so basically we have now this human Network and this whole human network is probably going to come undone so the whole layers of this midal management are going to be gun because now that we have the data flowing through the company very differently we don't need a bunch of human rouers so the strategy can be translated into execution in a different form right and so so what I think is going to happen it's very important to keep kind of track of this that you know we don't just think of automating low level human work right we do not come up with flying horses we have some enormous beasts coming up the pike right which will be transforming the actual structure of these companies so digital transformation 2.0 will be very interesting and there is a very important uh implication for AI so if this is going to be done through AI imagine that your company has going transformed by this right the next stage of uh human in the loop where humans will be senior managers 10x Engineers right there is a vastly different response to uh copilot for instance from Engineers so I think the mediocre Engineers are threatened by copilot the TX Engineers or like so-called 10x Engineers it was very interesting to observe their reaction at first they were very skeptical of this and then they tried it they basically said this is fine I now can complete my project 10 times faster right I can basically let it do all the boiler plate because they know what the boiler plate is versus the meaningful piece right and a junior engineer might not know this so I think will be vly different response right so low level clerks will probably be uh much higher level individual contributors with human and the loop capabilities and the middle management which as human routers probably will be gone completely I think right if it's kind of this kind of primitive old school middle management and the senior managers will have much more freedom in trying and executing strategy uh but we need to parency we need to have these systems clearly auditable right because there is a huge question of fairness there is a huge questions from across the company how these things Drive information through the company even now it's a very hard to to find this the whole there a whole bunch of startups which look at soal dark data the data trapped inside corporations and you cannot know like you need Discovery there was a whole bunch of startups a lot of them failed trying to basically data mine you know company email slack and so forth uh I think we'll have to see that these systems are auditable and transparent even if they're close to thats side world if you're inside the company they there will need to be much more transparency in how the systems will work so my second claim is that digital transformation 2.0 will drive the openness of these models inside the company even if it's not open to the outside world uh the third thing we are not talking about in Silicon Valley a lot of industrial AI and uh I've been fortunate recently to attend this conference called K first world uh which is a gathering of uh basically big companies employing AI in in GI factories machinery and so forth and so companies like Bosch and uh John Deer and kitachi and Panasonic rightly they have Samsung they have a lot of devices and now they're thinking hard how do we Empower these devices with AI and the way they're thinking about it is very different from what again we usually see in Silicon well it's all software based right if you if you want to for instance build a smart building so borch is in the business of you know digital twins now right so they they Kings of iot so they build a building and also they provide Power Tools let's say you you want to use a power tool like a power drill to drill into a smart building uh so what do you do to avoid heating Antarctic line you have the the B building information model and the power tool actually knows it connects to it right and it knows not to drill it will stop if you try to hit the power line it will actually stop so this is what we can do right now right in the in the future there will be enormous amount of this intelligence predictive maintenance has been the buzz word already five years ago uh and you know all of these tools will need to have certain intelligence but the kind of intelligence they will have is vastly different from my dialogue of chat GPT because a power drill does not need to know the meaning of life it need just needs to drill right and like it needs to do a very focused set of tasks that a tool does or let's say an industrial robot on a conveyor assembly belt you know of a Toyota factory it needs to know a lot of different things right like the cars can come in assembl them properly right the other humans or robots can make mistakes so you need to do a lot of different things autonomously but you have very specific context you have very specific spefic goal so uh a lot of folks in this area they do not actually think that we're going to end up with large language models they think we'll end with something called ssas small specialized agents and the Agents themselves will be EMP powered to make decisions in a very specific context and so what they really want they want ownership they want to own this little model which is installed of their tool they want it to be small because the tools are small and cheap and like you cannot put you know iPhone quality uh bionic chip into them right then need to be Compact and they need to do one thing and they need to do it well they need to actually be efficient right they need to do you know usually there is no like gener like very philosophical questions about these tools they usually do not need to be an example of responsib AI they need to drill into a building right but they need to do it very well so so there is this important drive and again this augures for open source AI right because all these manufacturers don't want to pay open Ai and they don't have any use for the you know open AI models they need to build their own specific models for their specific tools and they need to deploy them right but they're not a software business right they produce tools so they probably like Automotive Linux in Linux Foundation they want to gather together they need to develop standards tools tool chains to produce these models and deploy them so I think this is another very important Vector rarely talked here because we do not have large industrial base but this is happening around the world which is important to keep in mind and recently I think it's kind of percolating up our community so cland delang the co of hen pH he basically says the prediction is most companies will realize smaller models are essential so he generally echoed this as of last month um so now kind of the pieces of like it's all comes to kind of the components right what do we need to pay attention to so geni gener generally deals with data models applications right and also this piece of community validation I mentioned it's not explicitly articulated and I propose that we always keep it explicit right you cannot talk about technology of AI without Community validation Community validation should be a key part because ultimately the decisions of a will always be subject to human validation in real time or postf factum so um and you know there is a kind of set of tasks we need to do as a community it's emerging for training we need to know where the data is coming from I think it's there were several talks here which touched on this it's often over look but we need to know exactly where the data is coming from right because the data defines the biases and risks uh I think there was a lot of discussion about models uh Linux Foundation uh is finalizing um a badging scheme where we'll give badges bronze silver and gold to different levels of openness which is parameter weights that and training scripts and then ultimately an architecture and then everything plus the data right which is the gold so we'll hope we'll push people towards openness by clearly labeling their models right and making it very visible that they hide training scripts or they do not deploy like do not share the training data right it will be very clear they cannot make claims of openness until everything is open um and in September in Bilbo there was an open source Summit where we established this commment so there are four uh work streams there are basically data uh models Frameworks and education right and it's very traditional Lings Foundation group so I invite everybody to collaborate with us generally the structure is you know you need to have a member of f foundation for AI and data in the commment but anybody can invite a guest so if you you have strong opinions about open source a and want to contribute to our work streams please reach out to me I can invite you know you as a guest and we we meet monthly on Zoom as everybody else and we advance this work streams so please reach out if you want to participate in this and there is a conference now for AI development called AI def it's in San Jose December 12th and 13 so that's a very new conference we SP in the word and a lot of this discussion will continue uh at that II Dev conference uh and finally I think you know gen AI comment is one thing but we have many others we have uh ml commment which is the benchmarking nonprofit which keeps performance benchmarks ml perv who heard about ml Commons here yeah that's a very interesting group right the the cloud providers needed a common Benchmark so they can basically again adjudicate claims of performance in an objective fashion so they came up with this Benchmark ml perf and then the the ml commment was formed to host it and I think that's the way we're going to do trust and ethical AI benchmarks first of all we need to formulate ethical problems as code to reason about them without kind of word against word and then we need to put them in a place where everybody can trust the benchmarks are run in a trusted fashion right so I think malc is a very good place to put benchmarks in then we know we obviously have a lot of foundation models in scientific areas currently some of them are put in huging face I think there will be a lot of Niche and Industry specific bodies thinking how Foundation moles for climate or materials should be used and built and so forth we have partnership for AI which is doing a lot of um uh kind of political lobbying and so forth and policy thinking there is this new frontier model for which is probably the closed Source AI answer uh to to open source which we're going to see how it works out and all the major government bodies have oecd and you know World Forum they have working groups and I think some people even go to the Apex Summit uh this week in San Francisco there is a discussion there as well um yeah so that's basically all I have and I'm thinking uh I basically have three minutes here right and yeah so I can take like a question or two here and then I'll be available at the Q&A all Island thank [Applause] you any questions not okay thank you here's a question sorry raise my hand until after you looked away so uh I'd like to hear a little bit more about your opinions on open source and safety and how those are compatible I personally safety is not my big issue with with AI but since you mentioned it I'm really curious how you tackle it right so I think everybody right raised this concern that the models hallucinate and you know they recommend people do bad things or whatever it's a very general question right so I think as as far as we know we cannot solve this uh problem at the root for larish language models for foundation models and some people live in posit that it will never be possible to avoid hallucinations Hallucination is a feature of this architecture so some researchers go as far to say this is a dead end right it's impressive it's a parallel trick but in five years we're going to be done with this and something else will be the new AI right that's that's a legitimate argument uh other people say they follow this some small specialized model route and they say we don't need a thing right which starts make making up biographies or papers because we're going to scope it down very narrowly and tie it down to its specific task right and then if you so the same industrial folks who who do this they say if you know if startup kids in San Francisco think that they invented technology safety they should look back 100 years and look how industrial safety was done right like if you look at a plane plane has a normal amount of safety features engineered into it you know even the like the the old locomotives they have this dead machinist handle right if somebody Falls dead through a heart attack and releases the handle the thing will stop so the industrial safety goes back at least 100 years and there is a lot of knowledge in in various Industries right if you go to Kaiser they'll scan your bracelet every single time so they don't feed you you know their own medication because the most suffering through medical errors so so we need to look at all of this right like if you deploy something in in an industry in uh in the real world that area of the real world has probably safety measures already developed which you need to absorb you can just randomly deploy technology right so I think there is like there are different RS to do this but they go through specific examination of what is the context and safety measures instead of General kind of hand waving about safety right which is often kind of where it's going all right I think we're at time so I'll be at the Q andl and thank [Music] you