SBTB 2023: PANEL - Data, Code, and Life with LLMs.

SBTB 2023: PANEL - Data, Code, and Life with LLMs.

Recording: SBTB 2023: PANEL - Data, Code, and Life with LLMs.

who's been at the closing panel since some previous scale by the Bas so we have a debate right we we we started the debate panel and it was really working well eliminating the issues and so we follow the Oxford debate format in a kind of a little bastardized fashion which makes it better and easier so here is the the the way we're going to do it uh we're going to debate a proposition that open source AI is the future and if it's too General right sometimes it needs to be refined so you guys are welcome to address either the very general statement that open source I will win and we have two teams and we randomly divided folks into Pro Clos source and Pro open source it's for sports it doesn't mean people should believe this they can you you guys are welcome to reveal your actual preference in the end but for now please stick to the to the team so uh if you want a refined or more detailed uh motion we can form like this that in five years new billion dollar companies will be based on open source AI right and I'm saying new because established Legacy companies have their own histories and you can count open the ey as an established probably billion dollar Company by then right but new billion dollar companies in 5 years will be based on open sources this is the refined preposition or you can generally argue that open source CI will win in some fashion which you can Define so the uh we have one hour so we will be very efficient uh first uh you know each team uh will uh introduce themselves so every person has three minutes right and you can use one minute for general information about yourself one minute for your key points of interest which does not have to be without open source AI or generally yeah at all right tell us like who you are what are you about and use one minute to stake your position according to your team say why closed Source AI is the future or contrary say why open source AI is the future so after we're done with this introductions uh basically each team has three minutes to refine their position right so we'll go basically left right left right so the team basically will state more arguments and you can then address the arguments which other teams raised and we'll basically do several iterations of this and uh then we'll open the floor to questions from the audience uh so the goal of this uh debate is to sway the audience so first we're going to take the vote which will be approximate write a note how many people believe that in five years new billion dollar companies will be based on open source a who believes that the future is open source Ai and new companies will will be based on open source AI okay we have a really small amount of hands here so and okay who believes that in five years new billion dollar com who be based on closed Source AI we have a little bit more and okay who believes that nothing will happen in five years like who didn't raise their hands like we need so we have the basically a a very good Jon kind of group like un undecided okay so we you guys have something to work with very good all right so now I think I'll start with open source folks who are next to me so maybe introduce yourselves from Pete and uh to to Miguel hi I'm Pete scok I'm an angel investor and a adviser to Ai and data companies um I started my career uh working on retail uh profit optimization uh found my way to the internet worked at AOL search uh and then ultimately to LinkedIn where uh I joined uh when it was about 300 employees um and was early on the data team building things like skills and endorsements using machine learning um and in the Years following LinkedIn I built a company called skip flag which used natural language to organize and understand your company's internal knowledge and information so kind of a precursor of a lot of things happening uh now with llms um but this was in the days of uh I guess word toac and you know some of the earlier uh uh uh Explorations in in this space uh open versus closed source so I'm also an adviser to Common crawl um which is an open web crawl that you know like the pile and many of these uh llms are trained on or I guess all of them essentially um all the major ones um and I'm a big believer that more eyes make all bug shallow whether it comes to human systems in society whether it comes to code whether it comes to to technologies that we build um and I think that uh our team is going to make some compelling Arguments for this being much better for your business uh and for society to support uh open source not just in your software but in your models and in the systems that you build um it it's it's the better path and it's really the path we're already on so I don't see a reason to change it thank you awesome and I'm Eric Peter um I currently sit on the product team at data bricks um part of our Lakehouse AI which is our ml platform and I lead our efforts uh one around how we make rag applications easier for our customers to build and then two how we uh make generative AI models easier for our customers to fine-tune and train from scratch so I've I've been in the the data AI space for a while uh but I started my career on the what I call the dark side uh management consulting um spent a number of years there um and then transition to Google where I did essentially on the cloud side we built recommendation systems for our customers so also in the word to V days um using Vector databases before everyone thought that Vector database was the next big thing um is it that's debatable that we should debate we should debate is Vector database a real space or that's what I want to talk about I had a pitch in my inbox right before this panel so they're still going it's it's definitely another Vector database is what we need um but yeah then uh founded my own mlops company and yeah landed at data bricks after that so I I think that you know on the the open source versus closed Source debate I'll just kind of play this side of it for right now and on open source right there's some very critical advantages just like in software right being able to kind of know what's under the hood um kind of having full control if you want it being able to take that code and bring it in house be able to run it um as well as kind of complete customization um over those models so seems like you know that coupled with the fact that you know AI is fundamentally going to change the world in probably many ways that we haven't even thought of yet here um in this room it only makes sense that those systems that fundamentally are going to make critical life decisions for all of us probably up to including life or death in some cases should be open and transparent to the rest of the world so I'm excited for our arguments here in a brief moment awesome yeah my name is Miguel bernardon I work for pulse AI um we're company that Aggregates all the API endpoints for large llms so kind of thinking about if you have open eii or maybe entropic or a few other ones um you have a single API uh to be able to query against multiple uh models definitely check us out um but about me uh been in the distribut system space for for over 10 years um primarily started off from centure uh when I first discovered her dup um got Cloud certified was able to help our uh distributed data scientists be able to migrate their ML workflows on our cloud back end and uh since then you know our infrastructure has grown um I've learned a lot worked for after that I worked for U mesosphere because I converted the cluster over to mesospheric Cluster and learned a lot about distributive systems then um since then um made a product from there helped us do do uh multicloud deployments um with the single unified API uh our Command um and so I've been in the space for quite some time for managing distributed systems at scale um and we did a lot of GPU for doing ml um so fast forward that uh I went to cryptocurrency company um it's a lot in that space um but since then um I've been a user of a large language bottles for for quite some time and I'm sure all of you as well um but um I'm at Pulse um my sense on open source uh I feel like with when it comes to this argument you know we're really big users of uh Linux that's open source um and it's power as a lot of the the platforms that we have today uh I think for companies that are going to leverage that I think you have to have access for an autonomy to your resources your data and I think that's very important especially since the power and the capabilities of llms are just in you know Realms that almost are un uncomprehensible I think the the reach the capabilities are Beyond I think where we are today and I and I very much feel that you know companies that are going to make conscious decisions about protecting their data making sure that it's enabling their use cases enabling their Niche their access um their value ad I think an open source um component to be able to have that in-house is a significant uh requirement I think um and and I think even our compan is moving towards that step as well to ensure that people have that autonomy so there there is a strong play on on the open source side and I think over time you know I think the industry is going to kind of make the decision based on the needs that the company has but that's uh The Stance and about me as well thank you there we have the team open source now with Team closed Source please start with Manas all right start with me uh we were just talking whether we should go this way or that um hi I'm monesy if you were at my previous talk hello again thanks for sticking around um I'm the founder and CEO of Vera um technical background did my PhD in computer science at MIT built what now has become experiment management so that PhD work LED on to MLF flow and other systems that are in the space have been building systems for Model Management for model serving for better part of I guess 10 years which someone called me OG and I'm like wow am I that old I guess the answer is yes um to that have done data science have done infrastructure building we at Vera provide a model operations and management platform um we started off with you know your regular models as they are now called or traditional models and now llms um I spoke a little bit about our work on the gener of AI um workbench earlier and yeah really enjoy building tools for data and AI so that's a bit about me on closed source and open source um at least for this topic for this debate uh I'm on the Clos source side there's a few things I think Clos Source has going for it um one is just it's very expensive to build any of these models if it's meta or open AI um or Amazon or Microsoft like those are the only people who are going to be able to realistically build it unless there's like a significant change in technology that happens it takes insane amounts of compute it takes insane amounts of data um to train these things and also you need the expertise if you've heard the news like open AI is trying to poach folks from Google for a million or 10 million I think it was 10 million it's like most companies don't have that kind of cash to spend on you know individual hires and so I think it's going to be really expensive um second the Clos Source models have made their interfaces and their user experience really really easy and so one of the prospects that we were talking to just yesterday was like I don't want to have to hire uh really expensive ml people or data science people because we are using open source models I'm going to trust that open AI or whatever vendor they're using um are going to do that for us and that we can trust that it has gone through the tests it has been built the right way they're dealing with copyright issues that might come up I'm going to shift the burden of responsibility to them and reduce my cost center essentially and I think that is a really compelling reason for going with closed Source um and then open source doesn't mean that the model is perfect it's like if you think about you drive a Toyota or a Ford or Tesla I don't know you don't know all the schematics all you care about is like it's working it gets me from here to there and it's safe I think models I think of them as very similar as long as they do the job they were intended to do that's good enough and most people are going to be able to make do with that so lots more to talk about but that's kind of you know the one with NP awesome is mine on yes hey guys my name is Ryan I'm the founder and CEO of cadea uh we're focused on building identity management and access control into things like Rag and agents um so if that's something you're dealing with we would love to talk to you um I started my career studying bioinformatics quickly jumped in as a really early employee at nanic where I cut my teeth on Enterprise distributed infrastructure and uh eventually moved into sales engineering which taught me a lot and then uh after nanic started the first and I think largest data science boot camp in the US was called cpan Academy started right here or not right here in San Francisco and uh ended up selling that company to Galvanize and then scaling it nationwide um and then after after Galvin eyes I was pretty burnt out so took what was supposed to be one year traveling the world but ended up buying a camper van in Santiago de Chile and the rest is history drove all the way down to the southern tip of Argentina and back again so three years and then most recently I was CTO at a data science consultancy called tribe AI um it's about 200 practitioners focused on building you know solutions that deliver business value to to Enterprises um and I would say my stance on open verse closed is really born of my experience um while I think they each open source and closed Source has their place I've actually been surprised in the number of companies that I've talked to usually with you know a security bent in that they've already fine-tuned a lot of models internally on their own data and they don't actually when pressed they don't actually care if it's open or closed Source they actually care more about their ability to control it so is it hosted on their infrastructure is it secure is it monitored all of these things and so they would you know if if anthropic and I've talked to companies who were talking to them are willing to allow their model to be hosted and fine-tuned on someone else's infrastructure I don't think the companies are really going to to care and so I think this really comes down to which product is going to deliver the most value at the cheapest cost and we've seen that you know with Microsoft and their massive investments in data center infrastructure there is this kind of Tipping Point of where you're able to capture a lot of gains um if you can aggregate all this demand together and so I think they both have their place but from my experience companies are really worried about data privacy they're worried about controlling these models they're still it's still new it's still new technology and so I would ask like if a company takes an open source model fine-tunes it on the proprietary data is it still open source especially if they don't give it back right and if they don't if they don't contribute back to a library or contribute back to the model and like where would we be in an open source Community if llama hadn't dropped from Facebook you know for example so we a lot more to talk about um but yeah I'll pass it off that's great uh and that is a sort of a Nuance subtle like point that I think is worth we'll go we'll get back to and let me make a few boring commercial ones so I'm James champ I'm a VC at a firm called Bloomberg beta I mostly invest in developer tools and ml infrastructure e things like twio or um weights and biases or chroma and stuff like that and um and so it's 2023 you're saying in 5 years so 2028 like what will the biggest companies be like and will those biggest companies be open source or closed source and I and I think you do your historical analogy you do your Wayback machine you say Okay um if I started like let's use the web example if I like I don't know if I did Ruby on Rails if I was 37 signals right would I rather be who made the most money on Ruby on Rails and I I kind of say it's like Twitter and zendesk then they basically sort of figured out ways to make money on it built these great businesses and made a bunch of money there's a good question James are you arguing in favor of Open Source now closed Source no no I'm saying they were actually interesting enough Ruby ons is open source no but they wrote all this code themselves they kept all of it for themselves right ultimately their true profit Center all kept from came from stuff they kept for themselves they piggybacked off of it right so that's one another example would be I don't know you think about Linux but by the transitive property open source then open source wins well no until I think so I think you're exactly right in this sense that there's a period where it makes a lot of sense to be open source and then there's a different period of commercialization where all the returns go to the people who end up figuring out ways to commercialize and for our Better or Worse end up being closed source and so there's a argument to be made that like in 2024 should you be default open Source or should you be default closed source and I think it's not a crazy idea that you should be default closed Source if you're trying to make the billion doll company for 2028 or 2029 thank you we have the original positions so now uh there will be rebuttal so team open source I think you have your work cut out for you right the extremely compellent arguments from close off so let's give you five minutes to dismantle an attack with their position and strengthen your open source position and any order I I I hate to I I I hate to do this but I feel like now we're we're getting out of technical arguments and we're in in the chat GPT language debate Arena I feel like you were trying to do something kind of sneaky there James in that you are saying using open source and building that billion dollar company is actually actually because you don't give away all the code inside your company that you built on top of these open source language models that that counts for team close source and it obviously every company there's a gray area of they're not all pure closed Source or all open source probably every company uses open source in some way right um so like I I don't think it's like I guess there was this original what is it stalman like open source is a virus like once it's out there than every maybe this is better for you to follow up on being at data bricks and dancing that dance yeah I guess like CU it's it's interesting you're you're essentially saying hey like if you want to be commercially successful you have to hold some things back from the open source project and I I think that like this is a little potato pot do that to be clear we literally s this is [Laughter] SASS and I think it gets back to like well what what defines actually open source right is is it do to be called open source do you have to have 100% of everything open or or are you able to have some parts of your product close Source I'm going to say no like nobody has that expectation I think unless they're uh the the guy who doesn't use email right like he or I think that's what happens right like you if you're so extremist like okay I don't even type emails uh that that's the kind of vibe like nobody takes that seriously right every company it's it's always a mixture and that's why you have these build versus buy debates so I think the really like pragmatic question here is those companies four years from now I like this framing like the the companies that everyone's going to be like ah I wish I invested in XYZ and they built this amazing AI um I think you could even look at the ones right now right because it's not just open AI and everybody else and everyone else's open source there are a number of other I would even argue like if you look at what open AI is built on at the core it's an open source paper called that you know attention is all that you need and so and where would we have been had Google not open source do you have access to open ai's weights what do you have access to their weights or their according to chat GPT is example of Open Source so actually I actually I'd like to cite uh chat GPT answer that it gave me earlier I asked it um quote why are open source llms better than closed source llms and chat GPT GPT 4 mind you open source llms like the one developed by open AI offer several advantages over closed Source llms I won't read the rest of the the responses but I'm I'm pretty sure this is language models were very as far as I know it's it's it's open our Lord and savior chat has spoken so it's funny cuz I have a thread on my phone that's the exact opposite what I have I have a thread on my phone of the exact opposite with Chach gbt so I don't know who you know so maybe the future of debates is exactly this we will both leave thinking that we won the debate because Chach we'll just have the agents argue for us and then it'll just tell us who won okay but but seriously speaking like if you were starting a company in 2024 like at its core would you try to build a model in which you are going to give the weights to everyone else or would you think about it differently I think the question is also are you building your own model right like if you're if you want to get to Market there's different kinds of companies if the goal of your company is to build better models that's a different kind of company but if you're trying to build a smarter Zen desk or you want to build the next ride sharing company or whatever it is the quickest path to success is going to be the best model which today can continues to be close source so then you go and build on the close source and so there's I think there's another angle on that which is like where we are in the Arc of commercialization of like some technology and there's like there's a slightly terrifying thing I don't know if you ever look at like Microsoft's announcement they're spending a bunch of money on these data centers right and if people don't actually use these data centers to run things we're kind of all screwed because it's actually very like it's hard to lay people off it's harder to get rid of like huge huge Capital expenditures and so there's almost a way in which we need strong use cases right we need actual adoption use open source models being trained on top of Microsoft's infrastructure but beyond Co I beyond co-pilots and there there is a way in which like if we don't focus on building applications and to be honest most applications really are like closed Source we don't focus on building interesting applications we're all going to look really really dumb in like 2029 yeah so I have a a quick question so in the companies that I talk to they make these models work by incorporating proprietary data typically in sensitive context so defense healthc care insurance Banking and so they have created an artifact that they fundamentally cannot share um those companies like if I'm building a company five years from now I like the open source models will get as good as 3.5 or four or better and then the fine-tuning infrastructure is going to be there there and so really all you're competing on is quality of data we've seen that smaller parameter model sizes with better training sets outperform larger models with worse data and so I think the argument is who is going to be better at acre that data Advantage is it closed Source companies or is it the open source Community you making a great argument for open SCE trust me this is not my actual position I'm I mean like if we if we step out like I think the the interesting thing here is like if you take away the model for a second actually the bigger challenge is exactly what I kind of just read on my phone like is the model actually giving me the correct answer right when we talk to our customers what we hear is yeah great I just need a model that works for my use case and I need to know is that thing giving me a correct answer and I need to have confidence that it's giving me that correct answer how do I get to that and I think that to me is like that's the Quint essential debate we should be having is how do we evaluate these models how do we determine if these models are giving me a correct answer and how do I get the confidence to put chat GPT or llama into my call center and it's not going to hallucinate and say that you know yeah you can have some free stuff and so from your point of view is that a point of like costs like because certainly like folks do end up building a bunch of infrastructure before and after that turned out to be too expensive right now like is this a question of like as the costs go down these things will be more available or do you feel like they're genuine breakthroughs that people have technical breakthroughs that people have to go through I think if I may like I especially with the release of uh llama CPP that was such an unique breakthrough where you had an open source model a model that was open source but then the Community member had changed it to enable it using C++ and now you just opened up opportunities to have low level Hardware that's able to produce results um at an in comparable rates uh with Quant able effects and the fact that now the world can can take advantage of those models and open and start new companies and new Solutions on Hardware that's generally commoditized and easily accessible but now enabling new levels of AI new levels of capability that would have not been possible unless it was built on the shoulders of giants sure but would we even be having this conversation if Facebook a private closed Source company had not dropped their model weights online like every model that's come after that has been derived off of that base and I think it's an interesting question debate like why did Facebook make llama open source and like I I don't know I I think there's a ton of different reasons but either way no matter how you shake it they spent at least 10 if not $20 million on data alone for that model and who knows how many how much compute why did they make it open source and I would argue they made it open source because they wanted exactly things like llama CPP people to innovate on top of their architecture why did they make the license say you can only use data inside llama I'm pontificating but right but now that you're you're forcing Innovation on that architecture which will then acre back to Facebook directly and so I don't know I'd be curious your others take on why open source I mean it's the same reason IBM love Linux right in part because it commodified like sort of one of the key strengths of their biggest competitor and then over time it changes the dynamic of the industry right and I think like for meta the great thing is they don't make money off of this and all their competitors make a bunch of money off of it and so they kind of kick them the groin and feel really good about themselves right and then they become Heroes again and I think that that's that sort of commercial interest and like that straightforward strategy pie that's P torch right like I do think it is a a way we there's another aspect which is the human aspect of closed versus open source and people's careers to the extent that software developer is still a job that exists in 5 years right it's not all taken away by llms then we do believe there will still be software Engineers uh if you're going to work uh on the biggest thing which is AI and you're going to Google or you're going to Facebook or you're getting stolen away by open AI then you you it's a it's a mark of Pride right uh why do you have conferences like these right why are people talking about Scala and all these open source Frameworks that's how they get promoted in their company right let's be serious right like people a care about building these systems more than they care about which specific company they work for and and so by the way I agree with almost all of that and I would just contend though that if you were actually trying to make a really great business for 2029 that actually your core would not be an open- Source model but instead your value would come from a bunch of other places that end up being kind of proprietary for various reasons I think it kind of just boils back to like what is business it's it's you know if I want to charge someone money for something I have to have something that's worth charging money for and if all of my things are open yeah like right you know I mean well I mean you could I I guess conceivably you become a Services Company essentially right but but people want DCS don't like that well but they're great businesses they're greats but but but but product companies subscription companies companies that make Revenue at a like look Google I if you rewind what 20 years all the articles about Google were these mindblowing they only have end people and look at like the the revenue per employee is astronomical right and well look at I don't know open AI I don't know what their numbers are now but there's a similar story that these things the these llms the you know we'll just call it AI now I guess uh AI doesn't exist so it's always going to be called AI right we're not there yet uh but to the extent that open-source llms have Unleashed an entire sector like Facebook is a great benefactor right people spend a lot of time bashing Facebook but in this world what Yan laon and and and these folks have have been doing they've been on the Forefront of pushing for open source when it comes to Ai and I do think there will be companies there will be billion doll companies built on this stuff no doubt um it's it's just hard to know I'd say the question is how many Clos Source AI companies are going to make it I think it's very risky to be a close Source AI company right now right what do you mean by Clos Source AI like building a model that I think we're we're getting caught in the technicality between what is open and close because for everyone everyone it seems that if it starts open source it's open forever I would say open AI gpg 4 is closed right correct we the code is not available the weights are not available the paper is high level yeah yeah that that that is for my definition that is CLO I and it's so just confusing to have this debate open AI is closed yes okay Facebook is open let me try like another sort of like sort of approach on this which is sort of like a question like what is open source good at and what is it not good at there's definitely a way that open source is like terrific for so many things but it's notoriously bad for user interfaces right the you know soort of like we've just not had many any great examples other than there was never a good open source Windows right that's right that's right and there's almost a way in which like in those cases you had to like deal with a very sort of narrow deep set of questions that actually close source is better at addressing and one version one suggestion might be like the way we think about models in the future is that the ones that are going to be very successful are going to end up becoming more narrow and then you're going to have to build all you're like collect all this data that you're not going to want to sh with other people you spend all this time just getting like your little end to end thing working very very well and in that way you know sort of the the sort of like the models that will make that you'll make money off of as opposed to the models that you used to either get promoted or to piss on the pot of your competitors like those models will actually be more like user interfaces and thus like sort of more commercial are you arguing that the competitive differentiation is the workflow not the model like if I think that's kind of what you're getting at in some ways well actually no I'm I'm actually in this case trying to make the argument that like all the all the hard work you're doing around like collecting just the right data presenting just the right data like that all that narrow narrow narrow work is actually more analogous to the process of making a user interface than it is to the process of like solving like you know like you know Linux is great because it solves a wide range of problems that people have done in a distributed clever way right and I think like that's I I think that's another argument for why perhaps like um open source might not be the right primary way to think about models but what I guess when you think about when a company becomes close Source it starts from you know a group of people looking to solve a problem and that in that case that problem is still a problem because it's not a close Source solution that has provided a known solution for it so someone might take the stance of saying I'm going to go and create a solution and they have to almost come come up with a choice of looking for open source Alternatives and making a change augmenting it such that they have a secret sauce or a solution that then enables that that solution to come to to to fruition and then that becoming an a moe if it's in end of itself to solve that underlying problem that ultimately comes from open source principles as a foundation it turns into close Source because they don't release that mod because they have to license it they have to make money off of it but they can theoretically you know lease release certain components that they believe that the industry might be be able to benefit so I do think that open source it starts from an open source principle because they're going to be taking things that are off the shelf relatively available low cost but they're going to be providing that Niche that customers are willing to pay for to solve that specific problem and that IP is going to always remain private just like many other companies that have very strong IP that they can't just give away otherwise they would just no longer cease to exist do you think that's closed or open source closed why um because the they didn't choose it the foundation because it was open they choose they chose the foundation because it was there and available and you want to build your company as fast as possible if there was a close Source I mean it's by definition if I had to license it from somebody else but I had full control over it and it was 10x better than the open source solution which am I going to choose probably going to choose the Clos one because fundamentally it's not a philosophical Choice it's a business choice and so I just go back to all the businesses that I've talked to that like they don't have a philosophical position whether it's open or closed they just want to get stuff done but so I'll go just want I just want to go back to the data which is if you agree that data makes the model and that whoever provides the most value so has the best performance will win over time then and companies take open source models they then incorporate their own proprietary data but by definition can't send those weights back into the open source Community how is the open source model going to continue to advance at the same rate so for like an insurance adjuster or something right if they don't have access to that data and fundamentally it never will be open uh but I think that is organization dependent right so that was an organization that made a choice so for example like like a Salesforce uh in the earlier wave of AI ml work uh the the team the ml team working at Salesforce similar to many other Enterprise companies were constrained by the like Master service agreement that was in place where everybody is treated as a silo right I think I I I may be wrong about this but I think everybody has essentially their own Oracle database with Salesforce their data is locked away there um that in the same way the the AI team within that Enterprise company like Salesforce can't blend the data from all those customers right which is extremely limiting in terms of innovation now they still had a great AI team that did a lot of foundational work um but you compare that to Google or Microsoft where the agreements were set up in a different way where they could use that data uh to improve these models and I do think you start to see a a divide but I but I think that the idea that um you know the ability to feed back that data like like uh would would open source I don't think we've seen that to be the case right I think you know the seeds of this are the engineers and researchers at big companies who can subsidize the fundamental research or universities uh they compound and build uh the difference is there was open-source software and then there's been these worlds like scikit learn and other things where it's it's both research tensorflow P torch it's both at the boundary of of research and open research along with open source software and it's a rocky world where sharing data is still messy evales and is it polluted like replication is hard but we need it to be open anybody who does who Forks off I think in the long run they're they're crippling themselves because they still have to keep up with everything happening in the open now you've got twice the work right you've got your own code base and all this other but meaning are you suggesting though that it's a bad idea for someone to find tune a model and keep that model private for themselves no I'm I'm just saying that it's a bad idea to Fork the universe and develop proprietary models and not be in the ecosystem at all like Facebook is doing the right thing but keeping their keep they're they're keeping everything alive I think get the the the framing I've heard of this debate is well it's data and that's the only thing that matters in the model and like yes like I would agree at some level like you know you need your model to be fine-tuned on your proprietary data but at the end of the day that's not the only thing that impacts a model right this is exactly what F the architecture of the model you know the training regime how it's going to like all all of those those fun things that are like kind of hidden under the hood by a lot but there's a lot of complexity there that's what Facebook essentially did was they made that open source you could arue they leave everything open source but you know right they want people to build on top of that architecture so I would say to your point it's kind of it's about releasing that back you don't want to go build on my own version of Transformers I'd rather build on top of the the open standard for it f i mean you talked to lots of customers like sort of what parts of those that resonates and doesn't resonate yeah that's a great point I think soit like the talent thing is very real where people don't want to hire expensive ml Engineers because you need someone who can fine tunee the models and they know how to fidget with these weights and they deploy them if someone's doing it for them they're going to take that um what you were saying is it's less about open source or even closed sources can I run this securely I really don't care where it's coming from um and that's where I feel like it's if you're building the next company hopefully the model is only a part of it there's the data there's the user experience and there is probably some hopefully domain knowledge that you're bringing in the AI hopefully is a important but not the only portion and then it could be closed or open source it doesn't matter so I think from what we see in the market it's what is quickest what is highest quality what is cheap and what can I get talent for um I would say that seems to be winning out at least right now thanks I think it's a point I'd like to open this to the audience so members of the audience you can ask a question or you can make a statement and both teams can answer the question or respond to the stat statement this is kind of a statement kind of a question so they there's that Mike Tyson quote Everybody's Got A Plan till they get punched in the face and in this case I think everyone's got a a plan until they actually launched the thing and realize their inference costs are blowing them out and and so I I think in a lot of these cases the argument for open source comes down to and I'm sorry I forgotten your name at your point of yeah this prototype is lovely but once I run this thing in production I need to take these costs down really really fast and then all the accuracy needs drop out and all the desire all that other fun stuff drops out and I go to open source not because of a cheap license but because I can get under the hood tune the crap out of this thing and drop my inference cost does that resonate with you that would be the question Andor please tell me where I'm wrong I'm not sure I completely caught that that was for me correct or that was just okay I'm not open source good because you can aggressively fine tune and kind of optimize to the frontier of cost quality latency for your use case is that kind of what you're driving at yeah does that resonate does that make sense I would ask I would ask the question so if a company had a base model that was 2x more performant than open source and they offered the exact same customization and fine-tuning abilities and they were both the same speed would you use the open source model or the close Source model it depends on how much the thing costs on imprints that's my point if one's costing me a dollar and the other one's costing me 25 cents I'm going 25 cents every day yeah so you're going to go with the one that's the most efficient yeah yeah cost and I and I guess your point is like it's a great question which one's going to emerge is the better solution and I think it is a little bit of these questions of time scales right that there will be a period where it makes a lot of sense to be open source and there will be another period where it makes a lot more sense to be closed source and then there'll be a period after that where it makes sense to be open source and I think that like the my best version of why closed Source makes sense for like this next period is just that we're going to be at this point where like the value is going to be that the guys who end up doing a bunch of proprietary stuff and you're right long term then something's going to come modified you're going to sort of say oh let's make this part open source again because we can all work together and so I I I think that's I think that's the strongest argument for closed Source I'm not sure is that like Fair I'm I'm going to jump in one last bit of anecdote because I realize this may actually is a friend of mine working for somebody under NDA just went through llama 2 and like rewrote big chunks of in assembly code etc etc etc and managed to take it down to like a quarter of the gpus that anybody else can use and maybe that's underlying it just you understand that's my understanding of what I can do with open source that I can't do with open AI with open AI do you plan on contributing that back to the community that's a really open and interesting question but just to be clear that's what I'm thinking about is like look if I can take this thing down to run on one or two A1 100s instead of four or five that's worth a ton of money to me on a run rate basis once I'm doing inference and I can't do that with chat GPT so that that's where I'm coming from just so you understand anecdotally and and and I would I would 100% agree with that if you really think about companies that are able to deliver solution they're going with first open source principles they're going to start from a place where they have control of data like if you go close Source you're not really you have no autonomy to really make those decisions you're working with folks you can of course convince them that is a big use case for your company and that would potentially Drive significant Revenue but at the end of the day you don't really have full control if you're able to have the expertise um to be able to do it yourself now you create aote and going back to the initial question which one is going to be potentially a billion dollar company it's going to be the company that actually makes that decision to go open source and have the autonomy to make those changes you mean you mean build using an open source model that's that's start start in some ways like for you guys the key question is would you start with an open source model or not right and and I think we're approaching it from a different perspective which is you know before you make money yeah yeah exactly exactly and what's good enough like I think your argument makes sense because llama 2 is actually good if it was still llama one I don't think people would be using it at all and then let me try another one just out the size like so which is like I I I buy this idea that like you know sort of building hry makes lots of sense in lots of ways but if you building an application literally right now like and your first version would would your first version be on top of gbd4 in order to figure out what the efficient Frontier was and then you'd like you AR is that an argument for open source or closed Source an argument for using the best model well no no I mean like I'm just trying but I'm just L way I mean I'm justy but you could all right let's rewind uh I don't know what is it 15 years 10 years I I don't remember now you're a VC and you're talking to a founder and you asked them like why don't you build your first version on Heroku right and lots of companies you know and startups built things like on things platforms like Heroku and then their bills just go through the roof and someone you know another VC in a later round looks at the startup and says why the hell did you build on Heroku what what you're build are astronomical why don't you just use like an open source platform you know so now and so seriously speaking though like to me though like the interesting thing is in the case of building on top of Heroku initially is you just get the product Market fit and you can find that out much much faster and I think the same and then you outgrow closed source and use but remember like the question is where are you starting right and I think like the question like I mean long term you do a bunch of things right you know like but but as far as like where would you start I think the answer is you start with like the best model and for better work you start with the fastest path to Market to learn what you need to learn which you know in some cases could be open source in other cases be CL but I think I mean but you agree that mostly right now if so seriously like if you literally trying to make a decision like tomorrow you're building a new application right you would probably start playing with gbd4 before you figured out how to optimize on I mean it depends on the application right but but yeah like and and they have uh you know to uh chat PT to back up chat GPT who's unofficially on our team right I would say a open source model mind to they open they open source whisper and and and then somebody you know optimized uh what what they open source and now it's like six times faster right uh depending on how you measure faster why did they do that exactly that's what people are debating right now on Reddit and Twitter and everything else or X I'm sorry whatever it's called um I I think that uh that would be an example of if they were around today if like they made that same decision today would they do the same thing well just like Facebook they're probably thinking Well audio transcription is not our competitive Advantage who cares it's not that valuable to them maybe it's just a piece it's enough but but but it it engenders uh a lot of advancement that they could use maybe they just don't want to invest in that right now I'm just hypothesizing I mean my guess is that like in a lot of these cases you know sort of it's like kind of like the oh we can do this either we can show it off or you know sort of we can commoditize someone else or we can screw L of people's business models and I and I do think that like if if honestly speaking if you're like a 2024 and you're starting a business and you're like oh how do I make this the billion dollar one I I don't know that you start off with the assumption that we're going to open source everything we do yeah no I agree and even if you look at the open source companies today they're really open core right they they they open the core to enough to get developers in and then once you hit the hard stuff there's the SAS platform right and that's that's the Playbook and so i' really argue like is Lang chain really open source anymore all right guys I think uh we're coming to time open a whole cat open a whole cat of worms so uh here's you know uh the final part so now you heard everything right you asked some questions so there are closing statements so you guys are basically let's let's have each team five minutes to make given everything you heard each team will distill the best argument all right and then we'll vote again and see whether we SED any sved any votes so close Source go first with your best argument for close Source you guys sure you want to share your argument though is it close Source or open source I think that the essence of the question is if you're starting a company in 2024 right are you by default sort of one starting with an open source model open sourcing the work that you're doing and like for your core core value added parts of your business and I think the other question is where are we in the business cycle are we at the point where it makes like at the early early point where it makes sense to collaborate a lot in open source a bunch or are we also at the late part of the market when it makes sense to commodify everything and that's also open source or are we in the Middle where there are lots of gains to be had from people who end up building interesting things that match the needs from the market I don't know what are other angles I feel like you had a couple clever technical angles and you had a couple good experiences from your customers I guess we're summarizing that yeah I think uh if you're starting a company right now you're going to pick the best model which is open source right now so unless H sorry it's closed source we'll talk about our real feelings later um but this was in my talk earlier lesson one start with gbt it tells you where the boundaries are don't worry about open source until you get your thing working the thing that you're taking to Market is not the model if it is the case then you know different arguments if it's actually broader than the model just don't waste your time on models right now just get the damn thing working and then you can switch out models Etc um and these things are really expensive to build so fewer people will have access in the long term um okay I would say it boils down to a couple points uh one is just around like resource allocation and that closed Source companies have way more money to throw at these projects and uh data assimilation and cleaning is extremely expensive and um I think whoever is going to amass the most data in a particular domain will have some sort of Advantage um but I agree like a model isn't everything right it's not it shouldn't make or break your company AI should be a tool not a feature um I think it also comes down to Quality um and consistency you know I would say some close Source companies have great great code quality some don't but in general there tends to be more focus on things like putting in production and security and hardening and following best practices and compliance and and all this stuff um and then you know I think lastly it comes down to who's going to be able to deliver the model the quickest and at the low price and there seems to be this like Tipping Point in size of company where they're able to invest like Microsoft in this infrastructure to run language models very quickly for low cost and I think I would love to see but am dubious that we're going to see a similar type development in the open source Community but I would love to see it but I don't think it's going to happen is that the final word that's it sounds good you know there's there's one angle that I don't think we wanted to break bring up that I think folks use sort of a little unfairly which is like the National Security angle right that there is a way in which like potentially some of these models end up becoming quite important that like so of we have import export controls and um I don't know exactly whether like the models we're talking about hit that point but there will be some point where someone says oh wait a minute like you can't like let other countries use this and um that conversation I think we just didn't want to bring up but I'll throw it as the last anyway all right this is great this is great I really like the tie in to the AP Summit AP Summit is happening in in the city right now so this is great geopolitical angle thank you James all right now you heard the strongest statements from the close Source team open source team that's your final word um I I feel like this debate could have been a little less confusing if um we were open about the fact that everybody is using open source models right in some way even if you're at one of these closed Source companies just to keep up you have to be using open source models so uh I think that the idea that you're going to have these companies built on closed Source models and there like there could be let's say there's four to five bigname startups that everybody wants to go work for and use and that they're built on on some open AI I'm sorry they're built on some closed AI models I think it's very unlikely right like people mostly follow the Heat and you could look at like what uh Hadoop and those that era and the cloud era like somebody has a breakthrough everybody moves towards it it in this case you know yet again Google like gave it away for free right right and they and they publish papers and everybody's just like taking taking up the torch and it's it's it's now up to the people in this room to build on the back of that open foundation and to keep it keep it moving and keep Society moving forward so that's what I think is going to happen I think the we're like there's a lot of conflation that's been happening about you know open source and then how does my business model work and fundamentally I think you know in closing here right open source models look Le to Innovation they lead to all of these great benefits for for companies out there and that's different than you know having an open source model and then saying I have a successful business model and and I would encourage us in our vote here to not vote on the business model but to vote on The open- Source aspects of the debate originally raised and Ching B my closing argument the preposition of the argument at the very end okay cool if you don't like the conversation change it come on yeah and I would say for me would be you know with open source you're able to lower the costs of your your your your resources your run weight you're working with commoditized um you know assets uh you're able to move very quickly um if you do go with the close Source route of course there's not really a a mo there's not really an edge that you're able to do to distinguish your customers CU your customers can just go with the same close Source solution so if you're that company that's going to be really be that billion dollar company I mean you're going to have to come up with something that's unique and it's going to be built on open source principles that it's extensible modular and you know reusable like without any of that modularity or extensibility you you don't you wouldn't necessarily even have the The Leverage to be able to create that space I think um so I think it starts with the open source but then potentially changes but I think that's where the Journey Begins like I don't think anyone can just buy software that anyone can access and you become a bu billion doll company unless you do something unique but then you're going to be leveraging some open source capabilities that has extensible feature yeah we we have like Coke and Pepsi we have McDonald's and Burger King how we're not going to get five more open AI right like you've got open AI it could be a flash in the pan it could be the next massive company that eclipses Google and Facebook and Amazon uh we don't know yet that's a business model question that's not what we're here for right we're here for open source open source open source all right all right guys you open source open source open source and open source now open SCE let's we heard both teams let's take the vote again again I remind you the motion is that in five years the new billion doll companies will be built on open source AI who supports this motion now who believes that open source will be the way to build new companies all right we have approximately the same number of hands maybe maybe maybe even less maybe even less all right now who believes that in five years the new billion dollar companies will be built on closed Source Ai and we have even fewer hands for that position so I would say approximately it's a wash well but we have some change so I want you guys to thank thank our panelists I think we had a lot of great topics illuminated through this debate so really appreciate all the all the discussion and with that the 10th anniversary scale by the bay is officially closed thank you very much for attending and please counter join us in this Temple of cod data nii next year I think this is the our home from now on and the cfp opens in May so please think of your see you know your best submissions if your company will sponsor us sponsor for us we will be great this is one of the few independent conferences left standing so we did the first 10 years let's make it the next 10 years thanks guys so what do we ask