Scale By The Bay 2021 : Panel : Startups and the ML landscape

Scale By The Bay 2021 : Panel : Startups and the ML landscape

Recording: Scale By The Bay 2021 : Panel : Startups and the ML landscape

hello everybody welcome to our second scale by the bay panel as you know we kind of do really interesting panels of real interesting folks and uh i'm really happy to introduce the startup and the ml machine learning and ai landscape panel uh today uh i'm alexis cropper from the founder and organizer of scale by the bay also around the area ai which is one of the oldest uh ai meetups which actually was the very first i think and i'll beam it up until everything biologic has to become my eye right so we still remember if you remember uh data mining and things like that information is durable even uh and uh we have a great panel with us today we have uh james cham from bloomberg beta we have smarity who is uh uh stephen marity but gold by samarity and we have licia lee from rosebud and uh samaritan was at salesforce and met a mind and before that he was actually one of the first speakers at by the way events he was a speaker at decks by the bay in 2015 this was the very first ai branch of by the event so we started before again ai became fashionable to the extent as it is today and uh he actually talked about common crawl uh and so uh basically i think that's very indicative of the theme we have here which is you know big scale engineering uh feeding ai right this is it's not about notebooks uh and and just kind of math it's really about how to make things work at scale and build businesses with it um which work at scale so that's kind of the the origins of this uh and now i'll just uh invite you guys to introduce yourselves and for the audience feel free to ask questions in discord we have uh volunteers who will relay these questions to me and i will keep an eye on them so when they fit i will fit you know the questions ask of the panelists you can address them to all of them you can address them to individual panelists all right so welcome guys uh so i had the order here james lisa and smarity so please introduce yourself and that other james first i am james cham i'm a seed stage vc at a firm called bloomberg beta where um we invest in i don't know whatever the future of work but uh because we started about eight years ago i got lucky and got to sort of be early at a bunch of machine learning infrastructure companies and um and so i'm an investor in companies like you know initially crowdflower and weights and biases fiddler streamlit um whatever it is that severity is going to do and stuff like that and then i think that the exciting thing about the time that we're in right now is that we're just to the point where people [Music] sort of no longer believe in magic and are mildly disillusioned and thus see the massive opportunities of things that you can build with machine learning and i'm especially excited to hear what lisha thinks in part because she not only comes from academia but also was briefly a vc was smart enough to decide to take the jump and start a company and then she's pivoted and learned and so i'm very excited now to turn this over to her as she introduces herself even though i've stolen most of her thunder oh no no no worries at all thank you james um yeah i mean i don't know if it was a smart jump but it was definitely something driven by a need to to build yeah this particular product so um i had a lot of fun being a vc um yeah so uh hi everyone my name is alicia i am the founder and ceo of rosebud ai uh we are in a one-liner synthetic media platform for creators and what that encompasses is really you know what galvanized me to to build rose but there's a lot of like stellar research coming out of deep learning in particular in particular that made content creation you know more specifically visual content creation far more accessible and potentially a lot more intuitive um this doesn't maybe use the traditional computer graphics pipeline um and relied on a lot of just novel architectures that can produce um images and videos and motions um that were you know either stunningly real or extremely uh fantastical but but really artistically uh beautiful so um you know i i think what was really interesting um to to me about this tech was also that you know mainly obviously in the early days stem from academia um but it needed to be molded into a product and so there's that relationship between getting the right data from users thinking about the interface and like how you can reveal certain complexities to the user make it you know almost stupidly easy so it's actually a charming product to use and so i have a lot of fun basically thinking about that every day building rosebud and i have you know and so i wasn't just joking that i'm actually genuinely looking forward to sort of interrogating some of the process you went through because i think it's going to be true for many of the folks in this audience as far as your ability to pick interesting research right and then sort of find an interesting problem and then create an interesting product and then a business model and i feel like in part because you come from a unique intersection of academia and entrepreneurship and sort of uh nbc you'll have really smart points and hopefully good learnings for everyone although the real quest will be will you make something as valuable and entertaining and as important as as crawling the entire web hmm cue smarty sir hello yes my name is steven marady um and i've had a kind of fun history along this path uh as as alexi mentioned uh i was one of the earliest tests by the bay speakers and that was when i was working at common crawl and it turns out comic crawl were trying to crawl all over the web um kind of provide a competitor for someone like google give this data to startups and academics without them having to get their own crawling infrastructure turns out that was very useful for text analysis so i ended up trolling towards um a startup at the time called metamind where we did very early pioneering work in language modeling when in the earliest papers i had they replied with language modeling very cool but like it's completely useless to any practical application things have proceeded since then metamind was acquired by salesforce and uh basically in that time i was an ai researcher i worked a lot of new neural network architectures mainly for language modeling um and at some point i decided to go off leap into the the darkness and try my own um startup as well and uh thankfully i was supported by bloomberg beta and uh james sham specifically and um i'm kind of on a path as he mentioned of uh trying to find what research is out there um working out what the problem is pushing towards product and seeing if you can get a business model established at the same time and for me it's been especially interesting because uh i i think language is like humanity's longest running program in that we have put all of this compute into it it's cased all of this like knowledge and thought that we've done and we're really just scratching the surface of it and the fact that language modelling is now expanding the other modalities is especially interesting to me but the real problem and the reason i went from a researcher to um out in the real world is that if you think of the language modeling tech that we had even you know two three four years ago most people on earth still don't have anything like that the best they might run into is like gmail auto complete or the the text completion they have on their phone and it seems like there's a very large gap just waiting to be filled and that's a very different problem than you're scaling to 175 billion parameters as many of these companies are doing thank yous thank you guys this was an amazing reduction and i just want to kind of emphasize another theme so this is scaled by the bay right and in physical real world we actually are in the ninth year of running these seven first years we're actually running different locations by the bay the first was a santa clara and intel building which is donated to us and i think we charged 70 bucks for snacks and coffee which seemed like an outrageous sum then to self-sustain and then we kind of grew and um last i were in oakland in a beautiful masonic temple on the shores of lake merit i really hope you know we come back and uh we were at fort mason on the pier and bloomberg bed is located on another pier uh and you can pass by it if you go to exploratorium with your kids uh or friends and uh alicia i think hails from berkeley right which is a local uh famous place and folks come from stanford so we really i think are at the epicenter of intersection of research startup and community right i think the community is a very strong theme here and you know the internet archive is in san francisco also you know had a fun you know time visiting brewster there so i think we really kind of uh it's not for nothing we gather here by the bay virtually and i really hope we reconvene so we have a plan for the next year to summon food trucks uh near fort mason probably so i really hope you know if you're watching this that that you make some plans to to be here in reality uh and uh i would actually like to kind of you know start with these themes uh so what's really interesting about this panel that we have you know so-called deep tech founders right so and and obviously you know coming from research uh spaces uh it's it's it's a non-trivial endeavor so metamind i think started with a stanford founder and by the way right so metamind i think we signed up richard socher uh to connote scala by the bay scale by the bay when he was still met in mind and uh two companies which basically submitted talks were acquired in between cfp and the conference so i really recommend highly that you guys submit talks uh to our future conferences it's a very strong signal uh in that sense so right but we we really have here in the salesforce forces research labs right and i think i heard first for attention about attention in deep learning from merity at uh the rooftop of uh ai by the way so it was all new right it was it so happened in four years the whole nlp area was defenestrated all the grammars people lovingly cobbled together for decades kind of where outperformed right so so we're in the revolution driven by research uh and uh obviously it's not something like building a pet store in some sense but in other senses it is like build a pet store because or uber because you need to scale but you also apply research so i'd like you know for you guys to kind of uh talk about what does it mean what do you think about kind of the most interesting the the hardest uh uh part of building a deep tech startup right what you want to win by applying really breakthrough science and uh for james how to find this how to fund this right and as you said is in this progression from research to business plan what is the hard what are the pain points right it's obviously not easy in any order i mean i can jump it oh yeah i'm just jumping quickly i mean also just in reference to james's question i mean for myself it was really driven by this urge to um in the problem space of like creating content and visuals like that was something i was very intrinsically interested in you know there's a there's this kind of feeling of like okay there's so much like maybe storytelling or images in your mind that you want to project into the real world how do you make that easier and so the aha moment for me when i was looking at all this research coming out of deep learning is that this is the key to that but there's so much you need to build and so even though even though maybe like figuring out who the initial super fans are and like who will be sort of the most you know whose pain point you're gonna solve first is going to migrate as you do more user research and actually build ship and learn um it was really driven by that product urge and i think that's really what um uh what helps guides me as a north star i mean the name rosebud just comes from really just it's not actually a citizen kane reference though maybe that's the ultimate reference for it but basically in the sims there was a cheat code and rosebud was how you get like a hundred or thousand simians so i could just stay and build my little virtual world and it kind of felt like that magic but like a lot more powerful and you know alicia when you looked at the trajectory of things when you made this call you know sort of how did you decide that this was something that was close enough that you can build something versus something that was too far away that you should have that you know that you should maybe just fund some grad students to work on yeah um it was actually really the technical risk was the thing that i wanted to do risk first so in the beginning even just like shipping something on mobile even if it was kind of further away from say storytelling at large you know i wanted to focus on faces because i knew digital identity was important so actually the first thing i shipped was this like more simple makeup uh app um but um but once that was due risk it was like okay so models can be served in this way now it's kind of waiting for both the research to to keep on getting better and then us to be at that forefront and then figuring out interfaces so it was a combination of both the leap of faith faith and then de-risking that first but later on i realize it's always like the tech will catch up but it's actually the product side that presents more um more problems uh that that need be risking although i would just note that i feel like you're one of the first people to articulate to me sort of like an understanding of the architecture of the problem and i think that like although that's understood now there's always this realization that for in new types of technology oftentimes the real questions is not sort of can i solve this one problem but even just understanding what are the components and how did they string together and thus you tackled different problems than sort of some random business person or someone who's too academic and i think that was part of that mix of pragmatism and understanding what the structure should look like you know that architecture question i think is something that you figured out and it's just worth oftentimes misunderstood from someone from teams that are too technical oftentimes they won't ask that question from a pragmatic point of view and then from folks who are like too business-minded they won't even think about it right and they'll just think that this is just like building a normal sas application yeah i mean i definitely agree especially with the latter point because i feel like there was an unfair advantage in understanding where the research is now but what where each trajectory might go in a couple of years and then anticipating like okay well you know for you know back in maybe 2018 that's when we first or maybe early 2019's when we first saw style again it's like okay of course we're gonna get better at manipulating these latent spaces and making a lot you know easier to use and then you can anticipate it's like integration into more traditional 3d engine so it's like what do you need to build and when that hasn't arrived yet but to anticipate and move fast um when these things will arrive that's right because oftentimes people will talk about how brilliant xerox park was but it did take steve jobs to go visit them and say oh you know what all this stuff is boring and won't happen for another 10 years this stuff is going to happen in three years and this stuff we're going to focus on right now and thus we're going to build sort of their bad first gui based application the product leave the lisa and then build the next right one but like that sort of ability to you know see the big picture and then also figure out how to act i think is sort of one of your unique strengths and then now it looks like smarties about to say something so i'll i don't have a natural transition samarity so you'll have to just talk so the hardest part for me in terms of deep tech is exactly what you two have said which is that okay you have the research research is hard enough as it is you spend all this time de-risking the research and then what you now have is a start-up and as we all know they're incredibly easy um like once you've once you've de-risked it you've then got all the normal problems you're asking like do i actually what what is going to get me towards product market fit can i actually do something scalable and repeatable um what marks do i have once this is all out once i've proven that the research and can be practically used in the real world and shown what the problem can actually tackle how do i make sure that someone doesn't come and steal my lunch um and so you know even if you have all the right things in terms of when the research is what it's leading towards as you know xerox park did or general magic did yeah you're really hoping that you have the right skills or you can get the right people on your team to then push to that next level as well make sure you can you know keep your moat up and make sure that you're actually building correctly towards the future beyond taking research and putting it into the real world um and that that for me is a real question as well because uh i i really enjoyed the way that alicia has been approaching as you said you've de-risked it and now in my mind you're uh because yours is a you know image and animation manipulation product you've been exploring uh and seeing well increasing your luck surface area and pushing it out onto say tick-tock where it's a perfect testing ground for this thing because you know early tech for all these things is a little problematic um but tick tock is also the perfect audience for that because you know i wish i brought my hat but you know they're they're perfectly happy with putting a hat on and then that's me as a secondary character i pull off and i'm on the other side of the room um and that seems like the perfect space for for something like rosebud to actually bloom yeah i mean to respond to that i mean it was definitely a learning experience and in some sense also not completely driven by the data and more a little bit by personal taste but like at first i thought you know more stable specs could could be found about the product by focused on businesses but i found that in this particular area they just they move not only slowly but also they're a little bit afraid because it's synthetic media and there's both this tinge of like oh are we replacing humans versus what i really want to push and see in this which is like it amplifies our creativity and so in on the consumer side you can just test a lot faster i mean the business model is a little bit trickier but actually i think the things happening in web 3 and crypto if we ever get there is very exciting for that and you know finally finally we can like you know creators might um uh are kind of starting to to have much better business models um but at least for prototyping and testing and figuring out where the pain points are the consumer side has served me really well for faster durations and you know i just want to call out one thing you said which i think oftentimes gets lost as people develop ways to think about startups which is you know like i don't know there's data there's lack of availability of data and then there's the reality of like you're doing this because you want to make a dent in the universe in a very specific way and a big thing that i look for as a vc is to be honest like the will of the founder right and their desire to impose something because i don't know it's like kind of a terrible idea to start something you know you could get a much better job as a researcher that's one of the big companies and hang out and get you know even now very good food even though under covet is a little bit harder right you can you can it could be super reasonable and you know you don't have to commute anymore to those companies be really nice and to me like the thing about the great founders is like you know sure they're data driven or whatever but they really have this burning desire to do something right and that that willingness to impose your will rather than wait is like is i think critical and should not be underestimated you know as we end up talking clinically or theoretically about starting something yeah i completely agree yeah sorry go on yeah i i i ended up leaving a you know very well paying position as an ai researcher and yeah it was exactly for that reason it's it does help i will have to say like obviously incredibly well paid well compensated um and the fact that in terms of the safety net like the worst i can do is stumble and like return to your big tech companies um so that that does put you in a very different position um but you know that the the potential lost in terms of going out to start something of your own is in my opinion probably quite quite high compared to a lot of other industries um but yeah the the kind of will have found a resilience burning desire that type of thing i feel like that is entirely necessary um but also it's one of the biggest advantages because um as we're seeing all these existing companies try to reorganize to work out what like an ai first company looks like it turns out you know communication is one of the biggest problems and trying to reform communication at these large existing companies or reform the way that they tackle products it's an impossibly hard problem at the best of times let alone when you need to you know re-establish the way in which maybe art is being meshed and matched so copyright is starting to get a little more questionable and if you have an existing platform that's going to be a problem um or convincing you know the right people to give you a million dollars in you know hardware startup budget to go and tackle some of these strange problems so i have another question for you so i mean this raises another interesting question for alicia like i know when you were a vc you'd have to ask people so what's your remote blah blah blah blah right as you running day to day how much do you think about questions like that that's a super realistic one i mean for the long term i definitely think about it it's more like a balance of the practicality of like shipping things fast like you're probably not going to have the mo mechanism immediately just because the bigger risk is like well people even use this like is this something that they want and then you know having a plan for the mo later on so i guess yeah i mean from like a you know vc or even just like angel investing perspective i think what you were saying about the hunger to actually solve this because in all ways it's like almost irrational to be a founder there's too many ups and downs you know like maybe first six months is the honeymoon phase and then it's just like oh my god like the most high the highest highs but like the lowest lows um but it's just that urge to like okay i want to build this thing i don't want other people to build it or rather it's more like i have this specific division i don't think other people are building the right thing and and that's you know because i was having a good good time being a vc honestly it was great there's nothing wrong with it um but just like i wasn't building rosebud yeah have we taken too far afield alexi did you want to guide us in some direction because i have a bunch of other questions oh this is this is very good but actually i wanted to maybe make it a little bit challenging and specific so and also kind of close to my own heart by kind of asking uh hypothetical slash kind of you know uh test questions so so you know uh recently right i was around this conference full time i built a business running this in the community and i thought i'm gonna do it forever and of course you know black swan of covet happened so i had to go work for the man so i now work at ibm and actually what you described as a research uh lab it's it's beautiful right my parents worked in research labs and soviet union and actually i find this extremely collegial it's extremely collaborative right it's like university but your paid industry rate might be not google rate but it's still livable and uh what's what's very interesting right so actually i'm looking at um the question was how how do you productize large english models from the industrial point now and you know so i'm very fortunate in the sense that i connect to folks like you know stanford started this center for research and foundation model so now we have this new fashion right and i talked to samaritan about it before so let's kind of look at the kind of current fashion right which are large language models right so i think they kind of repeat like the deep learning cycle a few years before that right but in one hour since something captures people's imagination in terms of deep learning you know different things but in terms of language models you know human-like output right you know something talks something speaks is there intelligence in there so now we have like you know gpt3 obviously captured the imagination of a lot of people open ai kind of you know aligned with microsoft so now we have startups but i don't see any startups doing anything useful uh except marketing copy which is low risk and kind of you know you don't prove anything there right you can generate marketing copy so uh and and obviously this marriage is in the nlp state and and alicia is in the visual space but i think you can use these models for hybrid and james you are in the vc space so how do you guys look at this right so when you're the founders right and you see that this thing sweeps human imagination lots of money is going to be invested some of this is going to be lost maybe a lot but maybe something's coming going to come out of this as as deep tech founders who understand this how do you process this do you think i should do something in this space uh do you think like you know obviously you're not going to compete with the beginning academy of science they already have you know almost two trillion parameters lexi are you asking us whether you should start a company is that what you're asking us i'm asking you uh so you know like you you know it's like what's marriage when you're looking at this right and you're an open space how do you think about this technology should i alive with this technology should i go against the grain should i do something different or should i follow my stream and do it better and james what do you think should you look for outliers or should you uh invest in the mainstream and see who tweaks it faster or does it something cute with this but in the general vein of this momentum i'll start off by saying like uh when i left um ai research to go and do my own startup i i had the option of raising a large sum of money um but i purposely shied away from that um and the reason is primarily that i i don't think for a lot of these large language models startups that there is actually a good move there um as we've seen uh there's this fun game that you know we released some massive language model and then a year or two later either an open source collective or another company or whatever else have uh released an equally terrifyingly large language model um to compete with it and the the question then is you know what what do you have that is actually what are you building towards are you just you know burning millions to turn it into a few billion parameters that are outmoded in a year or two um and so that's why i've shied away from that the way that i primarily look at it is i think that it's entirely possible you can kind of invert the problem um and whilst it's not in the language modeling space it is a machine learning setup um descript if people have seen that which is you throw in your video or audio and it will automatically overlay it with text and you can highlight text and like delete a chunk of audio you didn't need a massive machine learning model for for doing that audio analysis and in fact you could have started up with it very early yes once you actually get to this good like ui ux feedback loop you can start throwing in magical features by throwing these you know large-scale models at them but the real problem and i think the the the question that i'm looking to answer is where can you actually apply these things and what can you then do with that what what justifies that you know multi-billion dollar uh sorry multi-billion parameter multi-million dollar you know language model or vision model or audio model um and for startups especially you don't you could raise the money and then use it for training this stuff or you could just try and solve the problem that you're going to have to face once you've finished training model which is how in the world do you actually use it productively um but i'm curious how james looks at because there's another option as well like an uber liftage angle of well you throw money into it and hope that you're picking the winning horse um and it may well it language models are certainly one of those things where you can keep throwing in money and get better results out so one of the best options at least for putting in more capital okay i have so many cliches that i'll throw out that i'll just throw out and then i'll have my actual answer which is gonna be another cliche um so one is of course in the long term we're all dead right so you don't have that much time you know depending on what you believe longevity you have between and whether or not you're eating too much food you have between 10 to 70 years left right to do something interesting and so like you could wait forever or you can actually act right now so like that that's one the the other cliche of course is like i don't know if startups never have moats there's no mode like there's like a ditch that you're that you're gonna dig really hard that you're gonna hope means that the huns don't come in and kill you right and so there's a little bit of a you can't really dig anything that's like you know like you just can't do anything really that deep and then the third cliche which is also true is you don't know until you do right that there are formulas and then there's stuff that can only be figured out through computation right by actually sort of like doing something and so i think that like for better or worse sort of we're at that stage where the where the business models are unclear enough that we're still trying to do discovery so we're not like dealing with vertical sas applications right with a vertical sas application honestly but the playbook is understood you know sort of go and read saster and you'll figure it all out right and then and the question is just can you execute better in this world part of what's exciting about it is that we're still in the discovery mode and we're still in the discovery mode not just the technical architectures though but also critically of business architectures and so i think one of the interesting things is the only way you can and the only way you can play that game though is either by being very close to the market and have everyone talk to you because everyone talks to smeardy right like that's one of his advantages of being where he's sitting or sort of like building an early version of something with technology that to be honest probably will be commodified five to ten years from now hopefully not five to ten months from now as lisa's doing right in the case alicia like someday maybe it's her company maybe someone else going to build infrastructure that makes it easy for normal people to do but she's got this she's got this lead right now where she can then explore sort of consumer applications and consumer use cases and then figure out where she's gonna go right and i think like that's that's sort of like the the game that you're in the middle of playing and then for you alexi to be honest like i don't know like if unless you feel like you really want to starve and put your family at risk you should because some idea is so compelling or some thing just driving you crazy then you should stay working and adding a lot of value at ibm right now it's a great organization they do good work it's important stuff their tv ads are spectacular you know like you should take advantage of that right and so so that's that's right okay and then now i do have to tell you though now as a selfish self-interested vc i'll tell you at least my dream right so um uh i think that we're at the baroque period of sas business models where like everything is well understood and it's all kind of boring and it's all sort of like diminishing returns i think that we're at the beginning of folks figuring out business models related to machine learning and i think that if you think about that relationship between how people build software and how they sell it right we've been through a number of distinct phases right sort of like if you were to think about ibm building sabre which is the airline reservation system for american airlines in 1960 american airlines paid the equivalent of 300 million dollars right to build that system and they sort of ran it but they didn't really make any money right and then if you don't think about bill gates and microsoft you know sort of bill gates gets paid whatever thirty dollars for every copy of ms-dos right but then the thing is like he gets paid for the exact same copy of the exact same software so his marginal cost is zero and so like that's the that's the most beautiful business model ever right that's why he can tell people that's why you can pay people to kill mosquitoes and so i think what's interesting though about that is like that shift in business model was not just because bill gates was grouchy and was able to impose his will it was also because technical architectures changed and the way that software was delivered changed and so then you go ahead to like i don't know whatever 1994 when the first browser comes out right and you go to 2000 and you imagine like mark benioff being a good sales guy at oracle saying oh no no you know like this whole like client server architecture is wrong we have ubiquitous client we have like these data centers where all the stuff should sit right and then he says everyone's freaking about out about the fact that the data is going to be shared in these data centers but don't worry it's the cloud right you know like like so you know like it's like marketing architecture right you know sort of like that plus the fact that he's been able to convince people to say hey we should paint a subscription business model right and like that though that subscription business model is is not just because it's like fun but it's because it reflects the technical architecture of the web and it also reflects like some of the unique business openings that the web provides right this sort of always-on always delivered software and i think something similar is going to happen with machine learning and and that there is going to be some you know machine learning is so like you guys forget this but machine learning is so terrible and is so full of all these issues around munching data between different clients and people and it freaks out normal people so much right and yet it is so powerful and is also so fiddly right and so so what's interesting about that is like there's some unique mix of marketing plus business model that fits that way of delivering software or benefits that's going to be interesting and they'll be dominant and then like in 2040 everyone will be complaining about that business model the same way that i complain about subscribing software and so that that's the search i'm on right i'm searching for that that guy or gal who's like likely gonna be like stuck in einstein right or stuck at facebook as a as a product manager and that's the and i think that's the that's the big as a vc that's the big opportunity so it's less about and that's a person right that the person who comes from a specific context with all sorts of biases and ideas but also like this vision of what it should look like and to be honest i just don't know what it is which is why i'm a vc and not lisa actually i'm curious james what um interesting um business models or maybe hinting at what you're what excites you about uh about kind of like exploring new business models here i mean i think you know the obvious the obvious one of course is what's the best sort of like uh well how do you say this like like the earl if you just say who does who did the best job of munging together user data and making a bunch of money off of it right and so obviously it was google right you know google did the spectacular job of sort of like co do create like i'm trying to figure out the right way to say this like you know sort of obviously search move to like the way they do search now is different than the way they searched before right but the way they did search before was like crazy collective intelligence problem right and so you know sort of their ability to do it for free and then take that and sell it to other people it's like sort of like that's that's like the that's like the classic sort of consumer business sort of collective intelligence business model i think and i think like that's probably going to be true until regulators catch up right you know and but i think that like to be honest like you look at how confused regulators are right now around dealing with the companies that are as old as google or as old as facebook they're going to be confused for a long time they're not going to have any idea how to deal with machine learning so you're going to be able to get away with a lot or i don't say get away with like you'll have a lot of opportunity to innovate and try different things so that's that's one on sort of my world which is more b2b world i look a lot at the data co-ops right all the different data co-ops we you know sort of whether you're talking about like there are a bunch of healthcare ones they're a bunch of like fair isaacs of the world right where um or give to get models and i think like those are interesting and then there's like two radical ones that i think about that are sort of almost inverse so like there's the insurance model which is i pay you a little bit of money every day right in hopes that you'll be able to tell me that you've protected me from something major at some point and then there's the investment banker model and the investment banker model which is the most interesting one to me and i don't know exactly how this could be software i have the screen what you should do with this but the investment banker model is so i'm the vendor you're the customer i pay you money like every day for a long long time and then one day you pay me a lot of money right basically the investment banker is like i will take you out to dinner you know or take that random thing you know so that seems like it's a lot of money but actually isn't that much money and then one day you'll pay me 30 million dollars for what seems like a weekend of work right and um and so um so those are probably like examples of the variation right [Music] actually related to i mean both of you hinted at the commoditization of some of these models so going on the very you know almost like an extreme part of the spectrum which is okay just assuming that there is no moat there and actually you know i think for a lot of these companies it is like that so like the whole source of companies being built on gp3 actually in defense of them i think they're doing a really good job just figuring out the productization of um of like where the pain points are so yeah sure we start with marketing copy because it is a huge pain um and paying for gpg-3 especially if they kind of molded the prompts in a even if it's easy work it's just molding the prompts and giving them a simple interface that was enough to grow you know for some of these companies from zero to like 10 million are in a couple of months nine months or something so i think the question definitely remains as to okay obviously then a lot of companies can do this but having that head start i mean to your point james i think then you get a lot more interesting user data about look at what other entrepreneurial problems do you have and then like when you know when they can ultimately have the capacity to train their own models etc they can feed into that cycle so moat is not a problem at first get that like right form factor for the product get into clearly validated by the actual money that you're making and then i think there's like a bigger potential after that so i actually i do like that flavor of like company building even if it is like against you know me being a technical founder it's like okay like they didn't have any expertise on the on the large language model side but i think they did really well in the product formation um that's a great great point i'm now looking at stephen marity yeah i i think that they have done a great job now the the question i guess for them is i i think the real genius of like gpt3 was more along the lines of as you said like they didn't have expertise on language modeling side what instead they did was ask a huge audience how can you make money with what we've given you right it not quite as a marketplace like it certainly hasn't gone that far but it suddenly has like the first hinges of that um and you can imagine i don't know people releasing like prompt packs or something um because if i i think where this theoretically fits very well is if your company isn't actually um if like language models aren't going to be your core focus then you just want to externalize it you don't want to have to pay incredibly expensive engineers for it you don't have to worry about like the gpu architecture the the standard idea of uh you know electricity for beard uh production or whatever electricity is commoditized in that situation um the two questions you then have left the left are if you are providing this as a service if you're a gpt-3 how do you compete against or how do you maintain that merge against um you know the say um i can't remember the name of gpt gpt no neo models like the these open source models or from other companies like tokyo are coming in um but you know that that's a good thing for the the end customer because they rapidly have choice um so maybe we'll just see the flourishing of that field anyway regardless of multiple competitors or the one that i'm particularly interested in is if say language modeling or machine learning isn't a um value add in your company if it's the core competency um how do you actually maintain that and how do you grow that as you know your your competitive advantage like what what problems and what industries is it going to be necessary either because you can't share that data or because your problem is so specific because they don't tailor towards the language that you're focused on um that you would actually need to have that as your your you know core competency so let me push back on that i'd say two things i'd say one you don't need to know at the point you know you don't really need to know the answer to that you just need to be in the position and have enough resources to discover the answer for that in the same way that if you were to look early days at google i don't know they had exactly the wrong business models right but in part because they figured out like con like their product on the headline was so technically innovative but also so used by so many people right that they were able to gather their resources so that they were to buy the company or copy the company that figured out the right business model right and in that that ultimately like what's their real mode is it real mode that search is better or is it remote that consumers think of them first or is there real mode the fact that they have all these like very clever hooks into advertisers all over the world right you know sort of like you know that that but that all gets built over time and then and then i'd say there and then let me tease up one other distinction which is you could choose to build on top of the language models meaning you're building applications or as you're suggesting which i think is why you're so exciting stephen is you're looking to say you know what that existing way that people think about these large models is basically wrong right and at some point the world will figure out that that that basic like that you have seen the future because you've lived the future and at some point people figure that out and while you're still playing around with that what that world looks like then you're willing to work in a relatively asset light way and i think that willingness to say i disagree with everyone i disagree with like sort of like whatever the like latest marketing term from stanford is right you know it's like i just grew up all that and i'm gonna keep on sort of going after the architecture i believe in i think that's part of what makes you inspiring and sort of why it's always fun to talk to you yeah it's a the fun but very potentially high risk which is exactly where you're supposed to be with startups but i i feel like it's a one area of research which is a part of startups that shares with research which is there there is obviously a lot of good progress when it comes to the field of machine learning um you know continuation of existing architectures pushing forward transformers these massive amazing levels um but i'm always kind of softly traumatized i was chatting with lexi beforehand um and he was working on engrams back in 2005 or something like that um and you know you jump back to that era neural networks do exist but they're not really used svms and all these other things are king um and yeah like the there needs to be someone who really perks and prods the completely different direction to see whether or not there's just something out there um and i think that's that's where i really love if we're talking about terms of passion and everything else like that's if i if i um work out one of these completely different soft lung ideas that actually ends up working that will have me hopefully is a successful startup but if nothing else i'll die happy i mean you know there's a way you think about all the people who thought that mainframes were terrible and evil and how long they were out of fashion until they weren't right you know you know sort of like and then and then that's one part which is like long-term mainframe for the wrong architecture but then they're the right architecture again right so like there's a thing in which like also the other reality is like i don't know some things are cyclical right you know some things are fashioned and they not fashioned in the bad sense of the word but they you know like they move back and forth depending on sort of where the culture is at so um okay alexi have we convinced you that you should stay at ibm or are you now sitting here i really want to be like stephen and lisa you know as much as i kind of you know had this idea for a while i uh you know i have four kids two cats and one dog uh i covered poppy which is a big puppy so i feel like yeah i am enjoying ibm thoroughly you know i'm like you know i just joined basically this you know uh year and so yeah but you know so i'm very happy to interact with you know scientific and startup and you know industry and community uh so but you know maybe i'll ask you so there is a question actually from the audience and i think it kind of fits with what we're talking about so the question basically is you know there is this cliche there is a problem looking for a solution solution looking for a problem and so normally we see steer people towards problem which looks for a solution right and so like all this traction um so other situations where solution looking for a problem is acceptable for of course there is i mean i think it's one of those stupid things that i'll save when i'm passing on a company right i mean it's like i don't know like i'll say that and do i really mean it maybe kind of sort of but you know i think of course what's the point of a market a market is a conversation right a market is a conversation between solutions and problems and hopefully they match up right and so there'll be times when people are able to see such great solutions or some you know have some interesting angle and they're just constantly running you're just running around trying to find some match right and so you could be problem centered or could be solution-centric it doesn't matter like both can win right both can win this is not like but you know so this is a case where you should never listen to vcs when they pass on companies they don't know you know they're all idiots i like i really appreciate the radical honesty um but no i mean yeah it's just hard to demarcate so i agree um i guess as long as people are just talking to users that would be my only one one add-on it's like don't don't build in isolation um obviously a definition of user can be very different whether you're building for researchers developers you know could be yourself if you're very close to the user profile but yeah just don't build an isolation yeah you know and i think right because that point is critical which is if you're going to build an isolation you can do that you can be a monk that's that's a reasonable way to live but it's not really the reasonable way to build a business right and so because they are conversations right they have to be conversations between both sides otherwise you'll never find the match i think the other interesting point for you know solutions looking for a problem is that sometimes the solution is what opens up the field towards actually having new and interesting problems or problems that are suddenly solvable um like one of james's uh favorite things is the advance in in audio um like speech recognition um he has this brilliant video of him with his google pixel out and speaking in real time and having it filled in and i think as a vc he's pained because like you can see such a beautiful solution and it hasn't been a price applied to so many of these different problems um and yeah like they're they're the the fact that you know machine learning has opened up all of these different modalities we really haven't scratched the surface like clubhouse we're still handling how to do walkie-talkies like chat rooms for audio with clubhouse uh we certainly haven't done anything particularly interesting with with machine learning and audio um but i don't know james what's your obsession with okay i think as you say this i have only one concrete piece of device like you should never listen to vcs about anything except for this one concrete piece of advice which you should all go on twitter and follow ethan malik m-o-l-l ethan m-o-l-l-i-c-k he's a friend of mine who's a now entrepreneurship professor at wharton and he does nothing but publish like sort of like papers that have actually been validated for long enough that like are relevant to your life as someone who's thinking about starting something and one of his great points is this reminder that like the technology is available and then it still does take an actual human being to like adapt the technology and make it useful with some product like it's not like these things don't happen magically right there's like money on the floor all over the place just waiting for someone who's like has the gumption to actually build the right product and then go out to the market in that person you know sort of like they're the ones who start companies and become incredibly successful and hopefully one day we'll be able to convince you know we'll be all going alicia asking her for money to you know for some political cause or you know to endow some chair from some university that's a great idea yeah and actually you know uh my degrees from upenn some very you know um uh have very warm feelings towards wharton so i'll make a note maybe you know just to buy next time i go go to philly that's you know they they're very well known for innovation uh and tech right mb like tekken b as well some famous people went there uh so um yeah no this is this is great so another question from the audience is i think it kind of connects in it's a little bit kind of vaguely framed so i'm not sure what the exact kind of question is but it's interesting why don't we yet have ai for communication i wonder what this means and maybe you guys can interpret it in any way you want don't we have ai for communication and isn't aren't variations of that what lisa's working on i mean yeah i obviously can like reinterpret this to to um talk about rose but again but i i mean i think yeah just at least the problem space i'm most interested in is just how do you express yourself most intuitively and in in terms of you know the thing that i lean towards is like visual i mean ultimately steven also brought this up earlier like language models are entering different modalities as well and you just see that intersection so interesting it's just because you know people want to bring out and tell stories and ultimately like on online their digital identities could be a lot more um in sync with their just like what's in their mind and what happens physically so that's the problem i'm working on i know there's a lot of other manifestations of that even all the way to search um you know if some income if somebody wants to take up google search and replace them i think that's a that's another thing on large language models and and communication but you know funny i have a friend who oh go ahead stephen oh i i see that as a my interpretation of ai communication is it the way that i look at language models is that it's like you're staring into a fireplace you know you see all these beautiful patterns and kicks and sparks they're linguistic sparks that are flying up they might give you an interesting idea or imagination but generally [Music] they still are old and falling off of that that is one of the i'm losing steven is everyone else losing yeah yeah i'm sorry yes uh but basically like i think a lot of the existing machine learning tech isn't actually advanced enough uh communication is one of the the most intricate forms of of back and forth that you can enter into with another human being or a machine we haven't reached that point where we can do that well yes um and i i think the area that a lot of machine learning is in right now is it's more the hallucination of communication um the images which give you like you know the linguistic spots if it's a language model giving you marketing pros that you then bash into something that's slightly more usable or in the case of you know many of the the most recent vision advances it it gives you that that randomness that you think okay i kind of if i squint my eyes i can see that angel in that picture where i ask for it to show me an angel you know like that that is i think an interesting stage for it but i i certainly don't think we're a great point for like communicating with ai okay exactly at least my dream go ahead lisa oh no yeah i mean i i don't know i see that the people who are sort of maybe i'm interpreting communication a different way but just even the the whole swarm of um gpt3 companies you know working on marketing copy that's a form of just making it super easy to do a very narrow set of things but you know making it easy because they are paying for it and then on the side of even descript you know you mentioned it's like making videos and audio clips like easy to to to kind of edit just like a script rather than having to re-record i think those are meaningful progresses and because it's like you don't expect agi to help with that i don't think or at least that's certainly not my um expectation of how where these products evolve and then of course on my side it's like it it's not meaningful if you're just like conjuring up random images but it is meaningful if people are trying to you know like how i think about um what's where it's interesting and bottoms up game building is you have these like large um gaming studios obviously producing amazing stuff but like how media is kind of evolving is having more and more of a creator class until the creator class is really blending into the audience themselves and to enable that you have to make the tools of creation far more accessible and so i i think when i interpret maybe communication more broadly it's just like what are we even doing in the internet and why what are people like what what do they want to communicate like let's help with that rather than maybe replace them you know lisa i think that's like a great angle that sort of that notion of saying we want to create tools that make it easier for people to create things i think one of the failure modes right now that you'll see often is the attempt to use ai or machine learning to come up with systems that are foolproof like you know sort of the moment you set up the problem like that you call it agi or you call it whatever then you're basically just inviting people to find out ways to make you look stupid and i think that when the emphasis is instead on have creating user experiences that in some ways are more about training the user on how to use the product rather than coming up with something that is promised to be foolproof right the more you do that the better off you're gonna be you know you know sort of reminds me of like i brought the siri guys in many many years ago to do a partnership presentation for their series a and i admit it went super well until one of the partners basically asked the question about like how far away pluto was and of course siri didn't know like there was not in the like they you know but but the way that it was set up was such that it was promised to be this universal answer thing and this is like 2007 or whatever and of course it couldn't do it and yet it was designed in a way to make people mad because they were gonna look for the boundary and then say oh this doesn't do what i want as opposed to a system which is like the way the script is working was like it's really i'm encouraged i'm being trained how to use the script and i'm happy with it and it does what i need yeah okay lexi have we convinced you to start a company yet uh we'll see we'll see uh so uh i think there is a one more question which will be the last kind of in this main uh segment and then i'll ask you guys for a kind of uh take away one take away one you know thought to take home and after that you guys can go to a special chat but the question is like this to sum it up uh essentially we have a rise of new things in tech so in the eye we see the growth of uh safety and ethics and responsibility now we have actually two talks commendalunky noted uh from huge and face on air ethics right so they they uh uh have harm mitchell uh then uh and ricardo basiates uh talked about responsibility so we have this one they have uh sustainability so people care about how much energy ai consumes right and and also there are things like ai and power there is a book called uh paul uh ai atlas which talks about aib in the tool of oppression and the instrument of the powerful uh and so the question is uh how do you think this affect the business of ai uh and technology in general how does it make you feel think and adjust do you think this is a direction where the business will go is it something which kind of will change the nature of the business i think that's the general question how do you grapple with these new directions where you know society grapples with the eye okay so everyone should buy rob bryce's new book i think it's called system error like so robbie and maron are friends of mine i have this conversation with murano often which is or i just had not often like we've had this week we've had this conversation which is like if we went back to 1950 when everything was written in sort of like like sort of machine code in 1951 someone writes a paper that says there's this idea of compilers and then we're in 1951 and grace hopper writes the first compiler right sort of what would be the sort of questions that you would have asked of grace hopper before she released the compiler or the program linker and then whatever whatever the next like i think that i think that in some ways it's far better to think about that as the thought experiment than to think about right now because it's just very hard for us to see what's right in front of our faces because we're in the middle of it and like i don't know i'm bothered by whatever political situation is happening or whether a geopolitical thing but if we stuck ourselves in 1950 and asked those questions and sort of said okay what are the limits that we want to place and not place i think like that's the interesting question and i would suggest that like there are like um it's just really hard to predict and by focusing maybe too much on the technology instead of the underlying values we end up coming up with stupid policy right and so to me i care much more about coming up with consensus around what are the things that we care about as a culture and then using that sort of like that underlying set of principles that we can agree on to inform what people do rather than sort of fight about specifics around ways that we should be talking about models or make them discoverable or not because i think like until like we come up with these broader underlying stories and like sort of fight about that i think it's hard to you know sort of do this other exercise and in some ways i worry a lot i probably worry more than other you know look i'm a vc so i've got my interest right so i'm probably more interested in experimentation than the most normal people right but i do worry about like sort of premature limiting of things that might turn out to be like perfectly fine right i mean that's such a good um thought experiment i'm going to take that with me and because it just really places i think why it's so difficult to do this i mean i'm clearly a technical optimist i'm like building stuff for the future and i'm obviously this is technology that's been talked about in very i think almost yeah it's like grayish ways a lot of um a lot of talk about how this is like not good for humanity and i think i you know i agree with james on that like we have to agree on the broader goals and i think you know people who are optimists about tech are you know they're not trying to bring down like the fall of humanity um and it's also hard as a startup founder to have any goal outside of like what your product is doing so as long as you're making sure obviously you care about these broader theses but you're like focusing on building your product so that it both satisfies that does no harm and then just focus on building like that's what you can do to actually bring forth like an effective change in the world like i don't think you should as a startup founder have like too many broad goals about like you know something that has nothing to do with your product so yeah i i don't think you can prevent the tech from getting out um and as james said about the grace hopper and the compiler the one that i always think about was my father when the first heart transplant was performed he was terrified that that meant that you know the rich would live forever because they'd get organ transplants he was a little too sci-fi on that but that was like that was a funny looking a good imagination and forward-looking clearly inherited but it clearly inherited traits but yeah you don't want to borrow problems from the future and you can't prevent this tech from getting out the the best thing in my opinion is just to get education and get as many people as possible as you can to to be able to understand and be able to implement play around and extend these technologies because that's that's where you know society starts playing around with and understanding it working with it and you know if we were thinking about like the personal system now postal boxes wouldn't exist because there are adversarial attacks against them that are trivial you know like there's if you're borrowing all these problems you can get every problem in the world you want whether it works in society and whether society can help you correct the deficiencies is the real question um and i had a friend deliver the book this is a book that james was talking about um i was part of the the stanford course with james and bloomberg beta it is a really brilliant take on a lot of these kind of ethical questions and problems that technology is bringing at us so okay i have a third recommendation so remember it's ethan malik system error and the third one is there's this guy named cesar hidalgo who wrote who um who's um i think in europe now but he was at mit for a while and he did this incredible thing which he doesn't even appreciate in which he would do basically a moral census of how people think about machines where he'd ask a lot of people questions about like if the machine did this would you trust it more than a human if a machine did that under what conditions would you trust more or less than a human and i think that it's the pro it's like it has limitations because it's only here in the u.s right but it gives you right now the best map of what people think about like the pros and cons of how people will will or will not want to work with machines and in part because morality shifts right like because as the technology changes the economics change and people change you know the what we what we are willing to do or not willing to do will shift and i think like his book and the research that he and his team did i think is like a really valuable contribution and also for startup founders it also shows you what are the holes in the market that you can take advantage of and what are the places that people will freak out thank you jen this is this is fantastic so i think uh you fulfilled that uh take home uh recommendation uh plenty right so you have three recommendations i recount ethan malik caesar dalga and system error so this is your three take home um tasks and everybody will be graded on this next year guys if you read this and what you thought so alicia now give us your take home recommendation oh sorry i didn't i didn't get the memo on that homework oh maybe come back to me also okay see if i recommendation each of you will give to the audience to think about so smear it if you have already yeah well the the take-home i would say is for me and then from the other panelists i i'm looking at these very strange ideas and what the mercs are for the future and what ai first companies are and how the feedback loops will occur the the one and it's been hammered home to me in the past as well but i think it's worth reiterating to the audience is yeah what is a head start you can get before something commoditizes how can you take advantage of that because for a lot of these deep tech companies that's that's where you're at like the the market the field the business models everything are currently evolving what's a head start that you can get if you can't work out emote because as it turns out you're going to be punching the face and all your plans are going to go to hell but you know what what's that head start you can get so that you can hopefully build your own before commoditization thank you very much and lisa if you can yeah i mean i have no specific resource to recommend but the thing that's top of mind for me right now because my users are creators are just there's so much demand right now to kind of better monetization uh for creators and it doesn't have to be solved by say you know nfts um though i know that is sort of like the the main uh it's very hyped up right now um but i think the problem though why it's so interesting for me to kind of look at these web three things as well as other monetization strategies is because it's a real problem like creators are not getting kind of their share of the the kind of upside and what they're creating but like we still enjoy what they're outputting on top of that there's a growing creator class and that all of us are kind of participating let's remix culture i mean um you know even with tick-tock eventually there's a spectrum where like most of the audience is actually participating by contributing i think this also gets really uh it kind of connected with your um point james about like data collectives and like how do you reward contribution for you know your data here like data is also just like contribution to content and curation um and i think that's all related so i mean just something to leave the audience think about is maybe look at the um the things that are happening around nfts through that lens and not to port over things from web 2 over to nfts and rather think about like how does that give the right structure and incentives uh toward more more creators thank you thank you alicia james i didn't mean to kind of you know uh summarize your recommendation if you have additional recommendations please go ahead okay i mean you should you should use rosebud that's my recommendation all right all right thank you that's very good that's very good so i want to thank all of you guys uh you know it's kind of uh the idea of this panel i think came on the whim as everything by the way by watching samaritan james took it on twitter and kind of knowing them and you know and i think you know jim say i must admit i didn't know how good of a penalty you are and actually i think you're an excellent moderator so next time i'd like to invite you to moderate the panel and also i want to commend you you have this amazing you know as a photographer i can really admire your focus because your your your photographic focus because you're in focus and the background is fuzzy this is this is like a beautiful leica look i don't know what you're using but it's it's it's it's very nice so really you know uh really appreciate it and thank you very much guys uh so uh if you like for those who have time and inclination you can go to a special chat which is the submersible environment see if people are there it's lunchtime so they might be already going for lunch uh break but feel free to kind of walk around and see if there are folks who won't answer asking any questions and uh until then you know i'll hopefully we'll see you guys in real world by the bay by the food trucks next year and you know in between and by air ai so definitely we'll invite you guys to speak thank you very much great thank you