SBTB 2023: Dean Wampler, OSS vs. Open Source Software: What's Different? What's the Same?
Recording: SBTB 2023: Dean Wampler, OSS vs. Open Source Software: What's Different? What's the Same?
um thanks for coming I'm Dean Wampler I also work at IBM actually I did report to Anthony who did the keynote this morning uh Carl and I did and uh we might report to him again you never know how things go at IBM um but this talk is kind of uh a result of some things I learned uh doing that experience of working with Anthony on science projects um just uh if you haven't been to any of my talks I just put photos of my um you know travels or whatever and these photos are from uh the bluid parkway about a month ago in Virginia including pulling onto the Parkway and there was like a 60s vintage car ahead of me on the road so a little bit of a throwback maybe you make it while moving well I'm sitting with my phone if you look carefully you'll notice there's actually Reflections from the dashboard in the picture there's you can just barely see there's like streak of brightness right here but anyway enough about the process all right let's let's talk about science um so I I actually started as a scientist I did nuclear physics and it was all theoretical stuff uh which sounds more impressive than it actually was I was mostly writing simulations for uh nuclear and particle processes you know basically doing kind of well not really big data so much but just lots of number crunching the sort of stuff we do today um when I did it back in the late 80s uh we had these things called minivac computers if anyone have any idea what I'm talking about all right some of yeah those of you that have gray hair too um I would start these simulations running come back two weeks later and see if they were done and that's why grad school takes so long basically um but anyway that's kind of actually I discovered I really liked writing software so in fact when I finished my doctorate I got into software as a profession and so I've kind of you know been in both worlds a little bit um and kind of can appreciate the needs and interests and motivations for both sides as well and so recently I I led the engineering team for this group that Anthony was running for Accelerated Discovery meaning like using software to accelerate basically chemistry U uh and trying to productize some of the software that was developed in um actually our Zurich Lab at IBM and and learned a lot of interesting things about it and enjoyed getting back into science just a little bit so I've had this fabulous life of uh you know science and software uh before I go over the topics how many of you would call yourself a scientist oh cool how many of you think of yourselves more as just a regular software engineer probably most of you how many of you are sort of in the middle you're actually a software engineer but writing for science okay oh okay cool great so here's what I thought I would talk about um first uh what does open mean because kind of the topic here is you know is is there an important idea here that we've can steal from software about open- Source software applied to science and what does that mean and how does that make things better for all of us what what are the different motivations each of these groups have and then how can they learn lessons from each other and and I had a definition of the when you see OSS it's open source software and then we I don't know who it was that researched came up with I guess that was Alex came OS osci for open- Source science all right so what does it mean um this is kind of an interesting topic because it means different things to different people so I took this definition or and it's not even the complete definition but a list of key attributes from this open- Source initiative and I'll tell you in a minute where that came from but part of the idea is free redistribution that uh you know if I wrote something and made it open sour then you could use it and run it yourself uh it doesn't necessarily mean Source actually in a great example right now are models like if you use the Llama uh generative AI model from meta you're mostly getting data of that represents the model weights and then some code that's actually run at runtime so we we're kind of although we tend to have thought about open source as being source code in fact it could be a lot of different things uh and including data as we'll get into to a little bit uh this was an interesting uh definition too because they get into a lot about you it should be non-discriminatory it should be open to anybody to use any institution any people uh non-discriminatory in the good sense but we'll also talk about ways in which maybe you don't want to be open in that sense in just a second um one of the big areas where licenses for openness differ is what do you do uh how is something licensed that is derived from your work uh and one of the controversial areas at least in software has been that some of the original proponents of openness namely like the free software Foundation um you out of MIT really wanted to say that if you take our software and use it you have to make the source code for everything you build open as well which doesn't really fly very well in a lot of Enterprise environments whereas other organizations are more flexible and they use licensing that says you can do what you want with it as long as you attribute your the code that you're using to us but use it as you see fit um and then usually there's a lot of details about how the license propagates with the codes you can't like change the license of someone else's software and so forth so let's talk about a few of the complications though about this and by the way I know that some of this is kind of like uh old hat for some of you and U hopefully it'll still be useful but uh I'm trying to talk about things that are maybe not so uh well known to scientists and also things that maybe are not so well known to software people all right so one of the big issues has been commercial versus non-commercial use and one of the big areas that's controversial right now is actually llama where uh at least for llama version one you were not allowed to use it for commercial purposes you could use it for research they actually loosened that license for llama version too again this is the model from meta but other organizations have gone the other direction and it's been very controversial like you know confluent changed the license for Kafka elastic search changed their license um I know some of you have worked with Scala a lot and lightbend and the company behind Scala originally and AA and and they decided to change the license for AA to make it actually require you to have a license if you're going to use it for commercial applications and the companies do this even though it makes everyone mad because they want to make money off stuff that maybe other people are using for free and they're not getting any benefit from so it's it's kind of a difficult part of the open source uh as a business model um now I mentioned that already that you know source code has kind of been the way we've thought about software and openness for a long time but really data is almost more important today in some context like AI where it's the model is almost like the Crown Jewel and not so much the code used around the model or even how you trained it and then this idea of non-discrimination is great in principle but some of you may recall that the Java license specifically prohibited you from using Java to run nuclear power plants um so similarly you might decide that you want it to you know be only used for good uh like no weapons or whatever you define good to be uh but hopefully not in any sort of like discrimination against various groups of people so again complications and if we keep going here um I mentioned that some of the licenses have been considered toxic because they impose restrictions on you that may be unacceptable like the gnu General license uh gnu public license GPL actually um really required you to release your source code if you use their stuff and that sort of is true for Linux which is based on a lot of GPL stuff but other uh systems you know have they've come up with different variants of that license to kind of address the complaints but it has been a real headache for people to decide what software can I use based on what license it supports and generally the rule of thumb in Enterprises is avoid GPL stuff you know with some exceptions like Linux um but it's but aache and MIT and Berkeley licenses are considered you know really easy to deal with and good okay anyway however you define openness a little history is kind of interesting and this is something I just found out this week as I was researching this from this Wikipedia article that one of the first examples of sharing in a sense was actually automobile patents there was a apparently a patent was very restrictive on uh two cylinder engines that was controlled by Monopoly at the turn of the you know the 20th century um somehow Henry Ford won a lawsuit against the patent holder and the outcome of it was they formed a an Automobile Association where they would freely share all their patents they would actually continue patenting stuff but they would just share it with all the manufacturers so they kind of uh decided it was actually better to work in collaborative ways like this than to enforce patents um IBM has actually apparently made the uh the source code for their operating systems and a lot of their tools uh basically available for people to use and there's this old organization called share which is not an acronym and another organization called guide which is an acronym where uh people would take code from IBM modify it you tweak it to do things they needed to do in mainframes and this community has existed now since the 50s uh BSD was probably the first famous Linux variant that was open source in a sense in our modern sense of the word and of course eventually Linux followed that model the actual term though uh wasn't the first term that was used free software was the term that was used by the people behind the ganu public license but the actual term open source was uh believed to have been coined by a lady of uh I forget what her actual job was Christine Peterson uh this was some time ago and then there a slew of things that started down this path of openness like Netscape Navigator famously made their their code open for the you know the first real commercial web browser um Eric Eric Raymond and some other people uh created that open source initiative and became evangelist for open source and then uh O'Reilly the publisher uh and you know the the company and Tim O'Reilly the founder became real Advocates and started the first open source Summits uh there's actually one that's run by the Linux Foundation today because O'Reilly no longer does conferences okay anyway so that's sort of a history of this idea that maybe it's actually better if we work together in the open on things that where we can mutually benefit from each other's sharing rather than try to keep everything proprietary so what are the motivations then for people to actually you know want to do this to want to um you know Embrace openness well for software engineers and this of course I'm preaching to the choir here one of the big goals is to actually collaborate with peers you know to get common needs done well one of the ways I I like to frame this when I'm talking to people about whether like say IBM should open source something or not is you know what's our core competency what's our crown jewels what what is the thing that differentiates us for making money versus things that we need but aren't necessarily going to make us money or not it would be better for us to collaborate with other people is one way I kind of like to frame that question and why do we collaborate well again we can save effort we can get the best ideas out there work together open source often has the best quality of software compared to proprietary software it also helps people learn ideas as they kind of steal each other's good ideas from various open source projects um actually one of my colleagues is a substitute uh speaker this afternoon Nick Hill's going to talk about optimizations for inference with language models and there's like this real uh sort of fervency going on is people have various uh open source and and limited Source um projects for inference and they're keep inventing new ways to optimize performance of something that's extremely expensive which is you know running stuff on gpus and so everybody's kind of stealing each other's ideas if you will um and actually there's also this whole social thing people loved the recognition of hey I help build that project or I'm a committer on that project I really like working with colleagues that you know share Sim similar interests and skills and all that stuff so it sort of meets that social need a bit so also when softer Engineers want to evaluate a project they're going to uh use possibly is they you know they really care about um there like they'll give you 10 or 15 minutes right if if they see a repo has claims to do something they need they want to be able to clone that open- Source repo very quickly uh you know figure out what it does build it if if buildings required install it maybe run some examples they want to know within a few minutes if this really works just WR out of the box and then start to explore if it's really right for them and so if none of those if any one of those things doesn't work for some reason or it's unusually difficult to do something with this software then it's going to be much harder for people to adopt it so and part of that then has instilled this notion that you really have to think about the end user the client if you will of your software if it's going to be good and if it's going to get traction in the industry whereas sometimes you don't think about the kind of low-level uh client or user or accessibility when you're working on proprietary stuff you may only care about the user interface that your end users are going to see and then lastly for software Engineers as a result then we really expect high quality and reliability and all ease of use and all these sort of ilities that are really important for us because we don't want to waste our time because we're always really stressed with deadlines right well the motivations of scientists are a little different but they're also similar in a lot of ways like typically they want to collaborate with colleagues I mean the joke of course is that scientists and software Engineers are real introverts and if you meet a scientist or software engineer the way you can tell if they're actually an extrovert as opposed to an introvert is do they look at your shoes instead of their own shoes right um but in fact people do like to collaborate and uh and you know share ideas and you get the best uh uh of Both Worlds going for them so they're not kind of on their own and kind of bored or whatever but one of the interesting things that's really a main priority that has driven openness and Science in the last few years is the need for reproducibility of results you know if you've done some serious simulation of let's say climate or I saw a really cool one recently simulating the the Collision of whatever body hit the earth to create the moon people want to be able to reproduce that to verify that it's you know good science and so forth so that's kind of driven this idea that maybe we should make everything data as well as code public so people can try it out and verify for themselves whether it works now I put an an asteris here when I say scientist I'm just sort of the umbrella of a team that could be software people as well as actual you know scientists with doctorates or whatever but one thing that scientists traditionally have not been all that worried about is kind of this idea that the software they're writing is going to become part of some you know shrink wrap in the old days or you know a cloud service today what we use this kind of not very great term production for that sort of software that has to be delivered reliable makes money for somebody or you know provides critical functionality usually it's it's a much less formal process in research software historically than that kind of uh sort of b a bar if you will of quality that uh production software people have to meet the and the thing that has been very interesting for me I'll come back to this a little bit later also is that historically probably data has been as important as the actual code in in the in science you know whether it's your lab data or whatever you use to feed into simulations that has often been just as important as any sort of code that you use to do this whereas that really has not been true in so in software um as I said open source the very word actually implies that it's really about source code but increasingly today especially in this AI world with models data can be just as important and I keep saying models but also what data was used to train those models so there's a lot of uh work about should we open source the data sets that we use to train them all right so what can uh you know scientists people who want to embrace openness in the science realm learn from open source software well um it turns out I I heard an interesting talk actually I guess it was this spring at go to uh Chicago which is you know a general developer conference like this one by a guy who'd spent some time talking to people in other Industries about you know their their pain points what do they do I mean people like Architects you know construction people um what were some of the others I think he talked to some printers and one of the things he kind of found out is that nobody does like reproducible processes quite as good as we do and it's actually a strength now we don't necessarily always do it like we should or to the extent we should but in terms of like having processes that can reproducibly cre recreate something that we have built in the past you know like give me version five because I need to patch it for this customer over here or whatever we're actually pretty good at that and those are techniques that are not the first thing that come to mind when you're writing uh science software per se where you might just be doing a lot of experimenting trying out ideas and hoping that you come on to something that's useful but the interesting analog though I think in science is actually like the lab notebook which today is typically you know online with Jupiter or something but the reason you do notebooks in historically in science was so that you could you know go back and look at what happened how did I get to this point why did that experiment seem to go better than the ones I've done since and so forth so it's your memory of what seemed to work and what didn't and similarly uh we have these techniques and software that we similarly Ed for reproducibility and some of those techniques include what I just called Asset Management this is where you store your code or maybe your data or the build tools you use your documentation whatever and version uh version it as you're evolving it and and the tool of choice now is get it's absolutely taken over the world in terms of this this process in software but it also provides ways of not only tracking your work but sharing it with others you know if I have a public git repo and GitHub then I can you know share it with colleagues either in the office or across the country that you in entirely different Industries or whatever and we can all work collaboratively on it or they can just use it if they want um and I think did I put that in here anyway the other point being also that we can uh yeah branch and release in the notes on the right that I can can work in parallel on some ideas while other people are working on stuff I can merge these things in a very controlled way so I know I didn't break anything and then when I release something and say yeah let's let's go ahead and ship this to production or whatever I know exactly what's in it I can come back later and reproduce it if I have to to fix bugs or whatever so it's a really mature process these days and although you do find that a lot of people don't use it as much as you would think and it's definitely like the first thing I would tell a science organization is start using these uh version tracking tools and get in this this uh Habit of just saving everything including Jupiter notebooks or whatever because you'll uh you'll really learn not only to think constructively and sort of rigorously about how how your work is proceeding but you can then go back and and re revisit things that you've already moved past on the other thing that we've uh gotten a lot better on just in my career it's actually this is one of maybe most significant ways we've improved proved is ways of verifying the behavior of something when I started you know back in the pine writing software the way you tested stuff was you installed it and you poked around and hoped that it worked and then you you know did another build later and hopefully you followed the same procedures you did but you probably didn't follow them exactly so any bugs you introduced likely went unnoticed but now we we've uh figured out how to write very rigorous ways of defining behavior and verifying that our soft Ware actually meets that behavior and this is another thing where if you're working in science you have this same idea too if you have a theorem let's say you want to verify uh that it actually works and you want to be your own worst critic right you don't want someone to tell you that you're your research is garbage and you have to retract your paper from nature or something like that you really want to be your own worst critic and this is the kind of thing that we all really find that we have to do together is figure out what's the best way to verify Behavior here so that we're really confident that what we think is going on really is going on does does the software actually do what we think it's supposed to do and does this experiment actually prove the theory that I think it does that kind of stuff but in software we've we've quickly learned and I think this is becoming more and more true in in science as well is that that process of testing no matter how good your tests are if they're manual it becomes extremely painful to do them also if if there's a non-trivial build and installation process c not only is it very tedious and time consuming but there's just plenty of opportunity for human error if you're doing them manually so automation becomes extremely essential um and to the point where really mature software environments basically automate everything so as soon as the developer you know commits to a branch uh some code changes pushes that Branch up to this common repo then automation kicks in to run all the tests to do some integration things that are hard to do on your laptop let's say to really build confidence that this software is still good that nothing has broken by these changes and similarly in science I think I remember some of the software I wrote in college or grad school was like these big Fortran programs this is long before I knew anything about test driven development but it was really tough for me to know that I hadn't broken something because I didn't know anything about testing at the time or automation but now I think that's really these are like power tools even in even if you don't think think you need them so much in science they are such powerful tools that they make you a lot more productive in the long run especially for long running projects like I know for example that some of the data analysis tools that like CERN some of these simulations for climate these are massive massive programs they are decades old I I don't know much about how they're developed but I would not want to be working on those things if they're sort of just ad hoc and there's no real processes involved in keeping these things sane automated tested and so forth the other thing that's to me always an interesting topic is what like language ecosystem am I going to use and I use the term ecosystem well it's not just me but the term that not only means uh the language but like the libraries and so forth one of the talks I was in this morning mentioned that you can find a library for anything in Python which is now kind of our deao choice in science for like the high level analysis stuff even if you're running Fortran libraries that are you know five decades old underneath which uh you probably are um languages that you know are easy to use easy to learn let you think about your problem and not how to capture it in code um but also give you the tools you need to you know do your work those are a pretty important decisions one of the things for me about languages I've always been interested in them um we seem to go through cycles of experimentation like I remember when I first got really got interested in softare in the 80s it was the era of object oriented programming which some of you may remember um and there were just all these object-oriented languages floating around there was like something called C++ um there was small talk which sadly kind of died off but people were experimenting with ideas and and how to represent these concepts of objects and ways that let them work efficiently and then we kind of Consolidated into uh languages like C and C++ we had another uh flurry of languages in the functional programming era a few years ago um well more like 10 or 15 years ago you know closure uh fshp um for Amanda back there um languages like that that really got people thinking in new ways about how to structure software and to think logically about it and then we kind of Consolidated into a few and that's that's where we are right now I think is we're all doing uh either python maybe rust maybe go but already from our keynote discussion this morning we're starting to see new Lang languages emerge that try to tackle the problems of our existing languages like Mojo So that's that's exciting to me okay to wrap up uh fairly quickly here I'm almost out of time but um one other thing that's really important that goes with reproducibility and all that is really thinking about the dependencies of libraries you pull in one of the problems we ran into a lot uh at IBM research was people would just slap Dash throw stuff in their projects because some little Library over here did this thing they needed and you'd even get situations where they actually had several they had sort of pulled in several versions of the same library in their whole system which were incompatible in some sense so uh thinking about managing these versions is really crucial to keep things sane for you and for the people that have to work with your software like the ones who are checking it out to maybe use and a related thing is keep those dependencies up to date because vulnerabilities are identified you know these are like security vulnerabilities all the time and if you keep your versions up to date and you'll you'll avoid some common problems all right quickly what can what can the software World learn from science well um one is this idea that data is actually really important and we need to actually figure out better ways to manage data like GitHub is not actually a great way to manage data it doesn't really like big things like it complains about the size of my presentations a lot if they get over like 50 megabytes so we need better ways to manage and catalog data uh that that it's more of our model of of working iteratively and quickly not just throwing stuff in databases where maybe it's not versioned as well as we'd like and this is becoming just extremely important because of of models and data sets for models and AI um the Science World has been doing distributed computing a lot longer than most of us have in software you know they invented uh supercomputers they invented you know high performance Computing um and this is stuff that we're we've we've kind of reinvented a number of times to figure out how to do this stuff ourselves and in some ways we've built things that maybe fit our needs better in software than highly optimized hardware and yet here we are in the AI world where everyone is trying to throw money at Nvidia for massive GPU clusters because we just need massive amounts of compute so I think learning what what um science has done with scaled out distributed computing can help us you think more clearly about how we should approach this problem in software right so just to wrap up um just make sure I got this right uh for science I think that you increasingly as always in the entire world that we live in software gets more and more important and it becomes part of the research you're delivering so people can reproduce your results and it's a way for of you collaborating with your peers on non-trivial projects so think about embracing these um actually I miss this should be Embrace open source software uh forget that bullet point and and kind of Leverage those techniques oh I I know what I meant by this sorry um think about not just keeping stuff closed but actually opening it up is it better for you to share your work you know at that level rather than just publish papers about it um because then you can collaborate more easily and people can verify that your work's good uh and then for software I think uh we don't know how to handle probabilistic and statistical stuff and that's something that the the science community has dealt with forever even apart from like Quantum Computing so this this is something that we have to get better at um but also being better at distributed computing is is something that all of us in software just can no no longer avoid even though we hide it under abstractions like kubernetes and so forth okay well I'll wrap up there since I'm out of time and uh I guess I'll just take questions in the room thanks by