SBTB 2023: Baruch Sadogursky, Developer Productivity – DIY (with ChatGPT).
Recording: SBTB 2023: Baruch Sadogursky, Developer Productivity – DIY (with ChatGPT).
in in the context of um developer productivity right so how I learned to stop worrying and love thei uh well my name is bar sadurski I'm J Bar on whatever social media of the week and whatever name it is um developer productivity Advocate with with gradal and um started as Java developer many many years ago and then pivoted to devops with um with J frog for 11 years and now talking about DP developer productivity engineering um called couple of books one of them devop stor with Java developers with meisa M she's here and she is also a speaker and she's amazing don't miss her talks and bunch of other very good folks and um liquid software about we really should release software in this DNH so um in anyway the most important slide of this talk is this one if you go to speaking. jar you will find a page dedicated for this talk um I will show it to you in a second and it has the slides already there it has the video hopefully I record um and if not we have the organ the organizers backup they claim that they record so but let's trust them uh all the links there will be a lot of stuff that should be linked and all the links are there um very easy to remember this URL because it's on the bottom of every slide together with um my handle and the hashtag of this conference by the way um and GP DP for developer productivity engineering all right so um what I want to do in this talk so first of all disclaimer it's not going to work um I don't know you you don't see a lot of live coding trying to do stuff with with llms and especially with with chat GPT uh for a very good reason it doesn't really work um what I'm going to do won't work as well uh but we're going to do our best and we're going to see what can be done in order for it to hallucinate code a little bit less um so we will start with general questions about developer productivity engineering what is developer productivity engineering why we want improve it what can be improved and and let's see how that works and from there we're going to take it up a notch and this where it's going to be fun how that worked ah here we go all right does it fall does okay here we go yeah not okay all right [Music] um uh you see my slide yes so um as I promised this is the speaking doj bar you can see here the larger presentation is for our presentation for for for now but what I need is my chat GPT okay so I'm I'm subscriber for chat GPT Plus or whatever it's called so I can use as the latest features that we will need today and we will start with something uh easy we will start is what uh not in Russian what is DP okay let's see what what goes um developer productivity engineering hey it knows about the acronym uh field focus on improving productivity blah blah blah excellent fine thank you very much um and that was great uh how can I improve okay so this is all very naive stuff okay optimize build and test times great use tools like gradal or Maven for official build manag it's it's way too small why are you not telling me that you cannot see anything here we go uh optimize build and test times automated repetitive task continuous integration continuous delivery monitoring and analytics code quality tools collaboration tools Etc okay so whil run I we're going to actually try and improve um build times in a real project it's not a real world project but it's a real project it's a Java project that has tons of uh very not useful classes like this and tests that don't test anything really like this but they do run and take some time so um yeah I'm going to run it um build and clean uh okay clean build I'm going to run clean build and um it's going to run for like a minute I guess um yeah so um here are some really good ideas and and you know what this talk it has many goals in mind and one of them is to talk about llms where we are and how can we improve it but also like a real deal about developer productivity engineering and the ideas that are here are actually very good ones so optimizing build and test times are very important for developer productivity engineering because they minimize Contex switch Contex switch is the single most significant killer of developer productivity uh by scientific studies uh getting back to a task in hand after a destruction takes an average 25 minutes so if your build is about 10 minutes this is probably where most of you kind of are and you don't wait for it just St ing in the screen instead you are multitasking and go and do something else it will take you 25 minutes two and a half times longer than actually just watching the build doing nothing to get back to what you used to do so um Contex which is terrible optimizing build and test times should be one on your top of your list of of ways to do to improve developer productivity automate repetitive tasks this is pretty obvious continuous integration continuous deployment okay monitoring analytics is also very very important because this is how after you optimized your build and now it's great now it's 2 minutes you can wait for it you will notice when it gain becomes four and five and 10 and you start multitasking again collaboration tools for sure improve communication and collaboration with the team using good tools I don't know if we can buy in on this list um but better tools so instead of coming to a colleague and trying to explain what went wrong if you can send them a meaningful link to aign in the build output showing what's wrong it's actually much better right work environment yes for sure um you know what Open Spaces do to people so yes um and focus on developer experience of course so those are very good ideas and let's see by the way it was all multitasking to wait for the build the build is actually 56 second I spent like what 5 minutes waiting for a minute build if you were just sitting and staring at the screen it would be much more efficient you probably would get bored so anyway 55 seconds not terrible not great um yeah so um let's improve a build uh what do we have Java monol with right uh build with Gradle uh cotlin diesel okay let's see what's going on okay incremental builds ensure your gradal build is optimized for incremental builds that means that gradal only will recompile and process changes since the build great build cach utilizing grads build cach very important parallel EX tion enable parallel execution in your gradal build to take advantage of multicore processors very important optimize dependencies sure gradal demon profiling and analyzing and find you task okay that's a good list what it's important to remember and I can ask CH GPT and it will tell you exactly the same thing that obviously optimizing build is not limited to Java builds or gradal build tool right you can do most of those things with other languages and other build tools we can do it with Maven you can do it with SBT you can do it with basil and you can do it on Stacks which are entirely not jvm it doesn't really matter I just happen to have an example built in Gradle because you might guess why but uh really it doesn't it doesn't really it doesn't really have to be gradal in any shape or form right so this is a nice list and now we're getting to an interesting part um you know what before we do that let's switch back to the slides to for a second right so we asked stuff like what productivity engineering why do we want to improve it and what can be improved and basically the answers are Developer productivity engineering has improve our overall developer experience the build is a good place to start as I mentioned it's not only gradal it's any other build tool and it's not only the build with pooke about other aspects as well okay now we are coming to where it is interesting and this is the code so um show me the code let's see what it can show us um okay so build. gradal okay this is I already have Java project but okay this is actually funny enough exactly the build grade the build that I have so wow this is impressing uh impressive so if I look at my build Gradle uh where is it here it is it actually looks exactly the same Java Library Maven Central couple of dependencies and tasks and tests that's exactly the same thing wow this is nice gr properties caching through p through Dem through this is nice I can just take it all there and go ahead and just dump it here and we'll see if it works [Music] um root project I already have Okay so let's see if it helps somehow I don't want to wait for two minutes for like the entire minute but yeah it still runs all the tasks tests although we didn't change anything so that's not great um okay um what does it say test optimization let's see how the test optimization looks like okay this is nice so what it says is you can parallelize using all the available processors this is a nice line so let's do that let's see how that works uh in build right right here uh okay it still took 55 seconds we shaved 5 Seconds out of the build nice definitely not enough now let's run it with Mar with Max parallel Forks still runs the tests wow six seconds okay folks now I remember I told you the show notes is the most important slide this was the most important moment you know why because I just taught you how to justify Max Pro M3 purchase from your employer I took the build from 60 seconds to six seconds just by utilizing the power of my local machine that was nice okay still not what we want right so uh the tests are so much faster okay I didn't change anything I don't want to T to to run them as well at all no build caching for test aha gradal does not cat the results of the test tasks it means test will run every time if you execute the build all right uh what I want is caching right H okay all right okay how do we test it enable build cache okay didn't we do it already uh gral properties yeah we did it didn't help what's going on why it didn't well okay so now this is where it goes into how do we say creative mode right so basically we didn't enable caching there is another configuration part that we need to add and it's obviously nothing of that this is really just random code that looks like it might help but but it really doesn't so yeah it really didn't go as blint what I wanted it to do is to actually show me valid code that will fix everything and didn't now the reason is the llm hallucinations and actually up until last week it definitely got worse as time passed right so the stuff like your um the cutout date I hate to stand behind the podium here you go the cut up date and uh for a while there was no browsing browsing capabilities at all so all the code was really stale for two years ago more than two well two years AG ago um it was probably okay for like stuff that didn't change in two years but for example the stuff that I'm showing to you today well it would be with fine with Maven because Maven didn't change for 10 years but with gr obviously it couldn't work because there is new versions I want to take advantage and it doesn't work right now oh yeah so the browsing is back and it's back for a couple of months now but um the reason why disable why they disabled browsing is because people did like stuff with it they bu they bypassed pay walls they used it for like all kind of Malian stuff so they kind of killed it until they figure out what to do now they did allegedly they figured out what to do but the browsing is very limited most time than not if you ask it to browse something it will go there and was like H browsing failed thank you that was useful so there are third party browsing plugins that stepped in when ch GPT disabled browsing but they're also very very picky on what they can and cannot read there is like page structure that should be supported and if it's not it's just we'll give up on that as well now this is all was through for last week and um then open uh death day of open AI happened and and basically apparently it's now all the way up to I think April 2023 which is much better obviously and browsing should actually work so it should be better really now the way you add your content to upto-date content to chat jbt is through What's called the embedding plugins the idea was you set up your own Vector database and then you feed custom data into it and then you teach CH GPT how to use your vectors using a standard CH GPT plugin the problem with that is that I don't know how about you I sign up for the ability to create and publish plugins with jpt a year ago when it was announced I'm still waiting they still didn't get the invite um so that's also kind of not a way to do it but you know what maybe we don't need it anymore and I will show you why the way to publish kind of your own plugin was using third party services like mum Ai and they encapsulated all that they have a already published plugin that is separated Within by apps and then you can publish your own app which will basically take your own material and publish it within the plug-in once you um sign into it but not anymore instead we had the death day and basically they killed a bunch of startups just by announcing stuff including montium well I hope they will manage to Pivot because they stepped up when needed but now we actually don't need that at all because what we have is GPT Builders custom gpts for we can now all of us can do it and the way it works is you add your sources to the custom GPT you instruct custom GPT to use only the sources no hallucinating anymore and then it should work that's at least the theory let's try it again with custom GPT all right so okay so um this is my custom GPT and let's go oh yep thank you someone pays attention this is my custom GPT and let's go to edit and I'll show you how it works so basically the idea is that this is the the UI for creating custom gpts it asks you um uh it asks you what do you want the GPT to do and you can answer stuff like for example I want it to provide upto-date documentation for the velocity a gradal in product and if it works maybe it will show me how we configured this particular GPT basically if if it doesn't I can tell you uh but let's see oh huh ah yeah okay but it figured it out great okay so basically we set it up and we started by defining the code uh the core identity and the purpose of our the velocity helper it was designed as support engineer specially spe specifically for the velocity formally known as gr Enterprise and we I called it how it should provide detailed accurate and current information related to the velocity now there is a trick here um the velocity is a new name for a product known as formerly known as gradal Enterprise all the documentation still says gradal Enterprise and when I didn't mention it it was hilarious the hallucinations it actually came up with an entire DSL for a product named the velocity bunch of code that does not exist because it's all old names in the documentation so I tell them okay refer to the velocity but in the search actually use Grand entprise and then I told it okay the most important part is here use the documentation that I provide and only use this stuff so it's somewhere somewhere here uh based on our feedback and yeah here um so basically updates to its knowledge base this is where I upload the documentation and um enhancing capabilities basically I told it only use the documentation now this is all pretty amazing of how it works because you use uh chart GPT to create a new chart GPT and you work with it in like for example if you go to mantium it will only very well like will be your traditional creating stuff like so we will have the UI it will tell you here upload our files and here are a bunch of options that you can configure and what's not here it's something else this is a a large language model you configure your new L language model using large language model it's actually very very nice so basically what you see here is a summary of what I told it to do and it was pretty impressive now um the most important part of it is that it is available for everybody who a subscriber for chpt plus which is like not a lot of money um it is available today for public consumption you can create your own GPT and and you can say I want it to be public and then it will appear in GPT store or I can say I can share it through link and by the way the link to this helper is in the show notes page speaking. J Bar so you can go and try it out and this is very impressive and then this is your our debugging interface um I can say here stuff like how do I and enable caching caching not chesing and um it will try and oh it renamed it to W okay so basically now the question of our history of creation became part of the GPT itself okay let's see if that will work um but we can always open a new chat and just work with it here you can say okay here we go the velocity helper support engineer for the velocity [Music] um how do I enable um uh remember how paralyzing tests save us tons of time you know what had more cores than your Max Pro M3 the cloud the cloud someone else um the cloud has unlimited cores and if we can take your local build to run the test in the cloud it will be even faster now let's see if that works um remember I told you no one does that because it doesn't work there you go I'm the one who does it and it doesn't work well maybe it will uh one of the interesting announcement during um Dev day was that you can actually now create chats by providing a seed like you provide a seed for a s random number sequencer that you can will get the same results every time you can do the same here you can provide now you can create a chat with not yet but you will be able to create a chat with the same seed and get the exact same results I'm going to use it for my talks because now every time is completely random I have no idea what will happen and if it will ever work um so yeah state of CH GPT for you people the announcements are great the ideas are great it really should work maybe it will um let's see how this is going same okay so the idea is it looks through the PDF files in the knowledge base and they're not that big and it doesn't really work but yeah okay here you have it great ideas less than optimal optimation I actually played with it earlier you see here today and actually was not bad at all so this was again a session with the velocity Helper and it actually provided me here with let's see let's see how to set up a remote cach for gradal and it gives me okay you do that this is not a thing this is a pure hallucination this is not how you do it um and I'm like okay how do you authenticate and it gave me another piece of code and this is not a real code this is hallucination that does not exist and this is after I instructed it not to use it and then I'm like uh okay this is not how you do it and I'm like oh yeah sure you are correct now this part is actually right so it took it three parts to like three tries to actually get the correct answer and the only reason why I could distinguish between correct answers and incorrect answers because this is what they pay me to know so yeah well there you go um any news here okay it's kind did you like the answer okay uh yeah so let's try it again we tried it again and it was whatever it was okay so how can we really improve our build if this thing actually worked it could tell us that we need to enable caching local and remote spoke about local caching to just skip the stuff that shouldn't be built remote is even more awesome because it can take your state of the project and then decide to skip it because someone else already run the build so they will just give you their results through remote HTTP protocol and you will have their results instead of building by yourself it's obviously very powerful the most powerful part of it that this someone who can share the results with you can be your CI server and you know what CI server do they build your stuff all the time so they will always have upto-date result of the build that you never run how cool is that parallel testing local and remote you saw that that's some powerful stuff right there predictive test selection we don't have time to talk about it because we're out of time completely take a look at that that's pretty awesome it basically decides which tests to run based on AI it will monitor your changes correlated with tests and then say you know what for these changes you don't really need to run this test it will never fail and this is pretty cool flaky test again something to look for and watch for the gradations we spoke about that stuff that you can do today for free is parallel local I just did it and it was awesome local caching remote caching although you will have to babysit your own node of remote caching if you don't way want to pay for something else to do for someone else to do it and build scans this is your monitoring and watching for degradations you also can win prizes for free or if you go to speaking doj bar you will see there what we call a speed challenge basically you speed up your tests you are blown away and you win prizes how about that what your company should pay for I already mentioned top level Hardware because it will make you more efficient developer that multitask l have less contact switches and do more and the velocity or similar if you can find something else this was how I paid for be here okay um learn more and try today uh the QR code goes to speaking. jaru you will find the maven challenge you will find the resources about developer productivity engineering and the developer productivity engineering handbook you will be able to watch the videos from DP Summit that happened R across the bay here um couple of months ago and with that thank you very much I'm at J Bar this is by the bay and speaking toj bar for the videos the slides and all the links are there thank you