LLM Avalanche Beyang Liu: Building Cody, a context-awareAI coding tool

LLM Avalanche Beyang Liu: Building Cody, a context-awareAI coding tool

Recording: LLM Avalanche Beyang Liu: Building Cody, a context-awareAI coding tool

thank you all right everyone having a good time okay good going strong all right so uh my name is beyond I'm the CTO of a company called sourcegraph and today I'm going to talk about building Cody an a context aware AI coding tool a little bit of background about our company so sourcecraft is a developer tools company it's used by over a million developers around the world at a lot of interesting companies to understand large code bases and make sense of what's going on in in the code so some cool logos up there uh I think that data databricks company sounds familiar um and one of the things that our Enterprise customers like about us is we take privacy and compliance very seriously so we do not use any of your code AS training data for uh General large language models okay so we built this coding oh someone's taking a photo I see you photo taker all right awesome uh so when we think about applying AI large language models uh to the domain of code we find it useful to analogize towards another AI application domain which is self-driving cars so similar to how there's a spectrum of levels of self-driving car ability all the way from you know no automation to fully self-driving we think of uh AI coding assistance as following kind of the same uh level leveling up path and so going from no AI a couple years ago to I think the first generation of AI coding tools that we saw were kind of is like inline autocomplete copilots so these are tools like tab 9 or GitHub copilot that sort of do the inline autocomplete so complete the next line or the next couple lines of code that I'm writing uh the way we're thinking we think the next Evolution beyond that is going to be context aware AI coding assistance so coding assistants that can take advantage of not just the local contacts that you're currently writing to do the inline autocompletion but can actually search across your entire code base they're actually aware of all the context across all the code that's relevant to what you're working on um and then further beyond that you know there's a lot of interesting work around AI agents and things like that but uh a lot of that stuff is still experimental we've run a lot of experiments there but I would say we're not a yet at the point where we'll call that production ready another way to tackle this problem is uh the approach that we're taking the AI editor assistance is really about combining these two kind of um classical domains of intelligence so one is obviously like the AI uh large language models the kind of like norvig School of how do we get to True artificial intelligence the other one is kind of like the Chomsky camp where these are kind of like classic static analyzers or compilers things like that a lot of people put these two into tension like either or but we actually think the correct approach is to combine them because they're very very complementary okay so what actually is Cody the AI coding system that we Built Well it's this AI that can do a variety of things so it can generate code in the style of your code base so not generating just generic boilerplate it can answer high-level questions about your code it can also fact check itself okay so let's pop into a demo and see what what's actually happening foreign I'm going to say a prayer to the live demo gods okay so so there's a lot of interesting um open sourced libraries these days some of which are featured at this conference so how many people have heard of llama index before a couple of you how many people have not Okay cool so for those of you have not I think one of the first questions I would ask is you know what the f is llama index so we're going to ask Cody this question and Cody is gonna what Cody's gonna do is it's gonna search across this code base for a documentation and code Snippets that are relevant to answering my question and so it's going to take a look at a variety of files uh and give me a pretty detailed overview so you can actually go through and look at the files that it's reading through so it reads through some of the documentation you get the readme and also some source files you can all also ask it to generate uh code so you know show me an example of how to use llama index to create a simple PDF chat bot so PDF chat Bots are all the rage I can ask Cody how to do this it will go through and read the source code and read the documentation and generate a code snippet that makes use of the patterns that are embedded in this repository so again not just uh not just kind of like the local inline autocomplete it does that as well but sort of like higher level questions about your code base since we're running short on time and we'll pop into the editor real quick so there's also a vs code extension that you can install you can ask it the same sorts of questions so I'm actually in the Kodi Source right now the Kodi source is all public it's it's open source you can go find it I can ask it hey where does uh or which file defines the Kodi recipe for making a unit test so we have these things called recipes in Kodi that uh expose common actions that you might want to do with the language model like generating unit tests or generating documentation um nope I gotta regenerate made something up all right let me ask it again see generating you to test all right it doesn't seem where I wanna get that file but that's the file then once I'm in looking at the file I can ask it to explain what's going on in this file and then while I'm doing that I can also show off kind of like the inline autocomplete feature so again this is what we view as kind of like the level one but I just want to show it off that we can also do this you can generate kind of like an entire method using Kodi and again this is going to use the context in your code base so it's not just looking at your current file it's not just looking at your open tabs it's actually searching across your code base for different examples that can use to complete this method and so over here on the right you can actually see it's explaining what's going on in this file it's explaining how Cody is generating a unit test and fetching stuff from our source graph API and feeding that into the language model as context so I did have a longer talk for other conferences but we're limited on time so not going to get into that we're actually at that table right there so if you want to come and stop by afterwards I'll be there come with all your questions [Applause]