Full-Stack AI Engineering from AI-Native Coding to Knowledge, Models, Infrastructure & OSS
hello everybody I'm Alexi kov the founder and organizer of a by the bay which enters its 11 year yeah by the way has always been an independent technical conference based on the community open source and its heart is in the meetups which we also have going in the Bay Area for more than 10 years Bay Area AI is the longest running deepest technical meup in the world on thei which went through NLP text mining data mining knowledge disc covering databases machine learning deep learning Ai and generative Ai and our tracks reflect the engineering core of the conference and the community a has always been for us the question of implementation in practice the question of reality the question of production the question of startup success so we Bas on software engineering and software engineering is not going away software engineering is transforming in our view AI engineering become software engineer for AI you engineer your software you may use different tools you may use different flows you now use co-pilots special Ides agents but at heart you are an engineer you are engineering the system and software engineering remains a discipline so in our view this field migrates closer to the best practices of software engineering such as types such as riger and our keynote on pantic by its founder reflects our vision this is the first keynote which we announce so the first track is thoughtful and AI native coding it used to be thoughtful software engineering and we focused on strongly up languages all kind of patterns Ty programming meta programming uh then continuous delivery different ID features different ways to increase developer productivity and enhance developer experience and definitely we're fans of advanced programming languages and it all Remains the Same the developer experience is now influenced by co-pilots you know codium vs code cursor you name it there are different kind of tools and there are tools like repet which aspire to build an application for you however you have to remember that you still have to be in charge and you have to still deeply understand what's going on one of the experience I had was one of the tools by one major companies I will not name it but they presented effectively magical AI completion of your thoughts and you become a click engineer I wouldn't call it AI engineer and the click engineer just clicks and accepts and clicks and accepts and there is a program and then you want to run tests the slight problem in that specific setup was that all the the tabs and the environment appearing or disappearing according to the context did not work as expected so the test flow disappeared by the time the developer wanted to test the app and there was nothing to click on and so that was a moment of truth because now you're supposed to click test and the thing is not there what do you do how did it appear why did it disappear what do you do to make it appear basically it stalls you so there is a whole bunch of internal Machinery going on and unless you understand what it's doing unless you really understand what the code is doing uh and unless you understand what your ID is doing you are not going to complete it so I don't think you know um yeah is going to take over in that sense uh it's human Centric you have to understand you have to be in charge for all kind of reasons right maintainability compliance uh actual observability serving humans in the end so that track will explore all of this from the rigorous standpoint not of human theop of human in charge that's the thoughtful nii native coding and we're welcoming all the cool and interesting programming language features for instance we're very excited about the rust uh tooling and guarding python edifice with UV and pantic and so many others we're very excited about Zig you know we are welcoming our friends from zml running super fast inference on zig and definitely welcome other typed rigorous type checking tools which help you write correct code ensure that code is observable debuggable and efficient that's that track uh the next track is knowledge data and models right and so here we take the view that llm is your run time effectively right it's compresses the world into Rich knowledge but there is high uh indeterminism uncertainty and barely relying on this as it is on the Royal Alm is not possible most of the times so instead of kind of building layers upon layers of interaction wrapping we really want to unwrap it we want to understand what's going on and we want to to actually put humans again in charge one view is that LMS are very good at compression of the world and indexing and reasoning is best in reason engine we have now LMS that reason again the way the reason you can see it you can maybe influence it but we don't have full control over how this is happening uh an encouraging development is uh agenic architecture with AI memory cognitive core where LM is quired the knowledge is extracted validated decorated enriched and put in some kind of Knowledge Management System which can be a Knowledge Graph and the reason engine helps to work on it and so graph rack pattern is one of the examples of this and uh we are very excited in the growth of thiso es multiple startups are appearing attacking from different directions and uh we're welcoming uh uh talks about data sets about models trained on these data sets with the eye of maintaining and extracting actionable knowledge so this whole track will be very interesting amalgamation and keep the end in mind what is the knowledge your business is good at your business is good because you know know something better than others maybe about making Rockets maybe about making electric cars or maybe making beautiful apps or better accounting and so forth you're good at something and so when you build a apps for your business you want to translate your knowledge of the world of the customers of your domain into your applications it's very important that you do it in such a way that it's observable debuggable explainable and can be subject to compliance which is coming uh so that's the track uh the next track is the I data infrastructure again this is emerging is one of the core topics in order to make the EI a reality you need to run effectively every app as an a enabled app and we at our conference had these talks for literally over a decade uh Carlos gestrin gave a talk where he said that every app will be an AI app I think around 2015 and now it's really becoming the reality it took a while we have the tools we have the means we have geni which can subject any domain to Ai and uh the real question is and when an app becomes an AI app it needs to run a inference so all this models trained extreme expense once now have to be deployed millions and millions of times and they have to be run so the question of Cheaper training does not obviate the need for massive inference so we don't see a infrastructure slowing down we don't see deeps making massive data centers unnecessary on the contrary the availability of dsek makes it even easier to make every AI every application enabled and now we need more we need more gpus tpus accelerators uh and we we need more of that deployed everywhere so yeah inference is the bottleneck and major companies are bringing down the cost of inference but also increasing the scale of inference and this track will look at data infrastructure in all respects including inference including data management including data provenance right the governance uh of data flows through your app are extremely important so the click throughs we see with gdpr will probably be something even more elaborate and in the back end you will need to be able to explain you need to be able to work closely with the models to understand where your dat is coming from and obviously a lot of this overlaps with previous tracks and we make no distinction in kind of no beef if some top is fitting different tracks but it will find the best match but you understand the spirit the spirit of this is AI data infrastructure as an area of development and all the envelopes AI Ops devops devops all these efforts are going here and finally open source AI so this is the conference of the mops by AI data for after party and Spark and Friends JF AI AWS Ai and so forth have huge amount of friends who ran these meetups also for a very long time and um have big analytics for instance one of the biggest Mups has Al also always been a part of this so we're partnering for all the Mups we're bringing in all the communities in this capital of geni World San Francisco B area uh and open source is the heart of this right startups scientists turned open source first uh meta turn to Lama powered this whole generation of Open Source Ai and I think at this point with deep seek we can say that AI as open source is the way to go open Source a is winning an open source a will win and it will win in multiple contexts it will win in Industry where you need to have on Prem compact observable limited agents doing you specifically with job design governments ftech everywhere that you care about latency efficiency ownership uh open source will win so we betting open source EI and open source EI means an ecosystem it means Stacks that are interoperable so if you're Building aurus Product think of where do you fit on the Spectrum we'll try to present a view of Open Source a one of these projects is called Oaks open source a knowledge stack and this is something that newj and partners uh such as zillis andt XT and others cogy uh are considering and conceptualizing and many others Opia from L Foundation is a very good example uh think of the way you interact with other open source Builders and the best talks here will basically explore how can we get out of the box open source gen applications built how can we adopt multiple existing leading tools L chain and llama index and so forth and build productive AI applications so these are the for tracks the cfp is open please submit a talk and we hope to see you by the way in November