DSPy and Graphs: Making AI Real
Bay.Area.AI meetup at GitHub, September 12, 2024 We had a great meetup following the AI Conference. Neo4j and the AI Alliance were top sponsors, and lots of my colleagues from both were there. I was happy to man the Neo4j booth as the new AI Community Architect. All photos from the meetup: https://www.f.photo/Meetups/Bay-Area-AI/Bay-Area-AI-20240912 The meetup theme was rigorous OSS AI engineering, beyond the prompting. Our first talk was on DSPy, a framework for programming, not prompting,
Bay.Area.AI meetup at GitHub, September 12, 2024
We had a great meetup following the AI Conference. Neo4j and the AI Alliance were top sponsors, and lots of my colleagues from both were there. I was happy to man the Neo4j booth as the new AI Community Architect.
All photos from the meetup:
https://www.f.photo/Meetups/Bay-Area-AI/Bay-Area-AI-20240912
The meetup theme was rigorous OSS AI engineering, beyond the prompting. Our first talk was on DSPy, a framework for programming, not prompting, LLMs, our of Stanford. I’ve discovered it very early, as I did with Spark and Kafka. Omar Khattab was a graduate student at Stanford, studying with Matei Zaharia and Chris Potts. (In 2012, I’ve created the very first Spark meetup with Matei, that became Bay Area Spark.) I’ve been immediately energized by the promise of a systematic approach to LLM query optimization as a whole system process. I’ve attended Omar’s talk at Berkeley and invited him to Scale By the Bay, where he presented DSPy in 2023. It followed the tradition of Matei keynoting and his lab presenting projects like Weld.
Now Omar has recommended Michael Ryan, a master’s student at Stanford and the author of the state of the art DSPy optimizer MIPRO2.
For the second talk, I’ve presented GraphRAG. This has been my first technical presentation at the meetup I founded and organized for 10 years, the biggest, longest, baddest, deepest, most technical AI meetup in the Bay Area and the world!
The RAG+GraphRAG theme was continued by Roie Schwaber-Cohen of Pinecone and Jiang Chen from Zilliz. The are seasoned devrel leaders and the talks touched on several aspects of RAG and GraphRAG and their interaction. My takeaway is that Q&A systems will be agentic, or compound, performing a chain of thought set of queries both sequentially and in parallel, and iterating based on the responses.
We’ve also recorded speaker interviews with all the speakers.
DSPy: Prompt Optimization for LM Programs
Michael Ryan, Stanford
It has never been easier to build amazing LLM powered applications. Unfortunately engineering reliable and trustworthy LLMs remains challenging. Instead, practitioners should build LM Programs comprised of several composable calls to LLMs which can be rigorously tested, audited, and optimized like other software systems. In this talk I will introduce the idea of LM Programs in DSPy: The library for Programming — not Prompting LMs. I will demonstrate how the LM Program abstraction allows the creation of automatic optimizers for LM Programs which can optimize both the prompts and weights in an LM Program. I will conclude with an introduction to MIPROv2: our latest and highest performing prompt optimization algorithm for LM Programs.
Michael Ryan is a masters student at Stanford University working on optimization for Language Model Programs in DSPy and Personalizing Language Models. His work has been recognized with a Best Social Impact award at ACL 2024, and an honorable mention for outstanding paper at ACL 2023. Michael co-lead the creation of the MIPRO & MIPROv2 optimizers, DSPy’s most performant optimizers for Language Model Programs. His prior work has showcased unintended cultural and global biases expressed in popular LLMs. He is currently a research intern at Snowflake.
Talk
Interview
The second part is a series of talks on RAG and GraphRAG from Neo4j, Pinecone and Zilliz.
Graphs and AI: Making it Real
Alexy Khrabrov, AI Community Architect, Neo4j
GraphRAG is one of the most promising architectures for enterprise AI. In this talk, we’ll explore technical and community efforts required to make GenAI ready for production, with the focus on the most recent advances in GraphRAGw with LangChain and LlamaIndex integrations.
Dr. Alexy Khrabrov is the founder and organizer of bay.area.ai and the AI Community Architect at Neo4. He is also a founder of opensource.science at NumFOCUS and a cofounder/convener of thealliance.ai. Alexy founded and runs scale.bythebay.io, a conference of the Bay Area developer meetups, for ten years.
Talk
Vectors and Graphs - Better Together
Roie Schwaber-Cohen, Pinecone
This short talk will explore how graph database and vector databases can be made to work in tandem in agentic (and semi-agentic) ways to deliver unique ways to analyze complex, interconnected data sets.
Roie Schwaber-Cohen is a Staff Developer Advocate at Pinecone, specializing in AI and data-intensive applications. With nearly 20 years of experience in software engineering, Roie has expertise in full-stack development, microservices architectures and data intensive applications.
Talk
Interview
Improving RAG with Knowledge Graph and Milvus
Jiang Chen, Zilliz
The talk covers the techniques of Knowledge Engineering that improve RAG quality and shows how to offline extract knowledge graph and implement a comprehensive retrieval method by storing and searching knowledge embeddings in Milvus.
Jiang Chen is the Head of Ecosystem and Developer Relations at Zilliz, the company behind the open-source vector database Milvus. He had previously served as a tech lead and product manager at Google, where he led the development of web-scale semantic understanding and search indexing that powers innovative search products such as short video search. He has years of industry experience handling massive unstructured data and multi-modal content retrieval. Jiang holds a Master's degree in Computer Science from the University of Michigan.
Talk
Interview