scala.bythebay.io: Davis Shepherd, Building a Machine Learning Orchestration Framework in Scala

ai.bythebay.io Nov 2025, Oakland, full-stack AI conference Scale By the Bay 2019 is held on November 13-15 in sunny Oakland, California, on the shores of Lake Merritt: https://scale.bythebay.io. Join us! ----- A scaleable and adaptable machine learning platform is essential for an organization to harness the full potential of their data. This talk introduces Meson, a General Purpose Workflow Orchestration and Scheduling framework. Meson is a Scala application that manages heterogeneous workloads integrating Spark, Python, R, Docker and other toolkits. We will demonstrate how a simple user interface and a DSL allows data scientists to design and run machine learning pipelines that seamlessly exchange data and artifacts between disparate jobs. We are working on open sourcing Meson and by this talk it may very well be open sourced. Davis Shepherd Senior Software Engineer, Netflix I started working with Scala when I was writing applications on top of Apache Spark. I spent 4 years building re-enforcement learning systems with these tools. I now work at Netflix, developing their next generation of ML workflow management software all in Scala