sfspark.org: Alex Oleinikov, Practical insights on monadic error handling in Spark

ai.bythebay.io Nov 2025, Oakland, full-stack AI conference Scale By the Bay 2019 is held on November 13-15 in sunny Oakland, California, on the shores of Lake Merritt: https://scale.bythebay.io. Join us! ----- Practical insights on monadic error handling in Spark Alex Oleinikov, Criteo Error handling in distributed systems is tough, but once done right it can make data pipeline's beavior predictable and incident investigations will no longer be a puzzle. The talk covers a monadic approach to error handling, tradeoffs of it's implementation in Spark, advantages, related technical difficulties and ways to deal with them. Alex works on data collection and knowledge extraction for the last 6 years. He developed media information knowledge base which powers the Media vertical at Yandex. There he also worked on consumer facing web services, their performance and scalability aspects. Lately he’s busy bringing the miracle of machine learning to search ads camapaign management at Criteo.