Building Real-time Data Pipelines Made Easy with Structured Streaming in Apache Spark

Data Council DEC-SF-18 session walking through a concrete streaming ETL example reading raw Kafka data and writing a structured table in under 10 lines, plus event-time aggregations, stream-stream joins and arbitrary stateful operations. Original datacouncil.ai talk page now 404s; the recording is distributed via Databricks' Higher Signal podcast feed.