TechCast #102 - Sameer Farooqui and Brian Clapper on Spark

Ken Rimple interviews Sameer Farooqui and Brian Clapper of Databricks about Apache Spark: how RDDs work, how Spark's optimizer keeps data in memory, and Spark's unified API across SQL, streaming, ML and graph processing. They discuss when batch intervals suffice for streaming and an application analyzing high-throughput zebrafish brain imagery.