Keynote III: Data Science at Scale with...

Apache Spark has been blessed as the replacement for MapReduce in Hadoop environments. It also runs in other deployment modes. Spark provides better performance, better user productivity, and it supports a wider range of application scenarios than MapReduce, including event stream processing, ad hoc queries, graph representations and algorithms, and iterative algorithms, such as those commonly used in machine learning. This talk discusses Spark from a Data Science perspective, it's strengths and weaknesses, the Scala, as well as Java, Python, and R APIs it offers for common analytics problems, what's missing, and what's planned. We'll look at support for ad hoc queries over large data sets, stream processing, machine learning algorithms, graph processing, and the user experience.