sparklyr + Spark 2.0 + AWS

Background The new ‘sparklyr’ package provides a ‘dplyr’ backend to interacting with Spark. It also opens the ML Spark library, which in effect expands the functionality previously available to R via SparkR. I wanted to see what it would take to get a functional Spark 2.0 cluster to interact with this package. AWS/EC2 Setup Go…