The question this section answers: Why is my Spark job slow, and how do I find out?

Nearly every slow Spark job comes down to moving too much data, moving it unevenly, or joining it the wrong way. Start by opening the Spark UI and looking at the slowest stage (a group of tasks that run without moving data between machines). Then read these in order:

Scale assumption: the advice here is aimed at jobs that process tens of GB and up. On small data, none of this matters much.