Transformations and Actions
Lazy computation.
Two Kinds of Operations
Spark operations split into transformations (build a new dataset, lazy) and actions (trigger computation and return a result or write output).
Lazy Evaluation
Transformations are lazy: they record what to do but run nothing. Spark builds a directed acyclic graph (DAG) of transformations and only executes when an action is called.
All lessons in this course
- RDDs and DataFrames
- Transformations and Actions
- Spark SQL
- Aggregations