Shuffling

$1$ point possible $($graded$)$

Which of the following are true about the shuffle operation in Spark?

You are encouraged to read more about shuffle here.

All Spark operations involve shuffling

Shuffling requires large amounts of network communication which makes the operation expensive

Shuffling is Spark's mechanism for re$-$distributing data across partitions

To avoid shuffling, we should think about ways to leverage existing partitions. For example, leveraging

partial aggregation to reduce data transfer

No communication between worker nodes is required for the shuffling operation