Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

There are seven stages to a MapReduce job: - Job setup - Load Split - Map - Copy - Merge - Reduce - Write part

There are seven stages to a MapReduce job: -
Job setup - Load Split - Map - Copy - Merge - Reduce - Write part
Consider a scenario where there are 2,000 mapping tasks each with a duration of 10 seconds which yield 100 intermediate keys to the reducer tasks. Each mapping task requires 3 seconds to access its load split. The reducer tasks have a duration of 20 seconds. Each reducer requires 5 seconds to write information to HDFS. The job setup time is 50 seconds and copy/merge stages have a duration of 120 seconds. Assume the duration of the copy/merge stages does not depend on the number of reducers. Consider a cluster of 500 workers. You can assume that the ApplicationMaster is not on one of the workers (i.e., does not use one of the containers for itself).
Based on the information about the MapReduce job and the cluster described in the information above, what is the execution time of the load split and mapper tasks?

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image

Step: 3

blur-text-image

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Students also viewed these Databases questions