Problem

3.( Number of Mappers and Reducers)

Suppose we have a cluster

with 240 rack

servers, and each rack server has 4 cores.

We want to run WordCount.jar on a text file.

Suppose the

size of the

input file is 36,853 MB. The chunk size

in the HDFS

is 64MB.

Please

answer the following questions.

1.

How many mappers

will be

created when you run

a Hadoop program on this cluster?

Answer:

2

. Can we set the number of reducers to

0.9 * 240 * 4 =

864?

If so, how many reducers will run

simultaneously?

Answer:

3.

Can we set the number of reducers to

1.7 * 240 * 4 = 1632 ? If so, how

many reducers will run

simultaneously?

Answer:

4

. Describe the differences of the two strategies for setting the number of reducers? One is 864,

the other is 1,632. Which one is better and why?

Answer:

5

. When we set the number of reducers to 864,

how many partitions are there in the program?

How about 1,632 reducers?

Answer:

6

.

Suppose we use RandomPartitioner and we set the number of reducers to 864

. Then how

many partitions do we have?

Answer