Description

The purpose of this project is to develop a MapReduce program on Hadoop to multiply two sparse matrices.

Project Description

For this project, you are asked to implement matrix multiplication in Map$-$Reduce. You should modify Multiply. java only. In your Java main program, args $[0]$ is the first input matrix M$,$ args $[1]$ is the second input matrix N$,$ args $[2]$ is the directory name to pass the intermediate results from the first Map$-$Reduce job to the second, and args $[3]$ is the output directory. All the input and output file formats must be text formats. There are two small sparse matrices in the files M$-$matrix$-$small. txt and N$-$matrix$-$small. txt for testing in standalone mode. Their matrix multiplication must return the result matrix in solution$-$small.txt$.$

Pseudo$-$Code

class Elem extends Writable $($

shot tex; one othe fodexes $($the other is used as a key$)$

double value;

$...$

$\}$

class

Pair extends WritableComparable $($ int i; int j;

$...$

$\}$

First Map$-$Reduce job: map $($key$,$ line$)=$

$//$ mapper for matrix M

split line into $3$ values: i$,$ j$,$ and v emit $($j$,$ new Elem $(0,$ i$,$ v$))$

First Map$-$Reduce job: map $($key$,$ line$)=$

$//$ mapper for matrix M

split line into $3$ values: i$,$ j$,$ and v emit $($j$,$ new Elem $(0,$ i$,$ v$))$

map $($key$,$ line$)=$

$//$ mapper for matrix N

split line into $3$ values: i$,$ j$,$ and v emit $($i$,$ new Elem $(1,$ j$,$v$))$

reduce $($index$,$ values$)=$

A $=$ all v in values with v$.$tag$==0$

B $=$ all v in values with v$.$tag$==1$

for a in A

for b in B

emit $($new Pair $($a$.$ index, b$.$ index$),$a$.$ value$*$b$.$value$)$

Second Map$-$Reduce job:

map $($key$,$ value$)=//$ do nothing

emit $($key$,$ value$)$

reduce $($pair$,$ values$)=//$ do the summation

m $=0$

for v in values

m $=$ m$+$v

emit $($pair$,$ m$)$