Because of the massive scale of WSCs, it is very important to properly allocate network resources based on the workloads that are expected to be run. Different allocations can have significant impacts on both the performance and total cost of ownership.

a. Using the numbers in the spreadsheet detailed in Figure 6.13, what is the oversubscription ratio at each access-layer switch? What is the impact on TCO if the oversubscription ratio is cut in half? What if it is doubled?

b. Reducing the oversubscription ratio can potentially improve the performance if a workload is network-limited. Assume a MapReduce job that uses 120 servers and reads 5 TB of data. Assume the same ratio of read/intermediate/ output data as in Figure 6.2, Sep-09, and use Figure 6.6 to define the bandwidths of the memory hierarchy. For data reading, assume that 50% of data is read from remote disks; of that, 80% is read from within the rack and 20% is read from within the array. For intermediate data and output data, assume that 30% of the data uses remote disks; of that, 90% is within the rack and 10% is within the array. What is the overall performance improvement when reducing the oversubscription ratio by half? What is the performance if it is doubled? Calculate the TCO in each case.

c. We are seeing the trend to more cores per system. We are also seeing the increasing adoption of optical communication (with potentially higher bandwidth and improved energy efficiency). How do you think these and other emerging technology trends will affect the design of future WSCs?

Transcribed Image Text:

Size of facility (critical load watts) 8,000,000 Average power usage (%) 80% Power usage effectiveness 1.45 Cost of power ($/kwh) $0.07 % Power and cooling infrastructure (% of total facility cost) 82% CAPEX for facility (not including IT equipment) $88,000,000 Number of servers 45,978 $1450 Cost/server CAPEX for servers $66,700,000 Number of rack switches 1150 Cost/rack switch $4800 Number of array switches 22 Cost/array switch Number of layer 3 switches $300,000 $500,000 Cost/layer 3 switch Number of border routers $144,800 Cost/border router CAPEX for networking gear Total CAPEX for WSC $12,810,000 $167,510,000 3 years 4 years 10 years Server amortization time Networking amortization time Facilities amortization time Annual cost of money 5% Figure 6.13 Aug-04 Mar-06 Sep-07 Sep-09 Number of MapReduce jobs 29,000 171,000 2,217,000 3,467,000 Average completion time (seconds) 634 874 395 475 Server years used Input data read (terabytes) Intermediate data (terabytes) Output data written (terabytes) 217 2002 11,081 25,562 3288 52,254 403,152 544,130 758 6743 34,774 90,120 193 2970 14,018 57,520 Average number of servers per job 157 268 394 488 Figure 6.2