In Section 1.6.4 of our textbook, we are introduced to a number of benchmarking programs that have been used over the years to compare the performance of computer systems. Among these are the classic Whetstones, Dhrystones, Livermore Loops, and LINPACK benchmarks as well as newer test suites such as SPEC CPU2006.

SPEC charges a fee for use of their benchmark code, but source code for the older benchmarks mentioned above is widely available, both in the original languages (FORTRAN in most cases) as well as translations into C, C++ and other more modern languages. One site with a good variety of publicdomain benchmarks, including source code and precompiled versions for Intel/AMD PCcompatible platforms, is Roy Longbottom's PC Benchmark Collection. In order to get some "handson" perspective about benchmarking, we will run some of the programs available on his site and examine the results we get.

To get started, go to the following link and download BenchNT which is a zipped file containing source code and precompiled (PC executable) versions of four classic benchmark programs mentioned in our text. Unzip the files to a directory on a machine you wish to test.

There are several ways you can use the programs you just downloaded, but probably the easiest thing to do is to open up a command prompt, change to the BENCHNT directory in which the executable programs reside, and execute the RUNALL.BAT file. This will run the complete set of benchmarks on your machine, which will probably take several minutes. At the end, you will be prompted to enter certain information about the system, including the system manufacturer and model, the CPU type and clock frequency, the amount of onchip cache memory, the amount and type of system RAM, the size and type of hard disk(s), the make and model of the graphics card and how much RAM it has, the OS in use, and so on. All of this information will be saved into the BENCHSYS.TXT file along with the benchmark results so that you have documentation of the test conditions. (You will probably want to gather all the relevant information about the system under test before actually executing RUNALL.) When the collection of benchmarks finishes running, be sure to save the BENCHSYS.TXT file containing all the collected data.

Assignment: Run the complete BenchNT suite on three different PCcompatible computer systems. To the extent possible, pick systems that have distinctly different characteristics. For example, you might choose a notebook computer, a barebones desktop, and a highpowered, multicore gaming machine (if you have access to one). To be "fair" to each system, exit all other applications such that the maximum amount of RAM and CPU time is available to run the benchmarks. Save the BENCHSYS.TXT file produced in each case.

Prepare a report (in Word or PDF format) in which you analyze the results of the benchmarking tests run on the three systems. In your report, you should describe the nature of each benchmark and explain what aspect(s) of system performance it is designed to measure. Describe each system's hardware and software configuration and display comparative results in tabular and/or graphical format for ease of comprehension. (You can use Excel or other tools to create tables and graphs from the results of interest.)

Give your best explanation of performance differences between the systems, as well as differences between various benchmark results for each system individually. (For example, how great are the differences between "optimized" and "nonoptimized" versions of the same benchmark running on the 2/2 same system and, if the differences are considerable, what were the nature of the optimizations that made such a difference?) Take note of, and attempt to explain, any particularly interesting phenomena or differences in the results for example, if system A outperforms system B on Whetstones but the reverse is true when running Dhrystones, what attributes of their system hardware, OS, etc. might explain this?

Extra credit option: In addition to the three PCcompatible systems, recompile the source code for one or more of the benchmark programs to run on a nonIntel architecture platform, for example a SPARCor MIPSbased machine. Run the benchmark(s) on this other machine and include the results (and analysis thereof) in your report. The amount of extra credit given will depend on how many of the benchmarks you ported to the nonIntel machine as well as (of course) the quality of your analysis of the results. Submit your report via UTC Learn (Blackboard). For full credit, make sure you submit it by the due date/time given above.

NOTE: BENCHSYS.TXT will be generated when you run the code.so, you just need to save that text file.