Topic 6 · Class activity 1 · Week 15 · Wednesday

Benchmark Your Compiler

Three benchmarks, four metrics, three repeated runs, one Amdahl calculation — and a number you could defend to a sceptic.

Format pairsTime 50 minNeed SPIM, your compiler, and your machine's specifications

The point

Produce the performance section of your Benchmark submission, with numbers you could defend to somebody who wanted to disagree with them.

Do this

  1. Pick three benchmarks of different shapes: one straight-line arithmetic program, one with a hot nested loop, one call-heavy (recursion is ideal). Each should run long enough to give a six-figure instruction count.
  2. Compilation time. Record the per-phase table for each. Then make a ten-times-longer version of one benchmark and record it again. Which phase grew fastest? Is anything super-linear? Plot it if you can — three points is enough to see a curve.
  3. Apply Amdahl's law. From your profile, which phase would you optimize first, and what is the maximum improvement available even if you made it free?
  4. Code size. Record TAC instructions before and after optimization, and the MIPS instruction count, for all three benchmarks.
  5. Work done. Run each in SPIM, optimized and unoptimized, and record instructions executed. To produce an unoptimized .s, point the back end at the unoptimized TAC list.
  6. Repeat everything three times and record the spread.
  7. Write four sentences: what you measured, what you compared it against, what the numbers show, and where the two metrics disagree and why.
Where to look when a number surprises you

The register allocator reports spills and reloads. A benchmark with far more executed instructions than you expected usually has a spill inside the hot loop — which is a real finding, traceable to the missing liveness analysis, and exactly the sort of thing that belongs in your write-up.

Deliverable

A performance table covering three benchmarks with all four metrics; the growth analysis and your Amdahl calculation; the repeated-run spread; the benchmark sources; and your four sentences. This is the performance section of Project 6 — write it once, properly.