The following table gives a short overview of the main HPC systems at RRZE and the development of number of cores, clock frequency, peak performance, etc.:

	original Woody (w0xxx)	w10xx	w11xx	w12xx + w13xx	LiMa	Emmy	Meggie
Year of installation	Q1/2007	Q1/2012	Q4/2013	Q1/2016 + Q4/2016	Q4/2010	Q4/2013	Q4/2016
total number of compute nodes	222	40	72	8 + 56	500	560	728
total number of cores	888	160	288	32 + 224	6000	11200	14560
double precision peak performance of the complete system	10 TFlop/s	4.5 TFlop/s	15 TFlop/s	1.8 + 12.5 TFlop/s	63 TFlop/s	197 TFlop/s	~0.5 PFlop/s (assuming the non-AVX base frequency as AVX turbo frequency)
increase of peak performance of the complete system compared to Woody	1.0	0.4	0.8	0.2 + 1.25	6.3	20	50
max. power consumption of compute nodes and interconnect	100 kW	7 kW	10 kW	2 + 6,5 kW	200 kW	225 kW	~200 kW
Intel CPU generation	Woodcrest	SandyBridge (E3-1280)	Haswell (E3-1240 v3)	Skylake (E3-1240 v5)	Westmere-EP (X5650)	IvyBridge-EP (E5-2660 v2)	Intel Broadwell EP (E5-2630 v4)
base clock frequency	3.0 GHz	3.5 GHz	3.4 GHz	3.5 GHz	2.66 GHz	2.2 GHz	2.2 GHz
number of sockets per node	2	1	1	1	2	2	2
number of (physical) cores per node	4	4	4	4	12	20	20
SIMD vector lenth	128 bit (SSE)	256 bit (AVX)	256 bit (AVX+FMA)	256 bit (AVX+FMA)	128 bit (SSE)	256 bit (AVX)	256 bit (AVX+FMA)
maximum single precision peak performance per node	96 GFlop/s	224 GFlop/s	435 GFlop/s	448 GFlop/s	255 GFlop/s	704 GFlop/s	1408 GFlop/s
peak performance per node compared to Woody	1.0	2.3	4.5	4.5	2.6	7.3	14.7
single precision peak performance of serial, non-vectorized code	6 GFlop/s	7.0 GFlop/s	6.8 GFlop/s	7.0 GFlop/s	5.3 GFlop/s	4.4 GFlop/s	4.4 GFlop/s
performance for unoptimized serial code compared to Woody	1.0	1.17	1.13	1.17	0.88	0.73	0.73
main memory per node	8 GB	8 GB	8 GB	16 GB / 32 GB	24 GB	64 GB	64 GB
memory bandwidth per node	6.4 GB/s	20 GB/s	20 GB/s	25 GB/s	40 GB/s	80 GB/s	100 GB/s
memory bandwidth compared to Woody	1.0	3.1	3.1	3.9	6.2	13	15.6

If one only looks at the increase of peak performance of the complete Emmy systems only, the world is bright: 20x increase in 6 years. Not bad.

However, if one has an unoptimized (i.e. non-vectorized) serial code which is compute bound, the speed on the latest system coming in 2013 will only be 73% of the one bought in 2007! The unoptimized (i.e. non-vectorized) serial code can neither benefit from the wider SIMD units nor from the increased number of cores per node but suffers from the decreased clock frequency.

But also optimized parallel codes are challenged: the degree of parallelism increased from Woody to Emmy by a factor of 25. You remember Amdahl’s law (link to Wikipedia) for strong scaling? To scale up to 20 parallel processes, it is enough if 95% of the runtime can be executed in parallel, i.e. 5% can remain non-parallel. To scale up to 11200 processes, less then 0.01% must be executed non-parallel and there must be no other overhead, e.g. due to communication or HALO exchange!