Performance optimization multipliers

Question

Can anyone provide an example of a system in which a performance improvement of x% to a single sub-component of the system results in an overall performance increase of Ax% (where A>1) for the entire system? In other words, can there be a positive multiplier effect for localized optimizations in a system?

I could have sworn that I have encountered such performance multiplier effects in the past, but for the life of me, I cannot construct even a hypothetical example of such a case.

That's not theoretically possible. If you get say 20% improvement in something that takes 50% of overall time, the entire process will only improve 10%. — Eric J., Jul 18 '12 at 21:32
@Eric, I think I may have gone off into the weeds on this question. I must have been thinking of a situation in which there was unnecessary blocking in a parallel system and elimination of the blocking resulted in substantial performance improvement. I just don't think I could accurately characterize the elimination of that blocking as an "x% performance improvement in a sub-component." — Dan Hermann, Jul 18 '12 at 21:38

score 2 · Accepted Answer · edited Apr 13 '17 at 12:53

The only way I know of to get a multiplier effect in software performance (without different hardware) is to exploit Amdahl's Law which just says if you make something take less time it's faster. (Boy, that's pretty deep.) If you reduce the time the software takes by a fraction X, then you have given it a speedup ratio of R = 1/(1-X). For example, if a program takes 100 seconds, and you manage to shave off 60, then you're left with it taking 40 seconds, which is 100/40 = 2.5 times as fast as it was.

Here's how X and R are related:

X    R
0.0  1
0.1  1.11..
0.2  1.25
0.3  1.43..
0.4  1.66..
0.5  2
0.6  2.5
0.7  3.33..
0.8  5
0.9  10
0.99 100
1.0  inf

Also, these effects compound. If you reduce time by 50%, the speed is multiplied by 2. If you take the result of that, and reduce its time by 50%, the speed is multiplied again by 2.

Notice that the second 50% was only 25% of the original time. The first time reduction made the second one bigger, by a factor of 2 !

Here's an example where that speed-compounding magnification effect produced a speedup of 730 times.

Thanks, this is what I was looking for. I just covered Amdahl's Law for a parallel design, so I should have remembered that. — Dan Hermann, Jul 19 '12 at 15:16

score 1 · Answer 2 · answered Jul 18 '12 at 21:55

The only example I can think of is in a system with some kind of hard real-time constraint, e.g. sub-component takes too long to execute, which results in a time-out and a re-try in some sort of I/O interface. Increasing the performance of the sub-component makes it fast enough to avoid the time-out + retry, thereby increasing overall performance significantly. Real-world examples that spring to mind are old skool disk controllers (before the days of cache etc) and network interfaces (time-outs result in packets being re-sent etc).

Performance optimization multipliers

2 Answers2