Power is the obvious one, and as it has been touched on in other answers but not directly. Performance. Faster clock does not mean faster code. ST has this magic cachy thing in front of the flash in addition to a real-ish cache in front of the flash (and disabled the arm cache it appears on the cortex-m4's I have tried). But in general the flash is your bottleneck, if as you see on a number of other vendor parts and sometimes ST, you have to keep adding wait states as you increase the system clock. So say at 16Mhz no wait states, at 32, one, 48 two and so on, depends on the system, you are dancing around the speed limit of the flash, making the processor sit around extra clocks while it waits for instructions to come in. And even on an st but easier to see on others, that directly affects your performance, you want to be perhaps right at the frequency where you go from zero to one wait states to maximize what you can feed the cpu.
Some designs the flash is already at half speed of the cpu/system clock where sram generally tracks and can cover the full range, so take the same code at zero wait states and run from flash then run from ram, on some number of MCUs the same machine code runs half speed on flash as it does on sram. some it is one to one and then when you add a wait state flash is half speed vs ram, and so on.
Peripherals have the same problem as mentioned. You may have to use a prescaler on the peripheral clock, so now reading a gpio pin that at 24mhz might have taken one clock at 80mhz may take three or four clocks.
PLLs are analog and jittery, they dont necessarily "lose" any clock cycles, but are worse as far as jitter as the oscillator which itself has a spec on jitter and accuracy as well as temperature affects. So the internal RC is the worst quality clock, direct from the oscillator bypassing the PLL is the best, then multiplying with the PLL will add jitter, but will allow you to go faster.
Back to power. The battery in your remote control on your TV might last a year or so (infra red, the bluetooth ones are days or weeks), they run the lowest clock rate they can to barely do the job and stay off as much as possible or in low power state. If they were to hop up to 120Mhz when they were awake and the battery now lasts weeks or half a year, vs a year, for no real benefit other than it is really cool to run at full speed. That doesnt make much sense. We heavily rely on battery based products now, if the microcontroller in the bluetooth module on your phone ran at its fully rated pll speed, and the microcontroller in your wifi module in your phone, and the one that drives the display, etc, all ran at max speed, your phone might not last even a day on one full charge. Nothing was gained by running faster, but something was lost.
For hobbies and stuff plugged into the wall, burn whatever power you want, but a noticable percentage of the mcu market is about price and power, chips that are screened to a higher speed cost more, lower speed parts, in some cases are just the chips that failed the higher screen, and cost less. tighter smaller code uses less flash you can buy a smaller/cheaper part, your clock can run slower because it takes fewer instructions to do the same thing than a possibly sloppy program and/or bad choice in programming languages, bigger part, lower yield both add to cost. then lowering the clock as low as you can go to keep your tightly written code to just barely meet timing uses the least amount of power ideally (As well as turning off or not turning on peripherals you are not using and prescaling the ones that are on even slower).
For cost and power you want the slowest clock you can tolerate with the smallest binary that is also tight and efficient so that you can just barely make timing. That is your ideal goal. But if you plan for field upgrades then you need to leave some margin for slower/larger code to be part of the upgrade and not have a dramatic effect on power consumption.
