That is a very broad question - and there probably is no "one fits all" answer.
I personally tend to leave most them alone in fall for four reasons:
- decorative purposes
Many "spent" plant parts look spectacular when covered in frost and snow and give texture to borders that would otherwise look plain and empty. Some gardeners prefer the "clean" look, though.
- frost protection
A layer of leaves and stalks can act as protection during the winter, both during dry frost and by "holding onto" light snow that might be blown away otherwise.
- ecological reasons
The leaves and stalks are a place to overwinter for insects and other small creatures.
- sheer laziness
In fall, I am usually quite busy, so I tend to do only the bare minimum. Come spring, some material will have already mostly disintegrated without the need to place it on the compost heap and back on the beds. And the spring sun draws me out into the garden where I gradually make room for new growth.
This concept has its limits, though: do not leave plant material with diseases but remove it promptly. I also cut back what threatens to smother other plants or looks plain ugly. But that is an ongoing task through the growing season, not specifically a winter preparation.