These words are due to my actual experience with that problem.
You have to use the support library, so some objects are renamed. And also some methods.
Preferences are absolutely left out from the support library (so, you have to use a PreferenceActivity for older versions and a PreferenceFragment for newer ones).
Take in account that you'll be missing some new functionalities.
And the system interfaces vary a lot from OS to OS (so, you'd better provide your cutom theme to overcome big differences in the overall aspect).
And often you'll find yourself comparing the Build Version to that of your running OS to execute this or that method (or to use a new functionality or not).
It's kind of a pain, but it's worth, because you will have more potential users.