Honestly, this depends a lot on what you're doing with the server, and what your expectations are.
If you're building a box that can handle short periods of (scheduled) downtime without problem (updates/upgrades) and isn't running a mission critical service, running Ubuntu's latest stable is going to be fine. Just remember that it will require updates if it's going to remain in production for a length of time, and plan appropriately.
If you're building a box that needs maximum uptime, or if you plan to run commercial software that might offer limited compatibility between versions, then an LTS release is going to be your best bet. This will allow you to run longer (with security updates available) before you have to upgrade or retire the box. Also, as mentioned, many commercial software companies won't support all releases, but will support LTS to simplify things on their side.
Now, as this is for a server, I would tend to lean more towards the LTS release. Server hardware rarely changes extensively once it's in production, and it's easier to verify that hardware before putting it into use. Additionally, on the server side 64 bit support is quite good (in my experience) and very stable. I doubt you'll have a need to upgrade frequently just for better 64 bit support.