If you want to use PowerPivot you have to install Excel. PowerPivot is an Excel add-in.
Now, SSDT only comes in a 32-bit version, there is no 64-bit version to install. Even if it were available you'd probably wouldn't see any performance improvement in your case because the data model is not actually "processed" by SSDT. You build in SSDT and then deploy to a SSAS server in tabular mode. If the model you build is slow it could be because a lot of reasons, one of them being that the SSAS server your deploying to is running on your local machine.
If you want to move your data model from SSAS to PowerPivot, try this http://blog.gbrueckl.at/2014/05/restoring-a-ssas-tabular-model-to-power-pivot/.
It involves taking a backup of your tabular model and then restore that in PowerPivot. It's worked for me in the past.