Because this is a vast subject, this answer is a community wiki, feel free to improve this answer
Website loading speed depends of many factor : language, framework, DB, CPU, RAM, caching, ...
By the way, the whole thing depends of the worse element. I don't think that Ruby or Rails are the limiting factors here.
To start you can see if Spree is a good framework. Here is a benchmark of Spree in different hosting and comparing to a pure Rails app : http://blog.endpoint.com/2011/05/spree-performance-benchmarking.html
As you can see, this go from 0.5s to 6s ! You can notice that Spree version inpacts a lot on performance. Unfortunately last versions are not on the benchmark... You should try to find an up to date benchmark. You can also notice that a raw rails app is, at first sight, faster.
Next you have to review your hardware : what is your server ? Dedicated or not ? How many CPU and RAM ? Can you increase ?
I think DB is also a thing to watch for a e-commerce case, because I can imagine you have a lot of products. So you can test different DB engines, and see in Spree community what is the recommended one.
Next you can interest you in caching : http://guides.rubyonrails.org/caching_with_rails.html
To go further you can try some memcached store for your putting your cache in RAM : http://guides.rubyonrails.org/caching_with_rails.html#activesupport-cache-memcachestore