I'm trying to speed up tests and so far have done things like remove unnecessary database writes (using new instead of create and not using factorygirl, less feature tests, combining tests) and when I run the files individually I get a reduction in time.
However, when I run the entire test suite I end up getting roughly the same times, and sometimes I end up even getting a slower run time on my improved tests.
Any way to accurately measure this?
Swapping out new instead of create with FactoryGirl is throwing the baby out with the bathwater. If you don't need persisted objects you can use FactoryGirl's build or build_stubbed and you will get the performance savings while still maintaining the parameter defaulting, etc. Beyond that all the performance tuning you're doing really is swamped by the actual time it takes to do things in a browser, and asset compilation (especially on smaller numbers of tests) so you really won't save that much. If performance is what you care about your best bet is to test as much as possible as in unit tests and then limit most of your feature tests to the happy path through your app.
If you still want to find the slowest tests to make sure you aren't doing anything like waiting for "not elements to exist" rather than "elements to not exist" and optimize them, you can specify the --profile command line option to rspec or specify 'profile_examples' in your RSpec configuration https://relishapp.com/rspec/rspec-core/docs/configuration/profile-examples
