0

I have deployed two versions of our singlepage web app: one master (A) and one branch where are some changes which can affect somehow load time (B). The change is usually some new feature on front-end, refactoring, small performance optimization, etc. The difference is not so big and the load time varies much more from other reasons (a load of testing machines, a load of servers, network, etc). So webpagetest.org even with 9 tries varies much more (14-20s speedindex) than the real difference could be (0,5s in average for example).

Basically, I need one number which tells me - this feature increase/decrease load time.

Is there some tool which could measure such differences?

My idea was to deploy Webpagetest to a server with minimal load and run Webpagetest randomly on both versions at the same time so I avoid most of the noise. Make a lot of samples (1000+) and check average(or median) value.

But before I start working on that I would like to ask if there is some service which solves that problem.

user2530120
  • 31
  • 2
  • Do you know what metric you're tuning currently? Then try to measure exactly this with the minimal tools needed. For example when you're trying to minimize load time, measure how long a http(/2) request takes. If you're refactoring a javascript function, measure how long it runs now in firefox/chromium. When webpagetest needs 14 seconds, its probably loading your site too slow to be a good measuring tool, assuming your site isn't that slow. btw. chrome has some nice dev tools for rendering benchmarks including making screenshots of the different rendering stages. – allo Sep 04 '17 at 13:12
  • I mean metrics as onload event, speedindex, visually complete, etc. We do a lot of smaller optimization and usually, we check that particular metric in devTools. But I would like to have some general tool which would tell us for example "optimization which took us 20MDs improved load performance by 1s". I mean some general metric which is easy to understand also for non-dev people in the company and we are able to compare optimizations as removing some not so important libraries from bundle with optimizing React re-renders, optimizing css animations, better ordering of loading js bundles, etc. – user2530120 Sep 04 '17 at 14:25
  • We're working on a feature for A/B-testing in k6: https://github.com/loadimpact/k6/issues/299 – Ragnar Sep 05 '17 at 05:44

0 Answers0