Performing load/performance testing in VDI, does it provides proper results

Question

Till now I was doing load/performance testing (load runner & jMeter) on my local instance(connected to LAN not over wifi) and I was sure about the results. But now I have to do it Virtual desktop infrastructure (VDI). Does it provide the same result as the local instance? Or is it good practice to perform the tests over the VDI?

score 0 · Answer 1 · answered Mar 23 '17 at 05:30

LoadRunner or JMeter don't care about underlying hardware as you have at least 2 abstraction layers: operating system and C and/or Java runtime.

So given your VDI has the same hardware specifications - you should be getting the same results for the same test in terms of delivered load. I would however recommend monitoring your VDI main health metrics, like CPU load, RAM and Pagefile usage, Network and Disk IO, etc.

In the majority of cases VDIs don't have fully dedicated resources, i.e. if you see 64GB of RAM it is not guaranteed you can allocate them all as the RAM may be shared between other VDIs on hypervisor level.

So monitor your load generator(s) system resources usage and make sure you have enough headroom for running your load tests. See How to Monitor Your Server Health & Performance During a JMeter Load Test guide for more details.

@Dmiti T, Thanks for the answer. When i host the application locally, it was fast while navigating between screens/pages. But when i host the same application on some server (client environment) and when i access it through VDI the response time was very low (here am not performing any load test, just myself using). My concern is, is it good to perform testing in that environment? — Patrick, Mar 23 '17 at 05:35

score 0 · Answer 2 · answered Mar 23 '17 at 12:34

Use a physical load generator as a control element. Run single virtual users of each type on the physical box. If you see that your control group begins to take on different performance characteristics (slower, higher degrees of variance as measured by standard deviation, higher averages, 90th percentiles and maximums) then you have a case for maintaining some physical infrastructure for testing.

The biggest issue directly attacking timing record integrity inside of virtualized load generators is clock jump. The system clock in the virtualized host floats slower with respect to the physical clock on the hardware. Occasionally this needs to be re-synched, causing time to jump. Inevitably this happens while the timing record is open and causes what appears to be a long timing record. Some people suggest that this doesn't happen until you start to see backups in the CPU queue length, which is somewhere in the 75-80% CPU range. I have seen it at even as low as 10% of CPU, because under those light loads the hypervisor can make decisions to parse resources to higher need virtualized instances and then when you get the CPU token back it is time to jump the clock.

The control load generator provides a check against this behavior. If necessary you can even use the control load generator in a statistical sampling model along the lines of manufacturing quality control. You can also show objectively to anyone demanding you move to a virtualized model what the impact of this change will be on the integrity of the response time samples collected.

score 0 · Answer 3 · answered Dec 03 '18 at 06:13

It depends on the VDI. Is it on same network as your localhost? If it is, the result would be almost same (depends on configuration of VDI too, very minimal overheads that we might not even notice though)

If it is not, the result would depend on how good is that network.

Performing load/performance testing in VDI, does it provides proper results

3 Answers3