Set user count & no. of iterations dynamically in load test in VS2015

Question

I know how to set the number of iterations and the number of users while doing web test/ load test in Visual Studio. But how can i set these dynamically via code. As a tester I just want to go and change the numbers in the code and just run the tests and not have to go to the edit options.

Say suppose I have an website to buy a phone. Iterations(Performance test)would be buying the phone for (say) 100 times. Increase in the number of users(load test) would be increase in the number of users buying the phone at the same time. A prime scenario for testing would be (say) 100 users buying the phone each 10 times, so a total of 100x10 times. Now in VS2015 I know how to go and edit the options in Visual Studio and change the numbers for Performance test and the load tests as mentioned in this link: https://learn.microsoft.com/en-us/vsts/load-test/run-performance-tests-app-before-release#createload. But I would like to do it programatically. I want to set the number of users and the number of times each user can buy the phone,through code.For instance,when I run the tests a cmd prompt is triggered,asking for 2 inputs, one: the number of the users that I want to test with and two: the number of times that each user can buy.The user enters both the inputs and then the test runs with the given number of users and iterations.

For example,this is just like setting an environment variable. A user can go to the machine properties and set the required environment variable manually. But he can also just write a small batch file and run it, which sets the required environmental variable.

Number of users is covered in https://stackoverflow.com/a/49039530/546871 . How and when do you want set the numbers? In code running in the test, via some GUI, or what? What do you want to happen when you change the number of iterations? Do you want to change them while the test is running? Or only before it starts? Really this question is too broad at present. Please [edit] the question to properly explain what you want and what you have already tried. — AdrianHHH, Mar 14 '18 at 22:48

score 3 · Accepted Answer · answered Mar 15 '18 at 22:08

The number of users (actually the load pattern of constant or step or goal) and the number of iteration to run are specified in the ".loadtest" file. It is an XML file and the values are easily found in the XML.

Here is a cut down version of a ".loadtest" file showing the parts wanted by the question, plus a few other salient values. This example is for a "step" load pattern but it is easy to example the file for a goal based or a constant load test and find the correct XML.

<LoadTest ...>
  <Scenarios>
    <Scenario Name="..." ...>
      <LoadProfile Pattern="Step"
          InitialUsers="1" MaxUsers="500"
          StepUsers="1" StepDuration="3" StepRampTime="0" />
    </Scenario>
  </Scenarios>
  <RunConfigurations>
    <RunConfiguration Name="Run" UseTestIterations="false" RunDuration="1800"
        WarmupTime="0" CoolDownTime="90" TestIterations="100" ...>
    </RunConfiguration>
  </RunConfigurations>
</LoadTest>

Given the simplicity of the ".loadtest" file it would be easy to create a batch script or a program that reads in the required numbers and writes them into a given ".loadtest" file and then starts the load test via a program such as mstest.exe.

Once a test is running the number of users can be changed by having a load test plugin write the required number into m_loadTest.Scenarios[N].CurrentLoad for a suitable N. This would probably be done in the Heartbeat plugin but could be in any load test plugin. I am not aware of any way of altering the number of iterations wanted at run time, but it may be possible via the objects and values passed into a load test plugin.

Note that big increases to the number of users may produce a test that fails because there are insufficient agent computers. Conversely having enough agents available for the highest number of users that you may ever want may mean that test runs with few users are very wasteful of agent capacity.

Set user count & no. of iterations dynamically in load test in VS2015

1 Answers1