1

I'm trying to run performance tests through Gitlab CI using Taurus and JMeter. I have a NavWeb.jmx containing many endpoints to test, but I'm trying to set up two jobs, one that runs the entire testplan and another containing a matrix of jobs where you can pick and choose which endpoint to test.

Example:

gitlab ci

(run_perf_test_QA runs all of them together while run_single_test lets you choose which test to run)

My solution currently is to have two JMX files, one where the tests are enabled and another where they're disabled. I then use the Taurus "enable" modification that enables one specific test. This solution isn't ideal because it requires keeping two copies of the JMeter script synchronized. Another solution would be to disable each test explicitly in the YML and then enable one of them programmatically. Is there a way to accomplish this with one JMX file and and without needing to explicitly list every test in the YML files?

Aditya Pandey
  • 113
  • 1
  • 5

1 Answers1

3

You can do it in a single test plan using:

  1. Module Controllers - to keep individual endpoints and their arbitrary combinations
  2. Switch Controller - to choose which endpoint(s) to run
  3. __P() function - to make the endpoint(s) selection externally configurable

Test plan outline:

enter image description here

So

  • if you run JMeter as:

    jmeter -Jtarget=all

    it will execute all 3 endpoints

  • if you run JMeter as:

    jmeter -Jtarget=endpoint1

    it will execute only endpoint1

  • etc.

If you want to trigger the test using Taurus (however I fail to see the valid use case for Taurus there) you can pass the property via -o command-line argument like:

bzt -o modules.jmeter.properties.target=all

or

bzt -o modules.jmeter.properties.target=endpoint1

Full test plan just in case:

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.3">
  <hashTree>
    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Test Plan" enabled="true">
      <stringProp name="TestPlan.comments"></stringProp>
      <boolProp name="TestPlan.functional_mode">false</boolProp>
      <boolProp name="TestPlan.tearDown_on_shutdown">true</boolProp>
      <boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
        <collectionProp name="Arguments.arguments"/>
      </elementProp>
      <stringProp name="TestPlan.user_define_classpath"></stringProp>
    </TestPlan>
    <hashTree>
      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Thread Group" enabled="true">
        <stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
        <elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
          <boolProp name="LoopController.continue_forever">false</boolProp>
          <stringProp name="LoopController.loops">1</stringProp>
        </elementProp>
        <stringProp name="ThreadGroup.num_threads">1</stringProp>
        <stringProp name="ThreadGroup.ramp_time">1</stringProp>
        <boolProp name="ThreadGroup.scheduler">false</boolProp>
        <stringProp name="ThreadGroup.duration"></stringProp>
        <stringProp name="ThreadGroup.delay"></stringProp>
        <boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
      </ThreadGroup>
      <hashTree>
        <SwitchController guiclass="SwitchControllerGui" testclass="SwitchController" testname="Switch Controller" enabled="true">
          <stringProp name="SwitchController.value">${__P(target,)}</stringProp>
        </SwitchController>
        <hashTree>
          <ModuleController guiclass="ModuleControllerGui" testclass="ModuleController" testname="all" enabled="true">
            <collectionProp name="ModuleController.node_path">
              <stringProp name="764597751">Test Plan</stringProp>
              <stringProp name="764597751">Test Plan</stringProp>
              <stringProp name="-1948168983">Thread Group</stringProp>
              <stringProp name="96673">all</stringProp>
            </collectionProp>
          </ModuleController>
          <hashTree/>
          <ModuleController guiclass="ModuleControllerGui" testclass="ModuleController" testname="endpoint1" enabled="true">
            <collectionProp name="ModuleController.node_path">
              <stringProp name="764597751">Test Plan</stringProp>
              <stringProp name="764597751">Test Plan</stringProp>
              <stringProp name="-1948168983">Thread Group</stringProp>
              <stringProp name="-1837756314">endpoint 1</stringProp>
            </collectionProp>
          </ModuleController>
          <hashTree/>
          <ModuleController guiclass="ModuleControllerGui" testclass="ModuleController" testname="endpoint2" enabled="true">
            <collectionProp name="ModuleController.node_path">
              <stringProp name="764597751">Test Plan</stringProp>
              <stringProp name="764597751">Test Plan</stringProp>
              <stringProp name="-1948168983">Thread Group</stringProp>
              <stringProp name="-1837756313">endpoint 2</stringProp>
            </collectionProp>
          </ModuleController>
          <hashTree/>
          <ModuleController guiclass="ModuleControllerGui" testclass="ModuleController" testname="endpoint3" enabled="true">
            <collectionProp name="ModuleController.node_path">
              <stringProp name="764597751">Test Plan</stringProp>
              <stringProp name="764597751">Test Plan</stringProp>
              <stringProp name="-1948168983">Thread Group</stringProp>
              <stringProp name="-1837756312">endpoint 3</stringProp>
            </collectionProp>
          </ModuleController>
          <hashTree/>
        </hashTree>
        <ResultCollector guiclass="ViewResultsFullVisualizer" testclass="ResultCollector" testname="View Results Tree" enabled="true">
          <boolProp name="ResultCollector.error_logging">false</boolProp>
          <objProp>
            <name>saveConfig</name>
            <value class="SampleSaveConfiguration">
              <time>true</time>
              <latency>true</latency>
              <timestamp>true</timestamp>
              <success>true</success>
              <label>true</label>
              <code>true</code>
              <message>true</message>
              <threadName>true</threadName>
              <dataType>true</dataType>
              <encoding>false</encoding>
              <assertions>true</assertions>
              <subresults>true</subresults>
              <responseData>false</responseData>
              <samplerData>false</samplerData>
              <xml>false</xml>
              <fieldNames>true</fieldNames>
              <responseHeaders>false</responseHeaders>
              <requestHeaders>false</requestHeaders>
              <responseDataOnError>false</responseDataOnError>
              <saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
              <assertionsResultsToSave>0</assertionsResultsToSave>
              <bytes>true</bytes>
              <sentBytes>true</sentBytes>
              <url>true</url>
              <threadCounts>true</threadCounts>
              <idleTime>true</idleTime>
              <connectTime>true</connectTime>
            </value>
          </objProp>
          <stringProp name="filename"></stringProp>
        </ResultCollector>
        <hashTree/>
        <TestFragmentController guiclass="TestFragmentControllerGui" testclass="TestFragmentController" testname="all" enabled="false"/>
        <hashTree>
          <ModuleController guiclass="ModuleControllerGui" testclass="ModuleController" testname="Module Controller - endpoint  1" enabled="true">
            <collectionProp name="ModuleController.node_path">
              <stringProp name="764597751">Test Plan</stringProp>
              <stringProp name="764597751">Test Plan</stringProp>
              <stringProp name="-1948168983">Thread Group</stringProp>
              <stringProp name="-1481272069">Test Fragment - endpoint 1</stringProp>
            </collectionProp>
          </ModuleController>
          <hashTree/>
          <ModuleController guiclass="ModuleControllerGui" testclass="ModuleController" testname="Module Controller - endpoint 2" enabled="true">
            <collectionProp name="ModuleController.node_path">
              <stringProp name="764597751">Test Plan</stringProp>
              <stringProp name="764597751">Test Plan</stringProp>
              <stringProp name="-1948168983">Thread Group</stringProp>
              <stringProp name="-1481272068">Test Fragment - endpoint 2</stringProp>
            </collectionProp>
          </ModuleController>
          <hashTree/>
          <ModuleController guiclass="ModuleControllerGui" testclass="ModuleController" testname="Module Controller - endpoint 3" enabled="true">
            <collectionProp name="ModuleController.node_path">
              <stringProp name="764597751">Test Plan</stringProp>
              <stringProp name="764597751">Test Plan</stringProp>
              <stringProp name="-1948168983">Thread Group</stringProp>
              <stringProp name="-1481272067">Test Fragment - endpoint 3</stringProp>
            </collectionProp>
          </ModuleController>
          <hashTree/>
        </hashTree>
        <TestFragmentController guiclass="TestFragmentControllerGui" testclass="TestFragmentController" testname="endpoint 1" enabled="false"/>
        <hashTree>
          <kg.apc.jmeter.samplers.DummySampler guiclass="kg.apc.jmeter.samplers.DummySamplerGui" testclass="kg.apc.jmeter.samplers.DummySampler" testname="endpoint1" enabled="true">
            <boolProp name="WAITING">true</boolProp>
            <boolProp name="SUCCESFULL">true</boolProp>
            <stringProp name="RESPONSE_CODE">200</stringProp>
            <stringProp name="RESPONSE_MESSAGE">OK</stringProp>
            <stringProp name="REQUEST_DATA">Dummy Sampler used to simulate requests and responses
without actual network activity. This helps debugging tests.</stringProp>
            <stringProp name="RESPONSE_DATA">Dummy Sampler used to simulate requests and responses
without actual network activity. This helps debugging tests.</stringProp>
            <stringProp name="RESPONSE_TIME">${__Random(50,500)}</stringProp>
            <stringProp name="LATENCY">${__Random(1,50)}</stringProp>
            <stringProp name="CONNECT">${__Random(1,5)}</stringProp>
            <stringProp name="URL"></stringProp>
            <stringProp name="RESULT_CLASS">org.apache.jmeter.samplers.SampleResult</stringProp>
          </kg.apc.jmeter.samplers.DummySampler>
          <hashTree/>
        </hashTree>
        <TestFragmentController guiclass="TestFragmentControllerGui" testclass="TestFragmentController" testname="endpoint 2" enabled="false"/>
        <hashTree>
          <kg.apc.jmeter.samplers.DummySampler guiclass="kg.apc.jmeter.samplers.DummySamplerGui" testclass="kg.apc.jmeter.samplers.DummySampler" testname="endpoint2" enabled="true">
            <boolProp name="WAITING">true</boolProp>
            <boolProp name="SUCCESFULL">true</boolProp>
            <stringProp name="RESPONSE_CODE">200</stringProp>
            <stringProp name="RESPONSE_MESSAGE">OK</stringProp>
            <stringProp name="REQUEST_DATA">Dummy Sampler used to simulate requests and responses
without actual network activity. This helps debugging tests.</stringProp>
            <stringProp name="RESPONSE_DATA">Dummy Sampler used to simulate requests and responses
without actual network activity. This helps debugging tests.</stringProp>
            <stringProp name="RESPONSE_TIME">${__Random(50,500)}</stringProp>
            <stringProp name="LATENCY">${__Random(1,50)}</stringProp>
            <stringProp name="CONNECT">${__Random(1,5)}</stringProp>
            <stringProp name="URL"></stringProp>
            <stringProp name="RESULT_CLASS">org.apache.jmeter.samplers.SampleResult</stringProp>
          </kg.apc.jmeter.samplers.DummySampler>
          <hashTree/>
        </hashTree>
        <TestFragmentController guiclass="TestFragmentControllerGui" testclass="TestFragmentController" testname="endpoint 3" enabled="false"/>
        <hashTree>
          <kg.apc.jmeter.samplers.DummySampler guiclass="kg.apc.jmeter.samplers.DummySamplerGui" testclass="kg.apc.jmeter.samplers.DummySampler" testname="endpoint3" enabled="true">
            <boolProp name="WAITING">true</boolProp>
            <boolProp name="SUCCESFULL">true</boolProp>
            <stringProp name="RESPONSE_CODE">200</stringProp>
            <stringProp name="RESPONSE_MESSAGE">OK</stringProp>
            <stringProp name="REQUEST_DATA">Dummy Sampler used to simulate requests and responses
without actual network activity. This helps debugging tests.</stringProp>
            <stringProp name="RESPONSE_DATA">Dummy Sampler used to simulate requests and responses
without actual network activity. This helps debugging tests.</stringProp>
            <stringProp name="RESPONSE_TIME">${__Random(50,500)}</stringProp>
            <stringProp name="LATENCY">${__Random(1,50)}</stringProp>
            <stringProp name="CONNECT">${__Random(1,5)}</stringProp>
            <stringProp name="URL"></stringProp>
            <stringProp name="RESULT_CLASS">org.apache.jmeter.samplers.SampleResult</stringProp>
          </kg.apc.jmeter.samplers.DummySampler>
          <hashTree/>
        </hashTree>
      </hashTree>
    </hashTree>
  </hashTree>
</jmeterTestPlan>
Dmitri T
  • 159,985
  • 5
  • 83
  • 133