0

scenario: I have a script, which has around 14 transaction, out of which few are repeated transaction, so we have them in fragments. And we are using the random CSV data set config for parameterisation

Issue: As this random CSV config is referred many transactions with in a single iteration and also the number of threads is more than 500, we encounter a memory issue during the run and samples get stalling, this is due to the file opening on for every user, for every iteration to use this random CSV data set.

Do we have any other alternative to this random CSV data set config element?

Note: Every script of mine has one or more random CSV data set configs, and I have around 8+ scripts witth more than 4k users for the entire scenario

trincot
  • 317,000
  • 35
  • 244
  • 286
prabhu s
  • 1
  • 1
  • This sounds like a perfect job for a database rather than a file, I imagine that it will eliminate the stalling you are getting. Alternately for a small project, you could just keep the file contents in memory rather that referring to a file every time. – sorifiend Mar 09 '21 at 01:40
  • If you are closing the resources properly you shouldn't have a memory issue because of them - but you might have a leak in your data structure that holds what you've loaded. And on many platforms a file used really heavily is going to be cached - so the resource isn't going to be the problem. OR you might just have a lot loaded into memory - if you have 500 threads with their own copy of say 200K of transaction data [4K lines * 50bytes is 200K] - that's about 2GB of memory usage ... Is that what you were expecting? – Mr R Mar 09 '21 at 02:22
  • we get an IO exception in jmeter log , while running tthis huge data file, (my csv files are more 48k). i am looking for alternation method for random csv file for my entire scenarios – prabhu s Mar 09 '21 at 03:12

1 Answers1

0
  1. You can use "normal" CSV Data Set Config instead. I don't like the idea of using the Random CSV Data Set Config at all because I want tests to be repeatable, but if for some reason you want/need random data - you can "shuffle" your CSV file(s) somewhere in setUp Thread Group using JSR223 Sampler
  2. Alternatively you can go for Redis Data Set which has "random" option
  3. HTTP Simple Table Server also supports RANDOM mode
  4. You can also put your data into the database and access it using JDBC Request sampler
Dmitri T
  • 159,985
  • 5
  • 83
  • 133