Questions tagged [benchmarking]

Benchmarking is the process of comparing two or more systems or processes under controlled circumstances in order to have a quantitative measure with which to compare or rank them. The benchmarking tag should be used for questions about how to perform benchmarking tasks or theory questions, not for lists of benchmarking results or requests for benchmarking data; those questions are off-topic for Stack Overflow.

Benchmarking is the process of comparing two or more systems or processes under controlled circumstances in order to have a quantitative measure with which to compare or rank them.

For hardware, benchmarking typically involves either performing a simple task many times or a complex task to ascertain the performance characteristics desired (often speed, but power draw, heat, memory usage, and other characteristics may be of interest as well). Common benchmark operations include FLOPS (Floating Point Operations Per Second), write or read time for a large file, write or read time for many small files, rendering large images, and downloading or uploading large files over a network.

For software, benchmarking typically involves running the different software of interest (different versions of a program, different programs that accomplish a similar task, etc.) on an identical system (either one system, or two identical systems) and performing tasks that take sufficient time to notice a difference. This is often performed on very small differences in code, such as to verify which approach is superior to solving a particular problem.

Benchmarking also includes industry standard benchmarks and common benchmarking suites, used to assist users in making purchasing decisions or otherwise comparing their current systems to other available systems. However, it should be used in this context for issues with building or understanding the code and behavior of these benchmarks, not for general recommendations on the benchmarks suites or on the tested products.

Include additional tags to your question to indicate what type of benchmarking the question is about, such as , or what type of programming language if software, etc.

3517 questions
2
votes
0 answers

How to use bechmark for testing new algorithm multiple sequence alignment

How can I use benchmark for testing new algorithm for multiple sequence alignment , the benchmark as Balibase, OXbench...
user3216969
  • 169
  • 1
  • 1
  • 12
2
votes
1 answer

How to verify data is loaded into mongodb after load generation phase in ycsb?

I am trying to use ycsb to carry out benchmarking of mongodb. I have downloaded and installed mongo db as per instructions specified on YCSB MongoDB After that I created "ycsb" named database in mongodb using below commands 1. use ycsb 2.…
Nachiket Kate
  • 8,473
  • 2
  • 27
  • 45
2
votes
0 answers

Benchmarking a squid proxy

We have a few squid proxy servers that have been running for a few years already. I have been tasked to benchmark the service the proxy give (not the server), ideally using urls parsed from squid's log files. I have been looking for benchmarking…
Bruno9779
  • 1,551
  • 2
  • 14
  • 31
2
votes
1 answer

Why does IntStream.range(0, 100000).parallel().foreach take longer then normal for loop

I am just starting to learn about the Streams and parallel in Java and I was wondering why a normal for loop takes less time than IntStream paralleled at adding items to an array. package parallel; import java.util.stream.IntStream; public class…
drhunn
  • 21
  • 1
  • 2
  • 13
2
votes
1 answer

Benchmarking in Code Igniter

CodeIgniter has a Benchmarking class that is always active, enabling the time difference between any two marked points to be calculated. If you want to measure time difference between two points, you simply add the following…
MrD
  • 2,423
  • 3
  • 33
  • 57
2
votes
1 answer

Error loading large JSON files using Scala Play Framework 2

I'm trying to use Apache Bench to load test a group of large (4MB each) JSON requests. When running with a large file and many concurrent requests I get the following error: Exception caught in RequestBodyHandler…
2
votes
1 answer

Benchmarking Mysql cluster using sysbench

When benchmarking a mysql clustering using sysbench, do you have to install sysbench on every machine in the cluster to benchmark the cluster performance? Is there a way to install sysbench on one machine and use it to benchmark other mysql servers…
The Georgia
  • 1,005
  • 7
  • 23
  • 59
2
votes
0 answers

where i can find apache spark terasort source?

import org.apache.spark.rdd._ import org.apache.spark._ import org.apache.spark.SparkContext._ object ScalaTeraSort { def main(args: Array[String]) { if (args.length < 2) { System.err.println(s"Usage: $ScalaTeraSort…
2
votes
5 answers

Why the same code in WPF is slower than in Windows Forms?

I made bunch of benchmarks of the framework 4.0 and older and I can't understand why the same code is slower when using WPF compared to Windows Forms: This is the code, it has nothing to do with the UI elements: Random rnd = new…
Marco Bettiolo
  • 5,071
  • 6
  • 30
  • 35
2
votes
1 answer

Fastest way to iterate over the keys in a hash

I initialized a hash as in this question and ran some benchmarks. Benchmark.measure { a = h.keys } #=> 0.010000 0.000000 0.010000 ( 0.019832) Benchmark.measure { a.each { |k| } } #=> 0.060000 0.000000 0.060000 ( …
V_H
  • 1,793
  • 3
  • 34
  • 59
2
votes
3 answers

How does one calculate CPU utilization programmatically?

I have a benchmarking program that calculates the time (in milliseconds and ticks), for a persistance to Entity Framework 4.0. Is there a way to calculate CPU load ? I am guessing that I would need to query Windows to find out my CPU frequency,…
Scott Davies
  • 3,665
  • 6
  • 31
  • 39
2
votes
3 answers

Codeigniter benchmarking, where are these ms coming from?

I'm in the process of benchmarking my website. class Home extends Controller { function Home() { parent::Controller(); $this->benchmark->mark('Constructor_start'); $this->output->enable_profiler(TRUE); …
Ropstah
  • 17,538
  • 24
  • 120
  • 194
2
votes
0 answers

How to observe "drastic difference" in shared and unshared cases when using JMH states?

I take a look at JMH tool. In sample file JMHSample_03_States they says: ============================== HOW TO RUN THIS TEST: ==================================== You are expected to see the drastic difference in shared and unshared cases, …
qwazer
  • 7,174
  • 7
  • 44
  • 69
2
votes
0 answers

How to prevent GC when using Caliper

When using caliper, I get the ERROR: GC occurred during timing. as some garbage gets produced in my benchmark, which I can't avoid. I guess, giving more memory to the target JVM could help, as there's not that much garbage. I'm aware about the -D…
maaartinus
  • 44,714
  • 32
  • 161
  • 320
2
votes
1 answer

Getting timing consistency in Linux

I can't seem to get a simple program (with lots of memory access) to achieve consistent timing in Linux. I'm using a 2.6 kernel, and the program is being run on a dual-core processor with realtime priority. I'm trying to disable cache effects by…
Jim Hunziker
  • 14,111
  • 8
  • 58
  • 64
1 2 3
99
100