Questions tagged [benchmarking]

Benchmarking is the process of comparing two or more systems or processes under controlled circumstances in order to have a quantitative measure with which to compare or rank them. The benchmarking tag should be used for questions about how to perform benchmarking tasks or theory questions, not for lists of benchmarking results or requests for benchmarking data; those questions are off-topic for Stack Overflow.

Benchmarking is the process of comparing two or more systems or processes under controlled circumstances in order to have a quantitative measure with which to compare or rank them.

For hardware, benchmarking typically involves either performing a simple task many times or a complex task to ascertain the performance characteristics desired (often speed, but power draw, heat, memory usage, and other characteristics may be of interest as well). Common benchmark operations include FLOPS (Floating Point Operations Per Second), write or read time for a large file, write or read time for many small files, rendering large images, and downloading or uploading large files over a network.

For software, benchmarking typically involves running the different software of interest (different versions of a program, different programs that accomplish a similar task, etc.) on an identical system (either one system, or two identical systems) and performing tasks that take sufficient time to notice a difference. This is often performed on very small differences in code, such as to verify which approach is superior to solving a particular problem.

Benchmarking also includes industry standard benchmarks and common benchmarking suites, used to assist users in making purchasing decisions or otherwise comparing their current systems to other available systems. However, it should be used in this context for issues with building or understanding the code and behavior of these benchmarks, not for general recommendations on the benchmarks suites or on the tested products.

Include additional tags to your question to indicate what type of benchmarking the question is about, such as , or what type of programming language if software, etc.

3517 questions
2
votes
2 answers

Timing program execution

I have a project written in C and I would like to know if there is a simple way to profile its execution time and memory usage under Windows. Thanks in advance.
user142162
2
votes
5 answers

C++ How to benckmark very fast operations

I'm in VC++2013, Windows 7-64, Intel i7 3.6 GHz. I want to measure the execution time of very fast math operations, for example I wish to compare the performance of the standard fabsf() function with alternate "faster" methods, or the standard…
Mark Miles
  • 706
  • 8
  • 20
2
votes
3 answers

Speed of if compared to conditional

I had the idea I would turn some of my if blocks into single lines, using the conditional operator. However, I was wondering if there would be a speed discrepancy. I ran the following test: static long startTime; static long elapsedTime; static…
2
votes
1 answer

How can I benchmark a tornado handler within the handler?

I have written a very simple tornado handler intended to test the upload speed of some devices that are deployed remotely. The main test is going to be run in said remote devices where (thanks to cURL), I can get a detailed report on the different…
Savir
  • 17,568
  • 15
  • 82
  • 136
2
votes
0 answers

Microbenchmarking base R and three packages on string pattern substitution

My question is whether my method and conclusion are correct. As part of my learning regular expressions, I wanted to figure out in which order to learn the various alternatives (base R and packages). I thought it might help to learn the relative…
lawyeR
  • 7,488
  • 5
  • 33
  • 63
2
votes
1 answer

Benchmarking my Golang webserver

I am looking for some tools or the Go tests package to run some benchmarks on different servers. Any idea how i get some nice profiling output in my console. Is it possible to simulate multiple users visiting the server? getting no output at from…
user3396016
  • 119
  • 1
  • 10
2
votes
1 answer

OpenJDK JMH sometimes prints (*interrupt*) in results. What does this exactly mean?

I use OpenJDK JMH 0.9.3 and sometimes i get a result log file like this one below. What means (*interrupt*) here? Forking 1 times using command: [/Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/...] # Run progress: 53,85% complete, ETA 00:01:26 #…
salyh
  • 2,095
  • 1
  • 17
  • 31
2
votes
1 answer

Cryptsetup Benchmark Values unrealistic

i am currently evaluating the AES encryption and decryption speed on my laptop and my workstation. when executing cryptsetup benchmark -c aes --key-size 128 I get normal results of almost 200MB/s without the AESNI extension. when i load the…
user3694354
  • 105
  • 8
2
votes
1 answer

svn-bench and fsfs-stats binary usage

I looked upon the svn-bench and fsfs-stats binary tools in SVN 1.8 as mentioned in http://subversion.apache.org/docs/release-notes/1.8.html#new-tools But I'm unable to use it through command line client. I'm currently using SVN 1.8.9. Is there any…
Coolbreeze
  • 906
  • 2
  • 15
  • 34
2
votes
1 answer

C++ threads: shared memory not updated despite absence of race

I'm experimenting with C++ standard threads. I wrote a small benchmark to test performance overhead and overall throughput. The principle it to run in one or several threads a loop of 1 billion iterations, making small pause from time to time. In…
Christophe
  • 68,716
  • 7
  • 72
  • 138
2
votes
0 answers

Why is Qpar faster than OpenMP?

I have a series of benchmarks that carry out the same calculations via CUDA, Multiple Threads and OpenMP, currently being tested via Windows 8.1. The threaded program required MS Compiler Version 18.00 for x64, from Visual Studio 2013, to produce…
Roy Longbottom
  • 1,192
  • 1
  • 6
  • 8
2
votes
2 answers

ZRANGEBYSCORE in Redis and Ruby

I am trying to optimize ZRANGEBYSCORE in my Redis code in Ruby. Specifically, the Redis website (http://redis.io/commands/zrangebyscore) states: Time complexity: O(log(N)+M) with N being the number of elements in the sorted set and M the number…
tjrburgess
  • 757
  • 1
  • 8
  • 18
2
votes
4 answers

Any ASP.Net benchmark tool?

I would like to measure the performance of the .Net application, especially web application in ASP.Net when running in the server. I need to know any benchmark tool enable for me to know my source code can be optimize. As long as the benchmark tool…
HexaHow
  • 21
  • 1
  • 3
2
votes
0 answers

How can I improve performance compiling for SSE and AVX?

My new PC has a Core i7 CPU and I am running my benchmarks, including newer versions that use AVX instructions. I have installed Visual Studio 2013 to use a newer compiler, as my last one could not fully compile for full SSE SIMD operation. Below is…
Roy Longbottom
  • 1,192
  • 1
  • 6
  • 8
2
votes
2 answers

Benchmarking in oracle jdk 8 for performance testing

Usually when I want to test performance I am using System.nanoTime() function before running code and after it and then I am doing a a subtraction. Recently I noticed that people are using…
Govan
  • 2,079
  • 4
  • 31
  • 53
1 2 3
99
100