Questions tagged [benchmarking]

Benchmarking is the process of comparing two or more systems or processes under controlled circumstances in order to have a quantitative measure with which to compare or rank them. The benchmarking tag should be used for questions about how to perform benchmarking tasks or theory questions, not for lists of benchmarking results or requests for benchmarking data; those questions are off-topic for Stack Overflow.

Benchmarking is the process of comparing two or more systems or processes under controlled circumstances in order to have a quantitative measure with which to compare or rank them.

For hardware, benchmarking typically involves either performing a simple task many times or a complex task to ascertain the performance characteristics desired (often speed, but power draw, heat, memory usage, and other characteristics may be of interest as well). Common benchmark operations include FLOPS (Floating Point Operations Per Second), write or read time for a large file, write or read time for many small files, rendering large images, and downloading or uploading large files over a network.

For software, benchmarking typically involves running the different software of interest (different versions of a program, different programs that accomplish a similar task, etc.) on an identical system (either one system, or two identical systems) and performing tasks that take sufficient time to notice a difference. This is often performed on very small differences in code, such as to verify which approach is superior to solving a particular problem.

Benchmarking also includes industry standard benchmarks and common benchmarking suites, used to assist users in making purchasing decisions or otherwise comparing their current systems to other available systems. However, it should be used in this context for issues with building or understanding the code and behavior of these benchmarks, not for general recommendations on the benchmarks suites or on the tested products.

Include additional tags to your question to indicate what type of benchmarking the question is about, such as , or what type of programming language if software, etc.

3517 questions
2
votes
1 answer

Benchmarking various node.js unzip libraries using benchmark.js

I'm trying to write a benchmark, using benchmark.js, for some popular node.js unzip libs, mainly adm-zip, yauzl & unzip. However, I'm not sure if I've written the benchmark code correctly, because I keep getting Error: UNKNOWN, open…
kodeninja
  • 339
  • 1
  • 7
  • 23
2
votes
1 answer

Simple cross-platform code benchmarking

I'm aware of tools in the Xcode utilities for monitoring and benchmarking applications, but I'd like to test a few different (low level) implementations of library components, preferably across osx/linux/windows. Is there any framework for code…
Thijs Koerselman
  • 21,680
  • 22
  • 74
  • 108
2
votes
4 answers

If you were asked if a system could sustain double growth, what 3 things would you do to answer?

Let's say at your job your boss says, That system over there, which has lost all institutional knowledge but seems to run pretty good right now, could we dump double the data in it and survive? You're completely unfamiliar with the system. It's in…
Mark Brady
2
votes
3 answers

Speed difference between first run and second run

When I did benchmark my code I noticed that the first run is too slow and the next runs are too fast. what is the reason? for (int i = 0; i < 10; i++) { var timer = new Stopwatch(); timer.Start(); var expression = new Expression('x'); …
M.kazem Akhgary
  • 18,645
  • 8
  • 57
  • 118
2
votes
1 answer

Elastic search batch insert using curl is not working

I'm performing a small benchmark on an Elastic Search cluster. My benchmark script is written in bash and uses curl. I'm writing the data to a file that I'm posting to the REST API: curl -XPOST 'localhost:9200/benchmark/external/_bulk?pretty' \ …
André Laszlo
  • 15,169
  • 3
  • 63
  • 81
2
votes
1 answer

Is there a way to determine when CSS parsing/rendering has completed?

I am wondering for benchmarking purposes if there is a DOM event or other way to determine when the browser has finished parsing/rendering the CSS. I'm trying to optimize CSS selectors and need a way to measure the drawing/rendering/reflowing times.
Ben Crouse
  • 8,290
  • 5
  • 35
  • 50
2
votes
1 answer

JMH Benchmarking - Concise way to compare runtime of alternate implementations

I have two implementations of the same custom class: private List a = Util.myCustomClass(); private List b = Util2.myCustomClass(); and would like to compare the runtime of each of their functions (which all have the same name).…
2
votes
1 answer

Measuring the time of PHP scripts - Using $_SERVER['REQUEST_TIME']

Are this methods a reliable way to measure a script: $time = ($_SERVER['REQUEST_TIME_FLOAT'] - $_SERVER['REQUEST_TIME']); or $time = (microtime(true) - $_SERVER['REQUEST_TIME_FLOAT']); Which one should be used? And what's the difference of each…
Ash501
  • 321
  • 2
  • 4
  • 10
2
votes
1 answer

Any Javascript optimization benchmarks?

I watched Nicholas Zakas' talk, Speed up your Javascript, with some interest. I liked how he benchmarked the various performance improvements created by various optimization techniques, e.g. reducing calls to deeply nested objects, changing loops to…
int3
  • 12,861
  • 8
  • 51
  • 80
2
votes
1 answer

How to modify STREAM benchmark (MPI) for multicore processors with Intel Turbo Boost?

I want to run a MPI Version of the STREAM benchmark on one node of a cluster to measure the sustainable bandwidth for different numbers of MPI processes. Each node consists of two Intel® Xeon® Processor E5-2680 v3 (12 cores). In the following I will…
el_tenedor
  • 644
  • 1
  • 8
  • 19
2
votes
2 answers

C#( Mono ) managed vs unmanaged arrays: benchmark favours managed?

Benchmarking the following surprisingly gives better results for managed arrays( 10% faster, consistently ). I'm testing in Unity, so maybe it relates to Mono? unsafe void Bench() { //Locals int i, j; const int bufSize = 1024 *…
Gregzo
  • 1,194
  • 7
  • 18
2
votes
1 answer

How to interprete this YCSB CouchDB benchmark?

I recently ran YCSB benchmarks on CouchDB with 2 different workloads. Both workloads were using a database containing 500.000 documents and both of them executed 100.000 operations. The distribution of operations for each workload was: Workload…
Anton Horst
  • 263
  • 1
  • 4
  • 11
2
votes
1 answer

ElasticSearch: index document benchmarking

I am looking for some benchmarking of how many number of request can be indexed per second by ElasticSearch server. Surely, it would be subject to many factors like analyzers used, capacity of ES nodes, number of nodes in cluster and so on. I am…
Vishal Shukla
  • 2,848
  • 2
  • 17
  • 19
2
votes
1 answer

Accessing a benchmark's result

I've seen there is a struct testing.BenchmarkResult in Go to accesss the result of a benchmark but I found very little documentation or examples to help me use it. So far I have only been benchmarking my functions like this: func…
Spearfisher
  • 8,445
  • 19
  • 70
  • 124
2
votes
1 answer

MySQL BIGINT(20) vs Varchar(31) performance

I have read that bigint like 23423423423423423637 for primare unique key is better than varchar like 961637593864109_412954765521130 but how big is the difference when there are let's say 1 million rows when I never will sort but only select/update…
Matt123456
  • 99
  • 1
  • 11