1

Perhaps this is a dumb question, but I've never thought about it until I was just recently forced to.

Assume a program is initialized with the exact same state and no undefined behavior in the way of uninitialized variables. Register states on the CPU can be different prior to code execution. Assume no outside phenomena is causing freak bit flipping.

Will the same sequence of numerical calculations always produce the same result (as in, the entire bit-representation of doubles and floats)? In other words, is numerical error deterministic?

EDIT: I know that pointer addresses can be different each time, but let's abstract pointer addresses away from being used in any calculations.

user650261
  • 2,115
  • 5
  • 24
  • 47
  • Could you show a code example of "_a program is initialized with the exact same state, and has no nondeterministic behavior (random numbers) and no undefined behavior in the way of uninitialized variables._"? – Khalil Khalaf Jul 28 '16 at 15:07
  • Please refine 'has no nondeterministic behavior'. The way you ask it, it's 'is it the same if it's the same?'. You're probably thinking about whether e.g. `double operator/(double, double)` is deterministic, but it's not that clear. Also I assume you consider time & network comm. non-deterministic. – lorro Jul 28 '16 at 15:08
  • What is unclear? What type of code would be suitable? This is a general question, so I don't want to confine it to a specific example. I am asking "Does there exist code that would lead to different outcomes," and in those cases, what are the circumstances? – user650261 Jul 28 '16 at 15:08
  • @lorro I defined it as random number generation, right next to that. EDIT: removed the misleading text altogether. – user650261 Jul 28 '16 at 15:09
  • That said, I suppose adding "nondeterministic behavior" was silly of me, as any pseudorandom numbers that are seeded the same should be the same. – user650261 Jul 28 '16 at 15:10
  • Pretty clear to me... Don't know the answer, though. I'd guess it's reproducible, but there might be interesting effects with last-bit errors caused by the fact that FP registers may be larger than `double` and the least significant bits might contain “garbage” of some sort. Or maybe not! Has to know the hardware, context switching and compiler features to figure it out. – Sergei Tachenov Jul 28 '16 at 15:12
  • @user650261: the way you ask it, 'I have the same state and my machine is deterministic. Is the output of `print_numerical_error()` determined?' - for which the answer is yes. However, noone stops you from implementing `double` in the hardware using analog amplifiers for `operator*()` and then you suddenly have indeterministic numerical error (and behaviour) which is still *not* UB and is (read: theoretically could be) standard-conforming. – lorro Jul 28 '16 at 15:14
  • @lorro Could you expand a bit? I suppose I am most curious about references to guarantees given by current C++ implementation standards. – user650261 Jul 28 '16 at 15:15
  • 1
    @user650261: AFAIK the standard only guarantees that `long double` is at least as good as `double`; `double` is at least as good as `float`; and a wide range of numbers is specifiable. It doesn't tell about the actual method you implement your calculation, hence that might be analog as well (as long as you specify other conditions). Not saying it's being done this way, I'm just saying it's not disallowed. – lorro Jul 28 '16 at 15:22
  • It seems to me that this question better suited for http://programmers.stackexchange.com/ – Random Davis Jul 28 '16 at 15:30
  • On a machine (processor) not influenced by random events, it should be deterministic. However, not having that, any change in the processing (e.g.: not keeping an intermediate result in a register, but storing it memory) may change a result. –  Jul 28 '16 at 15:30
  • You should clarify "initialized in the same state" as well. For example, should it include CPU / math processor states resulting from previous unrelated operations? What about the position of [cosmic rays](http://physics.stackexchange.com/q/32663/41010) in the universe at a given moment? – Jason C Jul 28 '16 at 15:33
  • @RandomDavis this is a specific question about C++. Why would this be better for programmers.stackexchange.com? – user650261 Jul 28 '16 at 15:33
  • @Jason C What is a math processor? Do you just want to know about registers in the CPU? I will edit to assume that that can vary. Let's not get esoteric about ridiculous phenomena. – user650261 Jul 28 '16 at 15:35
  • @user650261 because stackoverflow is about getting help with problems in your specific code, not for general questions about a programming language - or at least that's my interpretation. Whereas the programmers stackexchange is about general programming questions, which may be specific to a language or not. – Random Davis Jul 28 '16 at 15:35
  • @user650261 I think it's an all right question. I agree it probably shouldn't be on programmers. I can't speak for SO, I enjoy these types of questions but I also often disagree with some of the social norms here (not a criticism, mind you, just take my opinion with a grain of salt). It does enter the realm of theoretical, though, you might get a good answer on http://cs.stackexchange.com/. You might also be able to make your question more concrete by highlighting the specific issue in your code that you're trying to debug. :) – Jason C Jul 28 '16 at 15:36
  • @RandomDavis well this is important for me to understand where to start debugging my specific code that is having very small numerical errors. The programmers stack exchange is described as "site for professional programmers interested in conceptual questions about software development." I would not consider this conceptual, I would consider this quite concrete. – user650261 Jul 28 '16 at 15:36
  • 1
    @JasonC it doesn't really help because I'd like some evidence backing it up / more details about standards and/or specific implementations of standards. Right now, while I'm sure you are all very smart and knowledgeable people, the answers given are not much more than "yes" or "no" without a why - the answers therefore have to be taken on faith. That said, yes, I believe it's a duplicate :/ – user650261 Jul 28 '16 at 15:45
  • 1
    @user650261 Yeah you're right, there wasn't much supporting evidence given there. I'd definitely hit up CS if you don't get a satisfactory answer here. Personally I find the topic interesting, if I have the time today to dig up some background evidence and the question is still open I'll try to post something. Generally the answer is "no, unless the entire state of the universe is the same", but simplifications can be made, especially with a more constrained example case. – Jason C Jul 28 '16 at 15:50
  • @RandomDavis when referring other sites, it is often helpful to point that [cross-posting is frowned upon](http://meta.stackexchange.com/tags/cross-posting/info) – gnat Jul 28 '16 at 16:36

1 Answers1

3

Formally no, but in practice mostly yes. People were therefore quite annoyed when a bug in a DLL caused other programs to exhibit non-reproducible results.

The particular case I was thinking of involved a DLL that was loaded into every process on a computer. I don't recall the exact reason, but I think it was some kind of Input Method Editor (which is often used for non-ASCII languages). This particular DLL did some floating point calculations, and changed the floating-point rounding mode!.

MSalters
  • 173,980
  • 10
  • 155
  • 350