Incorrect output when calculating magnitude of 3 vectors in Atmel Studio

Question

I'm having some trouble trying to get a floating point output. What I'm trying to do is take the output of an accelerometer on 3 axes. The output from each axis has a high and low byte that are combined into a single variable. The magnitude of these three variables is then computed thusly: mag = sqrt(x*x + y*y + z*z). The output from each axis is output as a 16-bit signed integer.

At least, that's how it should work, but I can't even manage to multiply two large numbers together to produce the correct value in my program, and I see no reason for it as the variables should all be plenty large to hold any results. Here's the code:

    double xAccel = 0;              // 16 bit X acceleration value
    double accelSum = 0;
    xAccel = xAccelRead();

    accelSum = 10000*10000;
    char aMag[64];
    sprintf(aMag, "Accel Mag: %.2f", accelSum);

    clearDisplay();
    writeText(aMag,0,0,WHITE,BLACK,1);
    OLED_buffer();

The output of xAccelRead() is a 16 bit int. Normally, "accelSum" would be set equal to the magnitude equation given above, but for now the static numbers aren't even working. If I set the equation to 100^2, it works. But 10000 * 10000 doesn't work. The result of that should be 100,000,000. But the output I get is:

Accel Mag: -7936.00

I can't understand why this is. I've tried setting the types to int32, int64, and now floating point. Same problem with all of them. I set the correct linker options in Atmel Studio to allow floating point sprintf support, so that's not the problem. I'm guessing there is an overflow somewhere but I can't figure out where. All variables involved have types that are more than large enough to accommodate max values in the hundreds of billions, which is far more than I need anyway.

Say I set accelSum = 1000 * 1000. That's a mere million. Plenty small for an int32 to hold. But my output is:

Accel Mag: 16960.00

Even with 200 * 200, the output is -25536.00.

This has to be some stupid simple problem. If anybody can help me out I'd really appreciate it!

So if I add UL after the numbers, everything works! I guess when you type a number it is automatically typecast as a type that is then unable to contain intermediate calculations, resulting in overflow. Is that right? But how do I do that to a variable used in a calculation? — SVFeingold, Sep 06 '13 at 10:24

Pascal Cuoq · Accepted Answer · 2013-09-06T12:41:33.117

You have answered most of your question in your comment.

Your problem is similar to the surprise some programmers have with float f = 5 / 3; that the division is computed as an integer division, although it is destined to be assigned to a float. The solution in their case is to use float f = 5.0 / 3.0; instead.

If the expressions being divided or, in your case, multiplied, are not constants, you can convert them to a wider type with a cast: float f = (float)int_var1 / (float)int_var2; or, in your case, unsigned long foo = (unsigned long)int_var1 * (unsigned long)int_var2;. You could also convert to double so that the computed multiplication is a double multiplication: accelSum = (double)int_var1 * (double)int_var2;.

Only one of the casts is actually necessary because of “promotion rules”, but it is just as clear to write both.

If you are interested in the details, the C standard mandates that the type of an integer constant is selected as the first type that can represent the constant in a list. In C99(*), the list is int, long, long long. In your case, a 16-bit int is enough to contain 10000 or 200, so int is picked as the type of these constants.

(*) The idea was the same but the list was different in C90.

Thanks for the help! The rules in your first paragraph are new to me, but make sense. Perhaps not the most intuitive business, typecasting. Definitely have some changes to make here. — SVFeingold, Sep 07 '13 at 06:12

Incorrect output when calculating magnitude of 3 vectors in Atmel Studio

1 Answers1