0

I'm trying to write a program, which calculates lcm and gcd of two unsigned longs in assembly x86-64, but I get some floating point error, when I'm launching my program. The error is in polish, but I can translate it into "Floating point operations error".

Here is the code:

    .global lcm_gcd
        .type lcm_gcd,@function

        .section .text

lcm_gcd:
        mult = %rdx
        mov %rdi, %rax
        mul %rsi
        mov %rax, mult

.comp:
        cmp %rdi, %rsi      # gcd
        je .found
        jb .secPos
        sub %rdi, %rsi
        jmp .comp

.secPos:        # %rdi > %rsi <==> x > y
        sub %rsi, %rdi

        jmp .comp

.found:
        div %rdi

        ret

        .size   lcm_gcd, . - lcm_gcd

And the test program in C:

    #include <stdio.h>
#include <stdlib.h>

typedef struct {
    unsigned long lcm, gcd;
} result_t;

result_t lcm_gcd(unsigned long x, unsigned long y);

int main(int argc, char **argv) {
    result_t r = lcm_gcd(atol(argv[1]), atol(argv[2]));
    printf("%lu, %lu\n", r.lcm, r.gcd);

    return 0;
}
Michael Petch
  • 46,082
  • 8
  • 107
  • 198
  • When you run it in a debugger like _GDB_ what does it say? – Michael Petch Apr 15 '17 at 17:04
  • @MichaelPetch, I don't really know how to use it, but I've determined that the error occurs due to the "div %rdi" instruction. Without this line, the program doesn't crash. –  Apr 15 '17 at 17:26
  • I'll give you a hint. In 64-bit code [DIV](http://www.felixcloutier.com/x86/DIV.html) takes the 128-bit value in RDX:RAX and divides it by the operand (in your case _RDI_). You didn't set _RDX_. You likely want it set to 0. It likely has a non 0 value. If the integer division of a 128-bit number by a 64-bit number can't fit in 64-bit the processor will raise a Division Exception – Michael Petch Apr 15 '17 at 17:27
  • You really should learn to use a debugger. It doesn't take much to learn, and can be a powerful tool. – Michael Petch Apr 15 '17 at 17:28
  • @MichaelPetch thanks man, now it works. I'm definitely gonna learn GDB when only I have some time. –  Apr 15 '17 at 17:42
  • Possible duplicate of [Why should EDX be 0 before I use DIV opcode](http://stackoverflow.com/questions/38416593/why-should-edx-be-0-before-i-use-div-opcode) – Michael Petch Apr 15 '17 at 17:49

0 Answers0