movsql call results in a core dump

Question

Based on the core file, the where command was used first then the disassemble command.

(gdb) disassemble
Dump of assembler code for function prm_get_sta:
   0x0000000000414b80 <+0>:     push   %rbp
   0x0000000000414b81 <+1>:     mov    %rsi,%rbp
   0x0000000000414b84 <+4>:     push   %rbx
   0x0000000000414b85 <+5>:     mov    %rdi,%rbx
   0x0000000000414b88 <+8>:     sub    $0x18,%rsp
   0x0000000000414b8c <+12>:    movabs 0x6a53e8,%rax
   0x0000000000414b96 <+22>:    test   %rax,%rax
   0x0000000000414b99 <+25>:    je     0x414c90 <prm_get_sta+272>
   0x0000000000414b9f <+31>:    mov    0x80(%rax),%edx
   0x0000000000414ba5 <+37>:    test   %edx,%edx
   0x0000000000414ba7 <+39>:    jne    0x414c90 <prm_get_sta+272>
   0x0000000000414bad <+45>:    mov    0x8(%rbx),%edx
   0x0000000000414bb0 <+48>:    mov    0x18(%rbx),%rax
   0x0000000000414bb4 <+52>:    add    (%rbx),%rdx
   0x0000000000414bb7 <+55>:    cmp    %rdx,%rax
   0x0000000000414bba <+58>:    je     0x414ce6 <prm_get_sta+358>
   0x0000000000414bc0 <+64>:    mov    %rax,0x0(%rbp)
   0x0000000000414bc4 <+68>:    movswq (%rax),%rdx
   0x0000000000414bc8 <+72>:    movabs $0x6a2900,%rcx
=> 0x0000000000414bd2 <+82>:    movslq (%rcx,%rdx,8),%rdx
   0x0000000000414bd6 <+86>:    add    0x18(%rbx),%rdx
   0x0000000000414bda <+90>:    mov    %rdx,0x8(%rbp)

Next, info registers was executed.

(gdb) info registers
rax            0x1672fa0        23539616
rbx            0x7fff6ec02f40   140735051476800
rcx            0x6a2900         6957312                 <----------------Hex to Decimal value
rdx            0xffffffffffffcccd       -13107          <----------------Hex to Decimal value 
rsi            0x7fff6ec01060   140735051468896
rdi            0x7fff6ec02f40   140735051476800
rbp            0x7fff6ec01060   0x7fff6ec01060
rsp            0x7fff6ec01000   0x7fff6ec01000
r8             0x7fff6ec02588   140735051474312
r9             0x111    273
r10            0x0      0
r11            0x2abd31653c50   46992065903696
r12            0x414b80 4279168
r13            0x7fff6ec02b00   140735051475712
r14            0x414440 4277312
r15            0x0      0
rip            0x414bd2 0x414bd2 <prm_get_sta+82>
eflags         0x10297  [ CF PF AF SF IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0

Research: The movslq command supposedly does 32->64-bit 2's complement sign extension, extending by copying the sign-bit of the source to all the new upper bits

Documentation was found in reference to the unique usage of the data in parentheses : (%rcx,%rdx,8) which explained it like this :

(%rcx, %rdx, 8) Contents of memory stored at address, %rcx + 8%rdx*

If following the logic properly, I interpreted it to mean the following using the above registry data...

(6957312 + 8*-13107)

Using order of operations, this would first do (8*-13107) which results in : -104,856 then added 6957312 which is 6,852,456 and small enough number.

The -104,856 value results in a full 8 bytes and wondered if there could potentially be an issue there.

Questions:

1 - Is this assumption of what is happening correct (6957312 + (8*-13107)) ?

2 - What is the purpose of multiplying by 8?

3 - Is there anything obvious that would cause the core?

Answer 1

The instruction is accessing an invalid memory address and the reason seems to be a negative array index.

As you mention yourself, (%rcx, %rdx, 8) accesses memory stored at address %rcx + 8*%rdx.

In other words, we're loading an item from an 8-byte-element array starting at rcx with the index rdx.

In your case rcx is 0x6a2900 (likely a variable in data section, try info symbol 0x6a2900) and rdx is -13107 - a negative number. Indexing arrays with negative indexes happens rarely in real programs so you'll need to look at the source of the function and try to understand how it could have happened.