I am trying to debug a tricky core dump (from an -O2 optimized binary).
// Caller Function
void caller(Container* c)
{
std::list < Message*> msgs;
if(!decoder.called(c->buf_, msgs))
{
....
.....
}
// Called Function
bool
Decoder::called(Buffer* buf, list < Message*>& msgs)
{
add_data(buf); // Inlined code to append buf to decoders buf chain
while(m_data_in && m_data_in->length() > 0)
{
.....
}
}
In both caller and the callee, the first argument is optimized out, that means it must be somewhere in the register.
Caller Disassembly:
push %r15 mov %rdi,%r15 push %r14 push %r13 push %r12 push %rbp push %rbx sub $0x68,%rsp test %rsi,%rsi je 0x8ccd62 cmpq $0x0,(%rsi) je 0x8ccd62 lea 0x40(%rsp),%rax lea 0x1b8(%rdi),%rdi mov %rax,(%rsp) mov %rax,0x40(%rsp) mov %rax,%rdx mov %rax,0x48(%rsp) mov (%rsi),%rsi callq 0x8cc820
Caller Register Info:
rax 0x7fbfffc7e0 548682057696 rbx 0x2a97905ba0 182931446688 rcx 0x0 0 rdx 0x2 2 rsi 0x1 1 rdi 0x7fbfffc7e2 548682057698 rbp 0x4f 0x4f rsp 0x7fbfffc870 0x7fbfffc870 r8 0x40 64 r9 0x20 32 r10 0x7fbfffc7e0 548682057696 r11 0x2abe466600 183580911104 r12 0x7fbfffd910 548682062096 // THIS IS HOLDING buf_ r13 0x7fbfffdec0 548682063552 r14 0x5dc 1500 r15 0x2a97905ba0 182931446688 rip 0x8cca89 0x8cca89 eflags 0x206 [ PF IF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0
Called function Disassembly:
push %r14 push %r13 mov %rdx,%r13 push %r12 mov %rdi,%r12 push %rbp push %rbx sub $0x10,%rsp mov 0x8(%rdi),%rdx test %rdx,%rdx jne 0x8cc843 jmpq 0x8cc9cb mov %rax,%rdx mov 0x8(%rdx),%rax test %rax,%rax mov %rsi,0x8(%rdx) mov 0x8(%r12),%rax test %rax,%rax xor %edx,%edx add 0x4(%rax),%edx mov 0x8(%rax),%rax lea 0x8(%rsp),%rsi mov %r12,%rdi movq $0x0,0x8(%rsp)
Called function Register Info
:
rax 0x7fbfffc7e0 548682057696 rbx 0x2abc49f9c0 183547591104 rcx 0x0 0 rdx 0x2 2 rsi 0x1 1 rdi 0x7fbfffc7e2 548682057698 rbp 0xffffffff 0xffffffff rsp 0x7fbfffc830 0x7fbfffc830 r8 0x40 64 r9 0x20 32 r10 0x7fbfffc7e0 548682057696 r11 0x2abe466600 183580911104 r12 0x2a97905d58 182931447128 r13 0x7fbfffc8b0 548682057904 r14 0x5dc 1500 r15 0x2a97905ba0 182931446688 rip 0x8cc88a 0x8cc88a eflags 0x206 [ PF IF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0
The issue is, in the called function, it appears that "add_data" function achieved nothing.
So, wanted to know whether in disassembly of called function, do we see the "buf_" pointer being used anywhere (Register r12 in callee function).
I do understand assembly to some level, but all those code inlining has left me confused.
Would appreciate some help in demistifying called function disassembly.
UPDATE:
add_data does below:
if (m_data_in) {
m_data_in->next = data;
} else {
m_data_in = data;
}
