I want to unwind a x64 callstack, so I'm trying to follow the "UNWIND procedure" I found here:
http://msdn.microsoft.com/en-us/library/8ydc79k6.aspx
I understand that if the RIP is in the epilog, we need to compute the offset of the RSP considering the operations still needed to be done, but it's unclear to me how to find out if the RIP is in the epilog or not (in section 3.a)?
Can somebody explain this to me? (a link or code example will be also greatly appreciated)
From The Unwind Procedure
To determine if the RIP is within an epilog, the code stream from RIP on is examined. If that code stream can be matched to the trailing portion of a legitimate epilog, as described in section CNDJ6nn5us4RjIIAqgBLqQsCAAAACAAAAA4AAABfAFIAZQBmADQAOQA2ADAAOQAyADQAMgA1AAAA REF _Ref496092425 \r \h 0, then it is in an epilog, and the remaining portion of the epilog is simulated, with the context record updated as each instruction is processed
, and from Prolog and Epilog (I recommend entire article for better understanding)
These are the only legal forms for an epilog. It must consist of either an add RSP,constant or lea RSP,constant[FPReg], followed by a series of zero or more 8-byte register pops and a return or a jmp. (Only a subset of jmp statements are allowable in the epilog. [...]). No other code can appear. In particular, nothing can be scheduled within an epilog, including loading of a return value.
Note that, when a frame pointer is not used, the epilog must use add RSP,constant to deallocate the fixed part of the stack. It may not use lea RSP,constant[RSP] instead. This restriction exists so the unwind code has fewer patterns to recognize when searching for epilogs.
It seems that if code finds itself in such situation it justch checks currently executed code for those specific instructions and if matched, considers itself to be in the epilog.
