0

I am researching a code virtualization solution named VMProtect, which creates a custom stack machine with a virtual instruction set, to obfuscate code. What I want to do, is lift every virtual machine instruction handler, most of which consists of stack based operations, to LLVM IR, and then apply optimization and recompile.

I will be using llvm's IRBuilder for this.

Some of my questions are:

  1. How can push/pop be represented in IR
  2. What would a handler like this look like in IR
  mov     rdx, [rbp] ; get virtual stack operand
  add     rbp, 8         ; add to stack pointer
  mov     [rax+rdi], rdx ; move to scratch space
drew sicard
  • 1
  • I've done that (transformed a stack machine to IR) and also answered questions about the subject on SO. Your questions seems very general, IMO too general for SO, so I'll vote to close as "needs more focus". Please look at earlier LLVM-related questions and answers, [this for example](https://stackoverflow.com/a/55948669) and post questions if you have any concrete, focused problems. – arnt Jul 18 '21 at 16:56
  • 1
    [McSema](https://github.com/lifting-bits/mcsema) can lift a PE but I suppose you want to do it programmatically. I thought it was worth mentioning, though. – Margaret Bloom Jul 18 '21 at 17:24

0 Answers0