0

A friend of me challenged me to hack a small program he coded. Basically, it is an exe file that displays an image but in order to do so you need a key file with a password.

I started to reverse engineer it with ollydgb and I was able to figure out that a file with name key.txt needs to be present and contain the password. Another thing I realized is that my friend uses the password to calculate a memory adress and call it. So, if you have the wrong password, the application will crash since it will jump to a random address, probably causing a violation.

He basically stores de password in EBX. He puts a fixed value EAX. Then he does ADD EAX, EBX and finally CALL EAX.

So, knowing all this, if I know which address be executed next, I could just substract the address from the fixed value stored in EAX and I will get the HEX value corresponding to the password.

My problem is, how in the world can I know which should be the next address to be executed? I am fairly new to cracking...

I try to point to the next address after the CALL but it does not work. I also checked the libraries it is using and I certainly see opengl32 but I am not sure if I have to jump somehow to that library.

My question is, how can I figure out which is the next address to be executed?

John Saunders
  • 160,644
  • 26
  • 247
  • 397
martinvigo
  • 143
  • 3
  • 10
  • 3
    I love questions that start with *"A friend of mine..."* – Robert Harvey Oct 16 '12 at 22:14
  • 1
    Unlike forum sites, we don't use "Thanks", or "Any help appreciated", or signatures on [so]. See "[Should 'Hi', 'thanks,' taglines, and salutations be removed from posts?](http://meta.stackexchange.com/questions/2950/should-hi-thanks-taglines-and-salutations-be-removed-from-posts). – John Saunders Oct 17 '12 at 00:56
  • How about _show us the code_ (disassembly) ? Reverse engineering via chinese whisper is usually, when done, highly paid for - usually by clients of organisations like the CIA, NSA, or similar. stackoverflow isn't charging, so please make it a little easier. – FrankH. Oct 17 '12 at 07:02

2 Answers2

2

One of the attributes of the X86 instruction set is that despite being variable length (instructions are encoded using a variable number of bytes), instruction sequences will converge rather quickly. This means if you randomly pick a location as the start of a function and start disassembling from that point forward you won't get too screwed up. A few of the instructions you disassemble will be wrong, but eventually (and rather quickly) you will end up with real instructions.

That means you can do things like pick random addresses in the executable, and start looking for common patterns, like the prologue and epilogue of a function. Those can give you hints as to where the code is.

That, however, assumes that you friend isn't hand writing the assembly or otherwise obfuscating the code.

However, no matter what he does, he will have some calls to methods in the import address table, or system call instructions. These will either run the code to draw the image, or will modify the page table of the process so that obfuscated code can be deobfuscated. Those patterns would be more reliable. So a good place to start is by dumping the iat (import address table), and looking for calls to the interesting functions. If that doesn't work, try looking for system call patterns. Exactly what to look for depends on the particular OS you are on.

A few things to keep in mind:

  1. If your friend didn't turn off ASLR (address space layout randomization), then the code might not always work even if you have the right password.

  2. I'm not too familiar with olydbg, but I going to assume it uses recursive descent disassembly. If it does, it may not include your target function in the disassembly because it won't have any statically resolvable calls to it.

To solve 2 you will probably want to write code to do the disassembly your self, decoding the instructions. The best place to start would be to scan through the exe looking for indirect call instructions. After that try system calls.

Update

One more thing: It is possible for there to not be system calls in the image if the exe is configured to make the exe section writable or the data section executable (or to turn off the nxbit). Its a good idea to take a look at the section table in the image and see if there is anything unusual as a sanity check before you start.

Scott Wisniewski
  • 24,561
  • 8
  • 60
  • 89
  • ASLR shouldn't be a problem if the target function is in the code/data section of the image and isn't being deliberately moved/copied (by the programmer) somewhere else. If that's the case, the target function is at a fixed distance from the image base, which the question hints at by mentioning the constant/fixed value being added. – Alexey Frunze Oct 17 '12 at 08:10
  • That all depends on the particular encoding of the call instruction used, the value of the offset, and whether or not the appropriate relocation entries are setup. – Scott Wisniewski Oct 17 '12 at 08:58
  • ASLR does not break the image apart to move its constituents at different distances w.r.t. each other. Instruction encoding has little to do with ASLR. – Alexey Frunze Oct 17 '12 at 09:00
  • My point was not that it was difficult to make such code work (it's not). If he uses either ip relative forms of the encoding, or sets up the appropriate relocation entries such code could work. I wanted to point out that if the code was handcrafted, it could contain a bug that would make its behavior vary between runs of the program, and it might not run correctly. It's something that can be checked for by examining the call instruction. I'm advising him to ask the question "did my friend set this call up so that it would always work, or not." – Scott Wisniewski Oct 17 '12 at 09:10
  • Bugs, sure, can produce weirdest results. Valid point. Btw, `rip`-relative addressing is only available in 64-bit mode, which isn't the case here. – Alexey Frunze Oct 17 '12 at 09:12
  • That's not true (about 64 bit only). The "e8" version of the call instruction works in 32 bit mode and uses ip-relative addressing. – Scott Wisniewski Oct 17 '12 at 09:16
  • Shoot, you're right. It was a momentary lapse of consciousness. – Alexey Frunze Oct 17 '12 at 09:22
  • Thanks for all the advices. I am pretty sure there is no ASLR as my friend has no idea about security. Also, it is not really an image what is diplayed but more a real time rendered effect. I know he used two "security measures". One is the one I found (a call to a dynamic address using the password offset) and another is ((I think) using the password offset as part of the math to render the tunel effect the program displays. – martinvigo Oct 22 '12 at 01:56
0

Unless the program is tiny (and by that I mean just a few KB of code), you can't figure it out easily and need to perform some code analysis, make hypotheses and confirm or disprove them by trial and, inevitably, error.

The first thing that comes to my mind is finding functions that are present in the code but there are no call/jump instructions transferring control to them. You can narrow down the search by first identifying those functions that will have to be called in order for the picture to be drawn (the ones that call the relevant OpenGL/DirectX/GDI functions). And then you can work backward from there.

It may also be useful to see if there are any function pointers/addresses in the data section of the executable, the reason being that the compiler may remove functions that aren't referenced by any other functions or pointers and so for the function of interest to still be present in the compiled executable, there must be some references to it somewhere (unless, of course, the code is compiled with all optimizations off).

Either way, I'm afraid, you're going to have to plow through a lot of code.

Another thing to look at is the the executable itself and its sections (e.g. their names, sizes, locations or contents). That might give you some clues.

Alexey Frunze
  • 61,140
  • 12
  • 83
  • 180
  • The program is pretty tiny. I will look through the functions as you recommended, that may seem a good way to start. Thank you! – martinvigo Oct 22 '12 at 01:58