0

I'm trying to create a Linux software in C++ which need to run code in a protected environment on x86 and x86-64 processor.

My problem is to find a way to run code in protected environment, first, only on x86-64 (it's a technical part of processors way of working), I have see Local Descriptors Table, but I found it no more works on x86-64. I also heard about the Intel VT technology, but documents seems very complicated.

Have you any idea of ways to run code in a protected environment on linux and x86-64 inside a process?

My goal is to create something like an OS inside a linux process.

Like Windows or Linux does, I want the program runned inside my protected environment no to access part of my software, and make systemcall if needed. I believe I have found a way to do so, I esxplain it below.

Nicolas Bourdon
  • 115
  • 1
  • 8
  • First of all, IA64 is an official name for Intel Itanium. What you meant is most likely the Intel 64, a.k.a AMD64 a.k.a x86_64. Most likely you do not want to write for Itanium, unless you have a box with it under your desk. – Grigory Rechistov Mar 15 '18 at 20:47
  • Second, you really need to clarify what you want to achieve. Any modern OS provides you with a protected virtual memory environment, where all the applications (including the browser you use right now to read this sentence) are run. You can use it right away. If what you need is a bare-bone Protected mode (legacy 32-bit or long 64-bit), then you would essentially need to write a mini-OS for that. All of this is only remotely related to virtualization (in modern understanding of the word) – Grigory Rechistov Mar 15 '18 at 20:51
  • Thanks for your answer, indeed I want to program for x86_64. – Nicolas Bourdon Mar 16 '18 at 16:05
  • I send the first part of the answer by mistake.What I want to make is a software a little bit like java in simpler, I want the program runned inside the proccess not to access to not allowed part of memory of my virtual machine. Am I clear ? – Nicolas Bourdon Mar 16 '18 at 16:24
  • Then your question is more about writing a language virtual machine ("like Java"), not writing/using a system virtual machine or host system's protection capabilities. Rewrite your question correspondingly, please. And by the way, "I want the program run inside the process not to access to not allowed part of memory" — that is what a conventional multi-tasking OS does, which I also pointed out earlier. – Grigory Rechistov Mar 16 '18 at 17:52
  • It's quite conplicated in fact: and what I want is a little bit different from what you said. – Nicolas Bourdon Mar 16 '18 at 18:36
  • I both want to run a different language like java and have something like a virtual machine , but transform it into assembly code before running, and run the this assembly in a way it cant access to every part of the host proccess memory. That s possible I dont use the right terms, so I look for documents on thethe web and I changr my question after – Nicolas Bourdon Mar 16 '18 at 18:42
  • I know the operating system run code in process which cant access to firbidden part of the memory. To be clearer I d like to run something wich look like to an os inside a process of an os ( Linux) – Nicolas Bourdon Mar 16 '18 at 18:52
  • 1
    What you want already exists: http://user-mode-linux.sourceforge.net/. I don't know how "protected" the host is from `root` on the guest system, though. – Peter Cordes Mar 20 '18 at 22:52
  • Tanks peter, indeed the software you showed to me certainly use tricky to run code that I may use. I Certainly find a good way to do what I want finaly. I will explain it below. – Nicolas Bourdon Mar 21 '18 at 12:38

1 Answers1

0

I have found a way to do what I want:

Each time my program will switch from main part to the program inside, it will use mprotect (a function of Glibc on Linux) to change the right to access to lot of part of the memory of the process.

Each time the program inside will make a systemcall to my program, it will change back the right to access to the memory.

You may thinks it stays security issues, because the program inside can run any kind of code and access to system call to linux and so can access to not allowed things. But I believe I can use a tricky which would prohibit the code inside to start any kind of opcodes.

Nicolas Bourdon
  • 115
  • 1
  • 8
  • 1
    If you run arbitrary machine code, how do you stop it from making an `mprotect` system call directly, or `mmap(MAP_FIXED)` to break out of the sandbox, or simply `execve("/bin/sh", {"sh", "-c", "whatever command", NULL}, NULL)`? Unmapping glibc won't help, because the asm can contain a `syscall` instruction to make the system call directly. https://blog.packagecloud.io/eng/2016/04/05/the-definitive-guide-to-linux-system-calls/ – Peter Cordes Mar 21 '18 at 12:55
  • You are right about this point. My tricky is very complicated to explain: first I want to save code in my executable file in a format like a bitecode. – Nicolas Bourdon Mar 21 '18 at 13:00
  • And if you were going to just disassemble the machine code to see if it contains `syscall`, that doesn't work because the code could `jmp` into the middle of another instructions (which looked like an innocent `imm32` when you looked at it). For any detection method you come up with, more and more crafty asm can generate a `syscall` or `int 0x80` on the fly, especially if self-modifying code is possible. (e.g. if the sandbox runs with any pages with write + execute). In that case, detection is equivalent to solving [the halting problem](https://en.wikipedia.org/wiki/Halting_problem) – Peter Cordes Mar 21 '18 at 13:00
  • Anyway, the main important part of this answer is "I have a trick", which you don't explain at all. So this is not an answer! – Peter Cordes Mar 21 '18 at 13:04
  • Thanks for your comment peter, I always make a mistake by pushing the enter key too soon!II want that when the bitecode will be transformed into a assembler code, then it would create only bite code with no access to int or other badopcodes, but like you say it's not enought, for two reason: The segment of code may be written (simply change it to executable and readeable will be enought), but an other problem is that it can jump to bad places where bad opcodes (opcodes of systemcall for instance) resides. – Nicolas Bourdon Mar 21 '18 at 13:06
  • to resolve this last problem, I would create a part of memory only readeable to the program inside, and add address of consistent opcodes inside it. Each time the opcode would have to jump, it must refers to this read only table, transform the index in this table to the address of the code to jump to. – Nicolas Bourdon Mar 21 '18 at 13:09
  • Yup. x86's variable-length instructions make it possible to hide instructions inside other instructions, to make polymorphic code that runs differently from multiple decode starting points. This wouldn't be a problem on a fixed-width instruction set if you enforce `W^X` (write xor execute, never both on any one page in your whole process). – Peter Cordes Mar 21 '18 at 13:10
  • I don't understand your last comment. Are you actually writing an interpreter? Are you going to disallow indirect jumps, and somehow stop code from modifying a return address on the stack before running a `ret`? (Or just pushing a target address) Anyway, you should [edit] these comments into your answer instead of leaving the real answer in comments. – Peter Cordes Mar 21 '18 at 13:12
  • As only consistent opcodes are addressed, are started, and consistent opcodes always jumps to consistent opcode, then it should be ok. Do you see what I mean? – Nicolas Bourdon Mar 21 '18 at 13:12
  • I think you're checking all the `jmp` and `jcc` instructions. But I think you forgot about `push addr_of_syscall` / `ret` or other ways of modifying stack memory. Are you replacing every `ret` instruction with a `jmp` to some checking function? How can you do that, when `ret` is a single byte? Are you going to use `int3` and use `ptrace` to catch the debug exception? That's would be horribly slow if you had to catch a signal for every `ret` in the guest code. – Peter Cordes Mar 21 '18 at 13:15
  • I m trying to create an interpreter (to translate C to a kind of bitecode) and an assembler to run this bitecode inside the host process. I dont want to store a return adress in the stack, but an index to the table which holds adress of coherent opcodes. – Nicolas Bourdon Mar 21 '18 at 13:15
  • Huh, I thought you wanted to be able to run native x86-64 machine code. You're actually making something like a JVM with a custom bytecode you're going to interpret? The question title is asking about running assembler code. But I guess you don't mean x86-64 assembler, you mean some assembly language you're going to invent. – Peter Cordes Mar 21 '18 at 13:18
  • It would be a little slower, but I guess not so much. Each time the program would have to jump to a variable adress, it would have to translate the index inside this table to an adress to the code. – Nicolas Bourdon Mar 21 '18 at 13:20
  • My project is maybe to much ambitious, and if it begins to works, I d like to have some help from other people. My host program should transform bitecode from the exucutable file to a x86_64 code, so it would be very faster, and by the use of this different tricky, it would be secured. – Nicolas Bourdon Mar 21 '18 at 13:22
  • Hi again If you didn't understand, want more details, or think there is problem in this method, please let me know. – Nicolas Bourdon Mar 22 '18 at 22:11
  • I don't understand your exact goal or your method, and your answer still doesn't explain it at all. So you're making a JIT implementation of a C interpreter / VM? It's not supposed to be able to run user-provided x86-64 machine code at all? Just instructions generated by your JIT-compiler? IDK if you realize that it's possible to produce slow machine code if you don't have a very good optimizer in there somewhere. It can be slower than a well-optimized intepreter. – Peter Cordes Mar 22 '18 at 22:19
  • I'm very sorry if I won't succed to explain well, my english is not perfect, and the subject is quite complicated. It's not supposed to be able to run user-provided x86-64 machine code at all? No because i wouldn't be secure, it could put interupt opcode. First a compiler would transform C++ source files, into a bytecode, for instance this bytecode file would be named .BVM . And after this .BVM file would be runned by my software, exactly it would be transformd in x86-64 opcodes, and directly run by the processor. because it's not safe to run native assembler, my software would use tricky . – Nicolas Bourdon Mar 22 '18 at 22:38
  • Ok, so you need to create an *optimizing* compiler for this to be efficient. This is very hard, like many person-years of work for an expert. https://softwareengineering.stackexchange.com/questions/273698/why-are-there-so-few-c-compilers/273711#273711 – Peter Cordes Mar 22 '18 at 22:41
  • Yes, indeed, you are certainly true. The part of the program transforming c++ to bytecode could be taken from gcc maybe. I wondered if my tricky (using a table permitting not to jump to all part of memory) was not already used in java? – Nicolas Bourdon Mar 22 '18 at 22:47
  • BTW, the English word you're looking for is "trick" (noun), not "tricky" (adjective). Anyway, Java doesn't have pointers the way C++ does, so it's a language that's designed to only allow things that a JVM can efficiently sandbox. Taking the address of a local and overwriting the return address using an offset from that isn't even possible in Java, because that's not how the language works. JVMs prove stuff about what the Java bytecode code does, then emit machine code to correctly implement that known behaviour. They never get into a situation where they can't fully trust JITed machine code. – Peter Cordes Mar 23 '18 at 01:59