I'm developing a platform similar to hackerrank.com where someone can submit C code, and then that code will be compiled, and run on my server, but I want to limit the C instruction set that a person will be able to execute on my server.
For example: limit the instruction set to I/O only.
My first approach was to parse the code and look for malicious code, but that is pretty naive because it can be easily overriden (shell code, obfuscation, etc..)
My second approach (the one I think it could work) is to remove all the "unnecessary" headers, and just leave stdio.h, math.h, stdlib.h, etc... just to name a few.
But then I thought that it might be possible to limit from gcc the instruction set of C, but after reading the man entry for gcc I couldn't find anything close to what I need, so I wonder if that's even possible.
If that's not possible, what could be a safe way to solve this problem? Other than getting rid of unnecessary libraries.
Thanks!