Adding a new system call to Linux that requires root privileges

Question

I'm trying to add a system call to the linux kernel(version:3.10.91) that requires root privileges to run.

You can see my attempt below:

#include <linux/kernel.h>
#include <linux/linkage.h>
#include <linux/sched.h>
#include <asm/current.h>
#include <asm/errno.h>

asmlinkage long sys_set_casper(pid_t pid, int value)
{
    struct task_struct *process;
    
    if (!capable(CAP_SYS_ADMIN))
        return -EPERM;
    
    //valid values are 0,1,2,3
    if (value != 0 && value != 1 && value != 2 && value != 3 )
        return -EINVAL;
    
    process = find_task_by_vpid(pid);
    if (process == NULL)
        return -ESRCH;

    //modify the casper field accordingly   
    process->casper = value;
    return 0;   
}

Casper is just a task descriptor that I added. Basically, I want the process to be hidden(not visible to ps,pstree,top etc.) when the casper value is 1. Kernel recompiles jut fine(and I also did the necessary changes in base.c etc.).

I tried testing my system call with the following code, test.c:

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#define NR_set_casper 351

int main(int argc, char **argv)
{
    long y;
    printf("PID of current process: %d\n\n", getpid());
    
    printf("Call set_casper system call to set flag 1\n");
    y = syscall(NR_set_casper, getpid(), 0);
    printf("Return value of set_casper system call: %ld\n", y);
    if (y < 0)
    {
        printf("set_casper system call failed. Run with sudo\n");
        return EXIT_FAILURE;
    }

    return 0;
}

I compile and run as follows:

gcc test.c

sudo ./a.out

The output is:

PID of current process: 3852

Call set_casper system call to set flag 1
Return value of set_casper system call: -1
set_casper system call failed. Run with sudo

The weird thing is, even after removing the sudo control lines:

 if (!capable(CAP_SYS_ADMIN))
        return -EPERM;

and recompiling the kernel, I still got the same error.

Basically, why my sys_set_casper function returns -1 ?

EDIT

I've added this: 351 i386 set_casper sys_set_casper to arch/x86/syscalls$ gedit syscall_32.tbl

However, my system is 64 bit. Could this be the problem?

Probably, your `sys_set_casper` is never called. BTW, how you register this syscall? — Tsyvarev, Oct 25 '15 at 22:35
I'm adding a detailed description of the things I've done step by step. — SpiderRico, Oct 25 '15 at 22:37
your kernel probably doesn't have a syscall with number 351. Depending on the architecture, you're going to have to add an entry to a specific file. For `arm` this is in `arch/arm/kernel/calls.S`. — Guru Prasad, Oct 25 '15 at 22:39
when it returns '-1' what does `errno` indicate? What does a call to `strerror()` indicate? — user3629249, Oct 25 '15 at 22:44
I do have a call with number 351 but it might be in the wrong place. Could you check the edited question ? — SpiderRico, Oct 25 '15 at 22:46
the code comments in the test function say setting the casper to 1 however, the syscall in the code is trying to set it to 0. — user3629249, Oct 25 '15 at 23:11

score 2 · Accepted Answer · answered Oct 26 '15 at 00:17

2

The problem was, as others stated in comments, that I wasn't calling the system call at all. I simply added my call to the 64 bit system call table and tried everything again and it worked.

answered Oct 26 '15 at 00:17

SpiderRico

1,890
8
28
48

Adding a new system call to Linux that requires root privileges

1 Answers1