Can not get copy_to_user work from the linux kernel

Question

We have a kernel driver and a user space application that interacts with this driver. The user space application allocates blocks of memory with posix_memalign, and passes these addresses to the kernel driver as follows:

    struct dma_cmd
    {
      int cmd;
      int usr_buf_size;
      char *buf;
    };

    struct set_mem_addr_cmd_struct
    {
      int channel_id;
      char *address;
    };

    char* register_mem_for_channel(ssize_t f, int channelid) {
            struct dma_cmd cmd;
            char *buf = (char *)malloc(sizeof(struct set_mem_addr_cmd_struct));
            cmd.usr_buf_size = sizeof(struct set_mem_addr_cmd_struct);
            cmd.cmd = CMD_SET_MEM_ADDR;

            (*(struct set_mem_addr_cmd_struct*)buf).channel_id = channelid;

            if ((channelid < 4))
                    char *addr;
                    posix_memalign(&addr, 4*1024, 4*1024*8);
                    if (addr == 0) {
                            printf("ERROR: Can not allocate aligned frame memory\n");
                            exit(-1);
                    }
                    (*(struct set_mem_addr_cmd_struct*)buf).address = addr;
            }
            cmd.buf = buf;
            write(f, &cmd, 0); // Send to driver
            char *addr = (*(struct set_mem_addr_cmd_struct*)buf).address;
            printf ("SET memory address for channel %d to %p\n", channelid, addr);
            return addr;
    }

The kernel driver has the following decleration:

#define MAX_CHANNELS 8
char *user_addr[MAX_CHANNELS];

... then in the kernel, we read the message from the user space which contains the target user space address:

...
...
    case CMD_SET_MEM_ADDR:
    {
        int cid;
        if (copy_from_user (&cmd_buf, kcmd.buf, sizeof (struct set_mem_addr_cmd_struct)))
        {
            printk(KERN_DEBUG "KERN: Set Memory Address FAULT\n");
            rc = -EFAULT;
            return rc;
        }
        cid = cmd_buf.channel_id;
        user_addr[cid] =  cmd_buf.address;
        printk(KERN_DEBUG "Set Channel %d Addr : %p\n", cmd_buf.channel_id, cmd_buf.address);
    }

... then the driver spawns a kernel thread and this thread gets in a loop and tries to copy sample data to user space:

while (1) {
    msleep(333);
    for (i=0; i<64; i++) imagedata[i] = i;
    printk(KERN_DEBUG "copy to user buffer A at address %p", user_addr[0]);
    if (!access_ok(VERIFY_WRITE, user_addr[0], 64)) {
        printk(KERN_DEBUG "ERROR: Can not access userspace addr\n");
    }
    errcode = copy_to_user(user_addr[0], &imagedata[0], 64);
    if (errcode != 0) printk(KERN_DEBUG "ERROR: COPY TO USER A FAILED!!! (errcode:%ld)\n", errcode);
}

In the above code, access_ok always returns true, so we do not get the access error. However, the copy_to_user call always returns 64, indicating that no bytes were copied to user space.

In the user space, we have code that prints out the bytes and we can confirm that nothing is written into the target. Here are the logs from the kernel and user side:

Kernel:
[ 9594.668322] Set Channel 0 Addr : 00007f460d68b000
[ 9594.719297] copy to user buffer A at address 00007f460d68b000
[ 9594.719341] COPY TO USER A FAILED!!! (errcode:64)
User:
USER: SET memory address for channel 0 to 0x7f460d68b000
USER: SIGNAL RECEIVED: value 1
USER: Callback for data received
USER: FIRST 10 BYTES for addr 0x7f460d68b000: 0 0 0 0 0 0 0 0 0 0

Is there a way to figure out why the copy_to_user call is not working?

Is process with `driver main code` (where `while(1)` cycle is run) *differs* from one, which perform ioctl command `CMD_SET_MEM_ADDR`? Note, that only a process, executed user code, has access to user-space data. — Tsyvarev, Jul 01 '15 at 09:20
@Tsyvarev: I am not sure whether I understood your comment. Let me try to clarify: while(1) is running on a kernel thread within the kernel driver. CMD_SET_MEM_ADDR command is sent from the user code (unthreaded), and the CMD_SET_MEM_ADDR command is received by the kernel driver. — SomethingBetter, Jul 01 '15 at 10:43

score 4 · Accepted Answer · edited May 23 '17 at 10:30

4

Your spawned kernel thread simply has no access to user-space memory.

While kernel's memory is shared between all processes(threads), switched to the kernel code, user-space memory is per-process, and can be accessed only from the process, owned this memory. In other words, address space notion is applicable even when process executes kernel code.

Note, that access_ok check is rough, it just tells that memory is not a kernel's one. See, e.g., this question.

If you want to share memory region between kernel space and user process, you can create (in the driver) file in debugfs or as character device and implement its mmap functionality.

edited May 23 '17 at 10:30

Community

1
1

answered Jul 01 '15 at 11:31

Tsyvarev

60,011
17
110
153

Basically kthread will inherit one of the previous mm's (http://comments.gmane.org/gmane.linux.kernel.kernelnewbies/11846), so the user part is always alive. The problem here, is that mm wouldn't be the one author need. – Alex Hoppus Jul 01 '15 at 11:44
@AlexHoppus, mm's inheritance is true for *forked* process. Pure kernel thread (created inside kernel) has no access to the user part at all. First answer in your link also say that. – Tsyvarev Jul 01 '15 at 11:55
@Tsyvarev : So if I do the copy_to_user not from within the kernel thread, but rather, from elsewhere in the kernel drive code, then it should work? – SomethingBetter Jul 01 '15 at 12:23
No, it wouldn't work. Analogue: run one user-space program, print pointer to its `a` variable, then pass this pointer to another user-space program and expect it to work with `a` variable in the first program. Because two programs have *different address spaces*, second program has no direct(via pointer) access to data of the first one. – Tsyvarev Jul 01 '15 at 12:44
@Tsyvarev read carefully, and you will got it "if it is NULL, then the process doesn't have process address space, in other word this is a kernel thread. But you also aware that even kernel threads don't acess user space memory, it still needs to access kernel space. because kernel space is 100% identical for every process, kernel thread can freely use memory descriptor (mm) owned by previously running process" see mm and active_mm, also this may help http://stackoverflow.com/questions/16975393/current-mm-gives-null-in-linux-kernel – Alex Hoppus Jul 01 '15 at 12:56
So basically you can write (but you shouldn't, cuz it will lead to unpredictable things) to userspace part even from kthread, the userspace part of page tables will be preserved from previous uspace process. There is no such mechanism to forbid writing to userspace from kernel ... – Alex Hoppus Jul 01 '15 at 13:09
@AlexHoppus: 1. This is an implementation detail, that kernel thread has memory descriptor copied from previous process. 2. In any case, every access to adresses below PAGE_OFFSET(user area) is denied: "All the kernel thread needed is page tables referencing toward virtual address bigger than PAGE_OFFSET, other are simply ignored by it is assumed that kernel thread doesn't need to access user space". So even copied `active_mm` doesn't grant access to user-space part of the memory. – Tsyvarev Jul 01 '15 at 13:26
@Tsyvarev ignored by what means? There is only one thing (called MMU) stands on the way of memory access. And PAGE_OFFSET is a kernel terminology, and MMU knows nothing about it ... – Alex Hoppus Jul 01 '15 at 13:53
In case of kernel thread the page tables are looks like page tables for previous userspace process (kernel part is common), there is no need to spend time for removing references for lower part (userspace). This is not an implementation detail, this is how Linux works. – Alex Hoppus Jul 01 '15 at 14:07
@AlexHoppus: This is an implementation detail, because OP wants to write correct kernel module. Correctly written kthread **shouldn't access** user-space memory. And exactly this expectation allows Linux kernel to use `lazy TLB` (term referred by Mel Gorman's book, section 4.3). Yes, such Linux implementation actually does not reject user-space access for kthread, but normal module shouldn't do that. – Tsyvarev Jul 01 '15 at 17:30
@AlexHoppus Is it possible to save the user process' context and restore it somewhere else in a kernel function or thread? If possible, which APIs are available for this purpose? Thanks in advance. – Hangchen Yu Mar 23 '18 at 22:17
Hardly it is possible. E.g. when use user's context outside of its thread, you need to make sure that user program is still run, and the context is still alive. – Tsyvarev Mar 23 '18 at 22:20

score 1 · Answer 2 · answered Jul 01 '15 at 10:55

1

Basically that wouldn't work. Because of many context will be switched, before you will make copy_to_user ... so your can't guess at which process address space you will copy.

answered Jul 01 '15 at 10:55

Alex Hoppus

3,821
4
28
47

Can not get copy_to_user work from the linux kernel

2 Answers2