3

Is there a reason that opening a device file (rather than a regular file) using numpy's memmap shouldn't work?

self.surface = np.memmap('/dev/fb1', dtype=np.uint16, mode='r+', shape=(320,240))

I'm working with a custom kernel module that adds a framebuffer device, which works fine with python's regular mmap module. But using numpy seems to hang the kernel's mutex on accessing the filesystem or something (I'm really not sure exactly what's happening).

My question here is specifically is this something that numpy's memmap can't handle and I should go a different way?

I've asked another question on unix stackexchange, but I feel like it's 2 different questions so I've posted them both.

Obviously this is on linux (kubuntu maverick with custom kernel module)

Update:

Well, it turns out I can create the memmap fine. The problem it seems is that when I close the process without specifically closing the memmap object and it will just hang on the mutex in the kernel.

I have no idea if this issue is with numpy, or my kernel module, or somewhere else.

Community
  • 1
  • 1
Falmarri
  • 47,727
  • 41
  • 151
  • 191
  • Honestly this is the sort of thing that is probably best asked on numpy-discussion. http://mail.scipy.org/mailman/listinfo/numpy-discussion Most of the numpy devs don't usually check SO. (The only exception I know of is David Cournapeau...) Good luck, at any rate! – Joe Kington Feb 14 '11 at 03:09
  • The issue your having more likely is a python mmap issue, since python mmaps handle all the memory mapping and file closing for numpy memmaps. – kiyo Feb 17 '11 at 14:04
  • Well the odd thing is that python's mmap works fine. I still haven't figured this one out. – Falmarri Feb 17 '11 at 18:05

1 Answers1

2

If your code works fine with the python mmap module, you can use it directly instead of numpy.memmap:

>>> fd = os.open("a", os.O_RDWR)
>>> buffer = mmap.mmap(fd, 0)
>>> surface = np.ndarray((320,240), np.uint16, buffer)

This has the other advantage that you have more control over the memory mapping used.

Now, python's mmap has its own peculiarities. As the source shows, it calls msync on delallocation. Maybe this is where your program hangs? (You might be able to reproduce your issue with buffer.flush(), which also calls msync). Your solution of calling close() first probably works because it circumvents msync!

hrr
  • 1,807
  • 2
  • 21
  • 35
  • Even though this is kinda of old, and I don't have a solution ( I really stopped trying due to other projects and work), python's mmap works regardless of what I do. It's only numpy's memmap that locks up. There's slightly more information on my other unix.stackexchange post http://unix.stackexchange.com/questions/6671/how-do-i-debug-a-kernel-module-in-which-a-null-pointer-appears but not directly relevant. I'm chocking this up to kernel module bug. And since the code I had used is now deprecated, the issue here should be considered closed. – Falmarri Jun 22 '11 at 17:44