Python multiprocess plot issue in Mac OSX Sierra - exception type EXC_BAD_INSTRUCTION

Question

I've developed a GitHub repo with an example of a multiprocess script in python2.7 for plotting. Basically, the script allocates one process to generate the data, and such data are passed via a queue to a second process that is responsible to generate the plot. When I run both the examples 'multiproc_processIPC.py' or 'multiproc_poolQUEUE.py' I get the following error. Note that this error came up AFTER upgrading my mac to OSX Sierra. Has anyone a clue of the origin of this issue? Any help will be highly appreciated

===== ERROR REPORT =====

Process: python2.7 [82890] Path: /Users/USER/*/python Identifier: python2.7 Version: ??? Code Type: X86-64 (Native) Parent Process: python2.7 [82883] Responsible: python2.7 [82890] User ID: 501

Date/Time: 2016-12-04 22:34:29.874 +0100 OS Version: Mac OS X 10.12.1 (16B2555) Report Version: 12 Anonymous UUID: 689587A6-D759-0F81-671D-1079942B7C42

Time Awake Since Boot: 9000 seconds

System Integrity Protection: enabled

Crashed Thread: 0 Dispatch queue: com.apple.main-thread

Exception Type: EXC_BAD_INSTRUCTION (SIGILL) Exception Codes: 0x0000000000000001, 0x0000000000000000 Exception Note: EXC_CORPSE_NOTIFY

Termination Signal: Illegal instruction: 4 Termination Reason: Namespace SIGNAL, Code 0x4 Terminating Process: exc handler [0]

Application Specific Information: BUG IN CLIENT OF LIBDISPATCH: _dispatch_main_queue_callback_4CF called from the wrong thread crashed on child side of fork pre-exec

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 libdispatch.dylib 0x00007fff8f9ddd21 _dispatch_main_queue_callback_4CF + 1291 1 com.apple.CoreFoundation 0x00007fff7a97abe9 CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE + 9 2 com.apple.CoreFoundation 0x00007fff7a93c00d __CFRunLoopRun + 2205 3 com.apple.CoreFoundation 0x00007fff7a93b514 CFRunLoopRunSpecific + 420 4 com.apple.HIToolbox 0x00007fff79ed8fbc RunCurrentEventLoopInMode + 240 5 com.apple.HIToolbox 0x00007fff79ed8df1 ReceiveNextEventCommon + 432 6 com.apple.HIToolbox 0x00007fff79ed8c26 _BlockUntilNextEventMatchingListInModeWithFilter + 71 7 com.apple.AppKit 0x00007fff785c2b79 _DPSNextEvent + 1093 8 com.apple.AppKit 0x00007fff78cd81c3 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1637 9 Tk 0x0000000104b01285 0x104a47000 + 762501 10 Tk 0x0000000104b0171f 0x104a47000 + 763679 11 Tcl 0x00000001049d75f8 Tcl_DoOneEvent + 317 12 Tk 0x0000000104a55932 0x104a47000 + 59698 13 Tcl 0x00000001049695b7 0x10495b000 + 58807 14 Tcl 0x0000000104969e46 Tcl_EvalObjv + 59 15 _tkinter.so 0x00000001037dfe39 Tkapp_Call + 617 16 libpython2.7.dylib 0x00000001000a3302 PyEval_EvalFrameEx + 3730 17 libpython2.7.dylib 0x00000001000ac913 fast_function + 179 18 libpython2.7.dylib 0x00000001000a338a PyEval_EvalFrameEx + 3866 19 libpython2.7.dylib 0x00000001000ac7dc PyEval_EvalCodeEx + 2092 20 libpython2.7.dylib 0x00000001000ac988 fast_function + 296 21 libpython2.7.dylib 0x00000001000a338a PyEval_EvalFrameEx + 3866 22 libpython2.7.dylib 0x00000001000ac7dc PyEval_EvalCodeEx + 2092 23 libpython2.7.dylib 0x00000001000ac988 fast_function + 296 24 libpython2.7.dylib 0x00000001000a338a PyEval_EvalFrameEx + 3866 25 libpython2.7.dylib 0x00000001000ac7dc PyEval_EvalCodeEx + 2092 26 libpython2.7.dylib 0x0000000100035c1b function_call + 347 27 libpython2.7.dylib 0x000000010000c761 PyObject_Call + 97 28 libpython2.7.dylib 0x00000001000a3cea PyEval_EvalFrameEx + 6266 29 libpython2.7.dylib 0x00000001000ac7dc PyEval_EvalCodeEx + 2092 30 libpython2.7.dylib 0x00000001000ac988 fast_function + 296 31 libpython2.7.dylib 0x00000001000a338a PyEval_EvalFrameEx + 3866 32 libpython2.7.dylib 0x00000001000ac7dc PyEval_EvalCodeEx + 2092 33 libpython2.7.dylib 0x0000000100035c1b function_call + 347 34 libpython2.7.dylib 0x000000010000c761 PyObject_Call + 97 35 libpython2.7.dylib 0x000000010001e7d7 instancemethod_call + 503 36 libpython2.7.dylib 0x000000010000c761 PyObject_Call + 97 37 libpython2.7.dylib 0x000000010006673a slot_tp_init + 106 38 libpython2.7.dylib 0x0000000100062465 type_call + 229 39 libpython2.7.dylib 0x000000010000c761 PyObject_Call + 97 40 libpython2.7.dylib 0x00000001000a35a5 PyEval_EvalFrameEx + 4405 41 libpython2.7.dylib 0x00000001000ac7dc PyEval_EvalCodeEx + 2092 42 libpython2.7.dylib 0x0000000100035c1b function_call + 347 43 libpython2.7.dylib 0x000000010000c761 PyObject_Call + 97 44 libpython2.7.dylib 0x000000010001e7d7 instancemethod_call + 503 45 libpython2.7.dylib 0x000000010000c761 PyObject_Call + 97 46 libpython2.7.dylib 0x000000010006673a slot_tp_init + 106 47 libpython2.7.dylib 0x0000000100062465 type_call + 229 48 libpython2.7.dylib 0x000000010000c761 PyObject_Call + 97 49 libpython2.7.dylib 0x00000001000a3cea PyEval_EvalFrameEx + 6266 50 libpython2.7.dylib 0x00000001000ac913 fast_function + 179 51 libpython2.7.dylib 0x00000001000a338a PyEval_EvalFrameEx + 3866 52 libpython2.7.dylib 0x00000001000ac913 fast_function + 179 53 libpython2.7.dylib 0x00000001000a338a PyEval_EvalFrameEx + 3866 54 libpython2.7.dylib 0x00000001000ac7dc PyEval_EvalCodeEx + 2092 55 libpython2.7.dylib 0x0000000100035c1b function_call + 347 56 libpython2.7.dylib 0x000000010000c761 PyObject_Call + 97 57 libpython2.7.dylib 0x000000010001e7d7 instancemethod_call + 503 58 libpython2.7.dylib 0x000000010000c761 PyObject_Call + 97 59 libpython2.7.dylib 0x000000010006673a slot_tp_init + 106 60 libpython2.7.dylib 0x0000000100062465 type_call + 229 61 libpython2.7.dylib 0x000000010000c761 PyObject_Call + 97 62 libpython2.7.dylib 0x00000001000a35a5 PyEval_EvalFrameEx + 4405 63 libpython2.7.dylib 0x00000001000ac913 fast_function + 179 64 libpython2.7.dylib 0x00000001000a338a PyEval_EvalFrameEx + 3866 65 libpython2.7.dylib 0x00000001000ac7dc PyEval_EvalCodeEx + 2092 66 libpython2.7.dylib 0x00000001000ac856 PyEval_EvalCode + 54 67 libpython2.7.dylib 0x00000001000cb3d4 PyRun_FileExFlags + 164 68 libpython2.7.dylib 0x00000001000cc709 PyRun_SimpleFileExFlags + 409 69 libpython2.7.dylib 0x00000001000e01ca Py_Main + 2938 70 python 0x0000000100000f14 start + 52

Thread 1: 0 libsystem_kernel.dylib 0x00007fff8fb354e6 __workq_kernreturn + 10 1 libsystem_pthread.dylib 0x00007fff8fc1d622 _pthread_wqthread + 1023 2 libsystem_pthread.dylib 0x00007fff8fc1d211 start_wqthread + 13

Thread 2: 0 libsystem_kernel.dylib 0x00007fff8fb354e6 __workq_kernreturn + 10 1 libsystem_pthread.dylib 0x00007fff8fc1d622 _pthread_wqthread + 1023 2 libsystem_pthread.dylib 0x00007fff8fc1d211 start_wqthread + 13

Thread 3: 0 libsystem_kernel.dylib 0x00007fff8fb354e6 __workq_kernreturn + 10 1 libsystem_pthread.dylib 0x00007fff8fc1d7b5 _pthread_wqthread + 1426 2 libsystem_pthread.dylib 0x00007fff8fc1d211 start_wqthread + 13

Thread 4: 0 libsystem_kernel.dylib 0x00007fff8fb34f4e __select + 10 1 Tcl 0x0000000104a08271 0x10495b000 + 709233 2 libsystem_pthread.dylib 0x00007fff8fc1daab _pthread_body + 180 3 libsystem_pthread.dylib 0x00007fff8fc1d9f7 _pthread_start + 286 4 libsystem_pthread.dylib 0x00007fff8fc1d221 thread_start + 13

Thread 5:: com.apple.NSEventThread 0 libsystem_kernel.dylib 0x00007fff8fb2d41a mach_msg_trap + 10 1 libsystem_kernel.dylib 0x00007fff8fb2c867 mach_msg + 55 2 libsystem_kernel.dylib 0x00007fff8fb21ed1 _kernelrpc_mach_port_set_attributes + 150 3 libdispatch.dylib 0x00007fff8f9e7739 _dispatch_runloop_queue_handle_init + 171 4 libdispatch.dylib 0x00007fff8f9d84b7 _dispatch_runloop_root_queue_create_4CF + 133 5 com.apple.CoreFoundation 0x00007fff7a9091e6 __CFRunLoopFindMode + 502 6 com.apple.CoreFoundation 0x00007fff7a909889 CFRunLoopAddSource + 201 7 com.apple.HIToolbox 0x00007fff79ed9897 AdjustCGSSourceInRunLoop(void ()(__CFRunLoop, __CFRunLoopSource*, __CFString const*), __CFRunLoop*, __CFRunLoopSource*) + 180 8 com.apple.HIToolbox 0x00007fff79ed97bb AddCGSSourceToRunLoop(__CFRunLoop*) + 78 9 com.apple.HIToolbox 0x00007fff79eecc52 _BeginEventReceiptOnThread + 181 10 com.apple.AppKit 0x00007fff7870f4f8 _NSEventThread + 37 11 libsystem_pthread.dylib 0x00007fff8fc1daab _pthread_body + 180 12 libsystem_pthread.dylib 0x00007fff8fc1d9f7 _pthread_start + 286 13 libsystem_pthread.dylib 0x00007fff8fc1d221 thread_start + 13

Thread 0 crashed with X86 Thread State (64-bit): rax: 0x0000000000000307 rbx: 0x00000001003d93f0 rcx: 0x00007fff8f9f9530 rdx: 0x0000000000000307 rdi: 0x0000000000000001 rsi: 0x0000000000000006 rbp: 0x00007fff5fbfa160 rsp: 0x00007fff5fbfa0f0 r8: 0x0000000000000040 r9: 0x00000001003d9408 r10: 0xffffffffffffffff r11: 0x0000000000012068 r12: 0x00000001003d93e0 r13: 0x00000001003d94a0 r14: 0x00007fff986f5a80 r15: 0x0000000102076898 rip: 0x00007fff8f9ddd21 rfl: 0x0000000000010202 cr2: 0x00007fff9db1224b

Logical CPU: 3 Error Code: 0x00000000 Trap Number: 6

Binary Images: 0x100000000 - 0x100000ff7 +python (???) /Users/USER//python 0x100003000 - 0x100146ff7 +libpython2.7.dylib (2.7) <87FF0E63-058F-3190-A882-8E25F3D889C7> /Users/USER//libpython2.7.dylib 0x1002db000 - 0x1002dcff7 +_locale.so (???) /Users/USER//_locale.so 0x1002e0000 - 0x1002e6ff7 +itertools.so (???) /Users/USER//itertools.so <<<<<< LINES OMITTED FOR BREVITY >>>>>> 0x7fff8fc2b000 - 0x7fff8fc32fff libsystem_symptoms.dylib (532.1.1) <8FB7CA37-79EF-3651-B5B9-B5E1E0947067> /usr/lib/system/libsystem_symptoms.dylib 0x7fff8fc33000 - 0x7fff8fc53ff7 libsystem_trace.dylib (518.20.8) /usr/lib/system/libsystem_trace.dylib 0x7fff8fc54000 - 0x7fff8fc59ffb libunwind.dylib (35.3) <9F7C2AD8-A9A7-3DE4-828D-B0F0F166AAA0> /usr/lib/system/libunwind.dylib 0x7fff8fc5a000 - 0x7fff8fc83ff7 libxpc.dylib (972.20.3) <85EB25FD-218F-38EE-9E69-391CC8EBE6C5> /usr/lib/system/libxpc.dylib

External Modification Summary: Calls made by other processes targeting this process: task_for_pid: 0 thread_create: 0 thread_set_state: 0 Calls made by this process: task_for_pid: 0 thread_create: 0 thread_set_state: 0 Calls made by all processes on this machine: task_for_pid: 542465 thread_create: 0 thread_set_state: 0

VM Region Summary: ReadOnly portion of Libraries: Total=254.5M resident=0K(0%) swapped_out_or_unallocated=254.5M(100%) Writable regions: Total=96.5M written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=96.5M(100%)
                            VIRTUAL   REGION 
REGION TYPE SIZE COUNT (non-coalesced) =========== ======= ======= Activity Tracing 256K 2 CoreUI image file 144K 3 Dispatch continuations 8192K 2 Kernel Alloc Once 8K 2 MALLOC 61.8M 54 MALLOC guard page 32K 7 Memory Tag 242 12K 2 STACK GUARD 56.0M 7 Stack 10.1M 7 VM_ALLOCATE 48K 7 __DATA 21.0M 292 __IMAGE 528K 2 __LINKEDIT 115.0M 66 __TEXT 139.5M 288 __UNICODE 556K 2 mapped file 43.2M 9 shared memory 16.3M 12 =========== ======= ======= TOTAL 472.4M 747

Model: MacBookPro11,1, BootROM MBP111.0138.B17, 2 processors, Intel Core i5, 2,4 GHz, 8 GB, SMC 2.16f68 Graphics: Intel Iris, Intel Iris, Built-In Memory Module: BANK 0/DIMM0, 4 GB, DDR3, 1600 MHz, 0x80AD, 0x484D54343531533641465238412D50422020 Memory Module: BANK 1/DIMM0, 4 GB, DDR3, 1600 MHz, 0x80AD, 0x484D54343531533641465238412D50422020 AirPort: spairport_wireless_card_type_airport_extreme (0x14E4, 0x112), Broadcom BCM43xx 1.0 (7.21.171.47.1a8) Bluetooth: Version 5.0.1f7, 3 services, 17 devices, 1 incoming serial ports Network Service: AirPort, AirPort, en0 Serial ATA Device: APPLE SSD SM0256F, 251 GB USB Device: USB 3.0 Bus USB Device: Apple Internal Keyboard / Trackpad USB Device: BRCM20702 Hub USB Device: Bluetooth USB Host Controller Thunderbolt Bus: MacBook Pro, Apple Inc., 17.2

My guess is that you are running into issues with interactions between multiprocess and GUI code. Which mpl backend are you using? If the goal is to save files I suggest using one of the non-interactive backends. See http://matplotlib.org/faq/usage_faq.html#what-is-a-backend — tacaswell, Dec 05 '16 at 03:21
the default backend. However, if I use 'import matplotlib matplotlib.use('MacOSX')' Then I see no plot. Everything was properly working before updating to OSX Sierra, and now I've been fighting with this issue for a week... — Maximiliano Cgb, Dec 09 '16 at 20:01

score 3 · Answer 1 · answered Mar 28 '17 at 13:01

3

Try setting the "start method" to "spawn" instead of the default of "fork". See https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods for further details.

This worked for me on a similar problem.

answered Mar 28 '17 at 13:01

Neil

1,754
2
17
30

score 0 · Answer 2 · answered Dec 09 '16 at 20:37

Ok, more info: If I run a first process from terminal, whose script 'visl.py' is:

#!/usr/bin/env python
from multiprocessing import Process
import sys
sys.path.insert(0, './lib')
import visualizer

visualizer.Visualizer()

and then a second process from a different terminal instance, whose script 'logr.py' is:

#!/usr/bin/env python
from multiprocessing import Process
import sys
sys.path.insert(0, './lib')
import logger


args_logger = {'sampling_time_ms': 5, 'plotting_time_ms': 100}
logger.Logger(args_logger)

It works. Then: how to run two python processes from a single script?

BTW: I solved the issue with a work-around. Apparently there are some issues with the multi process management in Python. Now Ruby is used to launch and handle two different processes. It's quite an horrible solution I know, but is the better idea I came up with (I'm a hobbyist, not a professional programmer). The solution has been posted in the GitHub repo. — Maximiliano Cgb, Dec 10 '16 at 17:48

Python multiprocess plot issue in Mac OSX Sierra - exception type EXC_BAD_INSTRUCTION

2 Answers2