Performance issue with my code after instantiating QApplication

Question

I'm having an issue with an IO module that I'm developing, to access data in a file format (ASCII based).

It performs OK when it is used in a regular python script, but it is 5 times slower when I try to use it in a PyQt widget.

This is the simplified code that shows the issue:

from silx.io import spech5
import time
from PyQt4.QtGui import QApplication

with spech5.SpecH5("../data/mesh_and_mca.dat") as f:
    start = time.time()
    a = f["1.1/measurement/mca_0/data"]
    end = time.time()
    print("Simple access in python ", end - start)

app = QApplication([])

with spech5.SpecH5("../data/mesh_and_mca.dat") as f:
    start = time.time()
    a = f["1.1/measurement/mca_0/data"]
    end = time.time()
    print("Access after initializing QApplication ", end - start)

The first block takes 10 seconds, while the second identical block, after app = QApplication([]) takes 50 seconds. I can repeat the code several before and after the app instantiation, and all the blocks before initializing the app are relatively fast while all the blocks after it are slow.

The line a = f["1.1/measurement/mca_0/data"] causes the data file to be accessed 78026 times via a C function wrapped with Cython, to read a single "line" each time. The returned objects is a numpy array of shape (78026, 1024) and dtype float64.

Does anyone have a clue about what could cause this? Are there known issues/interferences between Qt and wrapped C code?

Your OS and details of your python environment might be helpful here — user3419537, Aug 02 '17 at 15:04
Debian 8, both Python 2.7.9 and Python 3.4.2 show the issue. I also tested PyQt4 and PyQt5, no difference. — PiRK, Aug 03 '17 at 06:35
The issue remains the same if I replace `QApplication` with `QCoreApplication`. The event loop is somhow slowing down my module. — PiRK, Aug 03 '17 at 06:40
[This](https://stackoverflow.com/questions/14504630/qapplication-instance-causing-python-shell-to-be-sluggish) doesn't help? (I suspect it won't but it might be worth a go) — DavidW, Aug 03 '17 at 20:43
@PiRK. But your script doesn't start an event loop, so that is not relevant. — ekhumoro, Aug 04 '17 at 19:07
Very strange bug. The problem is that we cannot recreate it because it uses the proprietary library silx from you. Maybe you can extract the important part of the spech5 module and make a self-contained example showing the problem? Otherwise I would be tempted to vote for closing the question because there is not much hope to get a definite answer. — NoDataDumpNoContribution, Aug 09 '17 at 10:16
silx is open source and available on pypi (`pip install silx --user`). But i need to change the example work with publicly available data. I will do this when i'm back to work, on Tuesday 17. — PiRK, Aug 10 '17 at 11:43

PiRK · Answer 1 · 2017-08-24T07:26:02.207

A colleague finally found the root cause!

The underlying C library does some extra work if the locale is not the 'C' default one (see https://github.com/silx-kit/silx/blob/master/silx/io/specfile/src/locale_management.c). And QApplication sets the locale to 'en_US.UTF-8'.

So I can replicate the issue without QApplication:

import locale

with ...:
    do the work

locale.setlocale(locale.LC_NUMERIC, 'en_US.UTF-8')

with ...:  
    same work, much slower

Sorry for this very specific question. I hope the information about QApplication([]) modifying the locale can help someone in the future.

Performance issue with my code after instantiating QApplication

1 Answers1