Include sys.stderr in a Pyinstaller project

Question

One of the subdependencies of my project is transformers. This only happened when I upgraded transformers from 4.16.0 to the latest version 4.25.1. When I try to compile the project with pyinstaller I get the following error:

Traceback (most recent call last):
  File "main.py", line 14, in <module>
  ...
  File "transformers\utils\import_utils.py", line 36, in <module>
  File "transformers\utils\logging.py", line 123, in get_logger
  File "transformers\utils\logging.py", line 86, in _configure_library_root_logger
AttributeError: 'NoneType' object has no attribute 'flush'

Upon further inspection I found the following function in logging.py. It seems that sys.stderr is being set as NoneType for some reason.

def _configure_library_root_logger() -> None:

    global _default_handler

    with _lock:
        if _default_handler:
        _default_handler = logging.StreamHandler()
        _default_handler.flush = sys.stderr.flush  # Error on this line
        ...

This is the file I'm using to compile the project:

# -*- mode: python ; coding: utf-8 -*-
from PyInstaller.utils.hooks import collect_data_files
from PyInstaller.utils.hooks import copy_metadata

datas = []
datas += copy_metadata('tqdm')
datas += copy_metadata('numpy')

a = Analysis(['.main.py'],
             pathex=['.'],
             binaries=[],
             datas=datas,
             hiddenimports=[],
             hookspath=[],
             hooksconfig={},
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=None,
             noarchive=False)
pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher)

exe = EXE(pyz,
          a.scripts, 
          [],
          exclude_binaries=True,
          name='MyApp',
          debug=False,
          bootloader_ignore_signals=False,
          strip=False,
          upx=True,
          console=False,
          disable_windowed_traceback=False,
          target_arch=None,
          codesign_identity=None,
          entitlements_file=None,
          icon="icon.ico")
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas, 
               strip=False,
               upx=True,
               upx_exclude=[],
               name='main')

I have tried setting the paths paramater: pathex=['.', 'path/to/env/Lib/site-packages']. I also tried including it as a hidden import: hiddenimports=['sys', 'sys.stderr']. But none of these seem to work. I know I can just downgrade, but I want to use the latest version.

Setting `console=True` should allow you to access `stderr` and `stdout` — Maurice Meyer, Jan 07 '23 at 09:15
it seems like you need to explicitly set the log handler so that it doesn't try to use the default — Alexander, Jan 07 '23 at 22:31

score 0 · Answer 1 · edited May 21 '23 at 18:37

I found this workaround:

Edit file venv/Lib/site-packages/transformers/utils/logging.py
Comment these lines.

def _configure_library_root_logger() -> None:
    global _default_handler

    with _lock:
        if _default_handler:
            # This library has already configured the library root logger.
            return

        # _default_handler = logging.StreamHandler()  # Set sys.stderr as stream.
        # _default_handler.flush = sys.stderr.flush
        #
        # # Apply our default configuration to the library root logger.
        # library_root_logger = _get_library_root_logger()
        # library_root_logger.addHandler(_default_handler)
        # library_root_logger.setLevel(_get_default_logging_level())
        # library_root_logger.propagate = False

This should disable the logging functionality, but also bypass the problem.

score 0 · Answer 2 · answered Jun 10 '23 at 19:13

0

If you use the win10 system, you could try to run it on the terminal with cmd python main.py. I ran the same code on Ubuntu and the same transformers version was successful. So I run it on the terminal (win10 shell), rather than on Pycharm enviroment.

answered Jun 10 '23 at 19:13

miller.fu

11
3

This does not provide an answer to the question. Once you have sufficient [reputation](https://stackoverflow.com/help/whats-reputation) you will be able to [comment on any post](https://stackoverflow.com/help/privileges/comment); instead, [provide answers that don't require clarification from the asker](https://meta.stackexchange.com/questions/214173/why-do-i-need-50-reputation-to-comment-what-can-i-do-instead). - [From Review](/review/late-answers/34527815) – dpapadopoulos Jun 16 '23 at 10:25

Include sys.stderr in a Pyinstaller project

2 Answers2