0

I am using anaconda Powershell Prompt in Win10. UnicodeDecodeError was reported when installing package with pip (version: 21.2.4). I found this error would reported with the kernel Python 3.6.7 and Python 3.9.7.Both the installation and the upgrade failed and reported the same error. I've tried the following command.

 pip install simplejson

 pip install simplejson --upgrade

The package can be anything and report the same bug.

Same error was reported using the Windows CMD. I tried the following command using Windows CMD.

py -m pip install simplejson

py -m pip install --upgrade pip

Here is the error report I got.

ERROR: Exception:
Traceback (most recent call last):
File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\cli\base_command.py", line 173, in _main
status = self.run(options, args)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\cli\req_command.py", line 203, in wrapper
    return func(self, options, args)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\commands\install.py", line 315, in run
    requirement_set = resolver.resolve(
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\resolution\resolvelib\resolver.py", line 94, in resolve
    result = self._result = resolver.resolve(
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 472, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 341, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_vendor\resolvelib\resolvers.py", line 172, in _add_to_criteria
    if not criterion.candidates:
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_vendor\resolvelib\structs.py", line 151, in __bool__
    return bool(self._sequence)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 140, in __bool__
    return any(self)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 128, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\resolution\resolvelib\found_candidates.py", line 29, in _iter_built
    for version, func in infos:
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\resolution\resolvelib\factory.py", line 272, in iter_index_candidate_infos
    result = self._finder.find_best_candidate(
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\index\package_finder.py", line 851, in find_best_candidate
    candidates = self.find_all_candidates(project_name)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\index\package_finder.py", line 798, in find_all_candidates
    page_candidates = list(page_candidates_it)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\index\sources.py", line 134, in page_candidates
    yield from self._candidates_from_page(self._link)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\index\package_finder.py", line 758, in process_project_url
    html_page = self._link_collector.fetch_page(project_url)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\index\collector.py", line 490, in fetch_page
    return _get_html_page(location, session=self.session)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\index\collector.py", line 400, in _get_html_page
    resp = _get_html_response(url, session=session)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\index\collector.py", line 115, in _get_html_response
    resp = session.get(
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_vendor\requests\sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\network\session.py", line 454, in request
    return super().request(method, url, *args, **kwargs)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_vendor\requests\sessions.py", line 528, in request
    prep = self.prepare_request(req)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_vendor\requests\sessions.py", line 456, in prepare_request
    p.prepare(
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_vendor\requests\models.py", line 320, in prepare
    self.prepare_auth(auth, url)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_vendor\requests\models.py", line 556, in prepare_auth
    r = auth(self)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\network\auth.py", line 208, in __call__
    url, username, password = self._get_url_and_credentials(req.url)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\network\auth.py", line 180, in _get_url_and_credentials
    username, password = self._get_new_credentials(original_url)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_internal\network\auth.py", line 145, in _get_new_credentials
    netrc_auth = get_netrc_auth(original_url)
  File "E:\Program Files\Anaconda3\lib\site-packages\pip\_vendor\requests\utils.py", line 216, in get_netrc_auth
    _netrc = netrc(netrc_path).authenticators(host)
  File "E:\Program Files\Anaconda3\lib\netrc.py", line 30, in __init__
    self._parse(file, fp, default_netrc)
  File "E:\Program Files\Anaconda3\lib\netrc.py", line 39, in _parse
    toplevel = tt = lexer.get_token()
  File "E:\Program Files\Anaconda3\lib\shlex.py", line 109, in get_token
    raw = self.read_token()
  File "E:\Program Files\Anaconda3\lib\shlex.py", line 140, in read_token
    nextchar = self.instream.read(1)
UnicodeDecodeError: 'gbk' codec can't decode byte 0xff in position 0: illegal multibyte sequence

I have reinstalled the pip, Anaconda, and the python interpreter. It still failed. I think it has nothing to do with the pip. Something went wrong with the input stream. The default decode format is "GBK". I tried to change decode format to "utf-8" with the following command. It failed again with the same error.

chcp 65001

Thanks the suggestion from @Mobfi, I tried other similar commands, which return the same error notification.

chcp 437
chcp 850

Thanks the suggestion from @BoarGules, I start python with utf-8 decoder and tried to install/upgrade package with pip.

py -X utf8

I started the python console in powershell. import pip pip.main(['install', 'simplejson'])

Besides the similar traceback, I got another new traceback:

*same as aforementioned traceback*
File "E:\Program Files\Anaconda3\lib\codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

Anybody can help? Any potential solution, clues or similar experiences is welcomed.

  • The `0xff in position 0` is probably a byte order mark. So the file is probably UTF-16 though under Windows it is possibly also UTF-8. Your message indicates that the code is expecting the file to be in `gbk`. Python opens text files using the encoding `locale.getpreferredencoding(False)` and on Windows that value is installation dependent. Futzing with `chcp` won't change that. That command affects only the current console setting. You can override what `locale.getpreferredencoding(False)` returns by starting Python with `-X utf8`. – BoarGules Mar 01 '22 at 09:33
  • Thanks @BoarGules, it returns similar error "UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte". – yonsen liu Mar 01 '22 at 12:17

0 Answers0