Publishing package to PyPi from Azure Devops hangs at twine upload

Question

I want to publish a built .whl package to PyPi.org from our Azure Devops release pipeline, but the script twine upload keeps hanging without error, completion or time out. So the actual upload of our package (which is extremely small) does not work.

This is how it is set up:

The yaml build:

trigger:
- master
pr: none
pool:
  vmImage: 'ubuntu-latest'

steps:
- task: UsePythonVersion@0
  inputs:
    versionSpec: '3.7'
    addToPath: true
    architecture: 'x64'

- script: python -m pip install --upgrade pip setuptools wheel
  displayName: 'Install tools'

- script: pip install -r src/requirements.txt
  displayName: 'Install requirements'

- script: |    
    python src/setup.py bdist_wheel 
  displayName: 'Artifact creation'

- script: python -m pip install --upgrade twine
  displayName: 'Install Twine'

- task: TwineAuthenticate@1
  inputs:
    pythonUploadServiceConnection: 'AzureML PyPi feed'

- script: |
   python -m twine upload --config-file $(PYPIRC_PATH) dist/*.whl
  displayName: 'Publish to PyPi through Twine'

The service connection:

The authentication method is specified by Authentication Token.
Python repository url for upload: https://upload.pypi.org/legacy
EndpointName: azure-pypi
Service Connection name: AzureML PyPi feed
[X] Grant access permission to all pipelines

The release logs (most relevant parts):

TwineAuthenticate step:

Starting: TwineAuthenticate
==============================================================================
Task         : Python twine upload authenticate
Description  : Authenticate for uploading Python distributions using twine. Add '-r FeedName/EndpointName --config-file $(PYPIRC_PATH)' to your twine upload command. For feeds present in this organization, use the feed name as the repository (-r). Otherwise, use the endpoint name defined in the service connection.
Version      : 1.165.0
Author       : Microsoft Corporation
Help         : https://learn.microsoft.com/azure/devops/pipelines/tasks/package/twine-authenticate
==============================================================================
75c64e89-xxxxxredactedxxxxx exists true
Adding authentication to configuration for registry azure-pypi
Successfully added auth for 0 internal feed and 1 external endpoint.
Finishing: TwineAuthenticate

Publishing step:

Starting: Publish to PyPi through Twine
==============================================================================
Task         : Command line
Description  : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version      : 2.164.0
Author       : Microsoft Corporation
Help         : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
Script contents:
python -m twine upload --config-file /home/vsts/work/_temp/twineAuthenticate/Wv8nMR/.pypirc dist/*.whl
========================== Starting Command Output ===========================
/bin/bash --noprofile --norc /home/vsts/work/_temp/0d772e31-148f-4e3c-999b-b2a43a02b287.sh
Uploading distributions to https://upload.pypi.org/legacy/

This keeps unchanged for more than 30 minutes, so I just cancel my release without the package being deployed. Any idea on this?

I have a similar problem, but I'm trying to publish the package to Azure DevOps Artifacts. Do you know how to generate the token in this case? Thanks in advance — walzer91, May 19 '22 at 09:19

Sam Vanhoutte · Accepted Answer · 2020-04-18T12:13:32.883

I have a temporary workaround for now:

I skipped the usage of a config file and executed the following code instead:

- script: |
   python -m twine upload --skip-existing --verbose -p $(pypi-api-token) -u __token__ --repository $(pypi-project-name) --repository-url https://upload.pypi.org/legacy/ dist/*.whl
  displayName: 'Publish to PyPi through Twine'

And there, I noticed a bit more (and better) exception logging, that pointed me out two things:

The token I was using for my config file was set to a project scope with a different name than the project I specified in my Service Connection
The repository url really has to end with the trailing forward slash, because otherwise you get a RedirectDetected exception.

Based on the above findings, I updated my yaml snippet to the following:

- script: |
   python -m twine upload --skip-existing --verbose --repository $(pypi-project-name) --config-file $(PYPIRC_PATH) dist/*.whl
  displayName: 'Publish to PyPi through Twine'

But when executing this, I now get the following exception message in the build:

Generating script.
Script contents:
python -m twine upload --skip-existing --verbose --repository arcus-azureml --config-file /home/vsts/work/_temp/twineAuthenticate/2QGKVH/.pypirc dist/*.whl
========================== Starting Command Output ===========================
/bin/bash --noprofile --norc /home/vsts/work/_temp/7605dac9-5fa9-4856-94af-e938018278a5.sh
Uploading distributions to https://upload.pypi.org/legacy/
Uploading arcus_azureml-0.0.2-py3-none-any.whl

  0%|          | 0.00/5.80k [00:00<?, ?B/s]
100%|██████████| 5.80k/5.80k [00:00<00:00, 52.4kB/s]HTTPError: 403 Client Error: Invalid or non-existent authentication information. See https://pypi.org/help/#invalid-auth for details for url: https://upload.pypi.org/legacy/

Content received from server:
<html>
 <head>
  <title>403 Invalid or non-existent authentication information. See https://pypi.org/help/#invalid-auth for details</title>
 </head>
 <body>
  <h1>403 Invalid or non-existent authentication information. See https://pypi.org/help/#invalid-auth for details</h1>
  Access was denied to this resource.<br/><br/>
Invalid or non-existent authentication information. See https://pypi.org/help/#invalid-auth for details

I also outputted the contents of the pypirc file, using the CAT command and the contents of the file are like this:

[distutils]
index-servers=arcus-azureml 
[arcus-azureml]
repository=https://upload.pypi.org/legacy/
username=build
password=***

Updated solution

So, the fix to have the 403 arranged was to change my Service Connection to use 'Username and Password' as authentication method , instead of Authentication Token, with the following settings:

Username: __token__
Password: The actual api token as password

After performing this, everything was working the way I wanted it.

Lifesaver. Thank you so much. The doc for Azure devops is absolutely awful. For an "official" source about the authentication trick with `__token__` as username, see https://github.com/microsoft/azure-pipelines-tasks/issues/11719. I had another issue. In my service connection, I had set EndpointName to my package name. Setting it to pypi did the job. Don't ask. — Jérôme, Nov 06 '20 at 20:16

score 0 · Answer 2 · answered Sep 26 '22 at 20:12

In my case, I needed to publish to testpypi. My AzureDevops twine update kept hanging on the "Querying keyring for username" regardless of a service connection with Authorization Token or Username and Password. Solution was to set the Python repository url to https://test.pypi.org/legacy/ rather than https://test.pypi.org. Once that was solved, I too required the service connection to be Username and Password, with username set to __token__

Publishing package to PyPi from Azure Devops hangs at twine upload

The yaml build:

The service connection:

The release logs (most relevant parts):

2 Answers2

Updated solution