0

Since the company I work for will not allow any non-Microsoft released packages from the Azure Marketplace, I need to create a bash script to be able to upload my built/published asset to the Github Enterprise repo. For some reason I always get redirected when I try to get the info of a release tag, be it a specific tag or latest, does not matter.

The following is my script:

set -e
xargs=$(which gxargs || which xargs)

# Validate settings.
[ "$TRACE" ] && set -x

CONFIG=$@

for line in $CONFIG; do
  eval "$line"
done

# Define variables.
GH_API="https://git.[company].com/api/v3"
GH_REPO="$GH_API/repos/[owner]/$(Build.Repository.Name)"
GH_TAGS="$GH_REPO/releases/tags/$(Build.SourceBranchName)"
AUTH="Authorization: token $github_api_token"
WGET_ARGS="--content-disposition --auth-no-challenge --no-cookie"
CURL_ARGS="-LJO#"
tag="$(Build.SourceBranchName)"
filename="BaseRepoName_$(Build.SourceBranchName)_$(Build.BuildId).zip"
echo "tag is: $tag"
echo $AUTH
echo "Repo: $GH_REPO"

if [[ "$tag" == 'LATEST' ]]; then
  GH_TAGS="$GH_REPO/releases/latest"
fi
echo "Tags url:  $GH_TAGS"

echo "Validate token ..."
# Validate token.
curl -o /dev/null -sH "$AUTH" $GH_REPO || { echo "Error: Invalid repo, token or network issue!";          exit 1; }

echo "Get api endpoints"
apiresponse=$(curl -sH "$AUTH" "$GH_API")
echo "API:  $apiresponse"

echo "Read asset tags: curl -sH "$AUTH" $GH_TAGS"
# Read asset tags.
response=$(curl -sH "$AUTH" $GH_TAGS)
echo "Response: $response"

## In case of success, we upload a file
upload=$(echo $succ | grep upload_url)
if [[ $? -eq 0 ]]; then
    echo Release created.
else
    echo Error creating release!
    return
fi

echo "Get the upload url for the given tag"
upload=$(echo $upload | cut -d "\"" -f4 | cut -d "{" -f1)
upload="$upload?name=$filename"

# Upload asset
echo "Uploading asset... "
succ=$(curl -H "Authorization: token $perstok" \
     -H "Content-Type: $(file -b --mime-type $filename)" \
     --data-binary @$filename $upload)

The logs of the Bash task:

2020-03-13T05:53:35.4274045Z ##[section]Starting: Bash Script
2020-03-13T05:53:35.4654068Z ==============================================================================
2020-03-13T05:53:35.4654773Z Task         : Bash
2020-03-13T05:53:35.4655254Z Description  : Run a Bash script on macOS, Linux, or Windows
2020-03-13T05:53:35.4655535Z Version      : 3.163.1
2020-03-13T05:53:35.4656320Z Author       : Microsoft Corporation
2020-03-13T05:53:35.4656864Z Help         : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/bash
2020-03-13T05:53:35.4657312Z ==============================================================================
2020-03-13T05:53:36.4699984Z Generating script.
2020-03-13T05:53:36.4703019Z [command]"C:\Program Files\Git\bin\bash.exe" --noprofile --norc -c pwd
2020-03-13T05:53:36.8704279Z /d/a/_temp
2020-03-13T05:53:36.8755504Z 
2020-03-13T05:53:36.8794400Z ========================== Starting Command Output ===========================
2020-03-13T05:53:36.8801355Z [command]"C:\Program Files\Git\bin\bash.exe" --noprofile --norc /d/a/_temp/c7a40af4-c1f2-4127-a2cf-4aa11ac19e48.sh
2020-03-13T05:53:36.9965374Z which: no gxargs in (/mingw64/bin:/usr/bin:/c/Users/VssAdministrator/bin:/c/hostedtoolcache/windows/Python/3.6.8/x64:/c/hostedtoolcache/windows/Python/3.6.8/x64/Scripts:/c/Program Files/Mercurial:/c/ProgramData/kind:/c/vcpkg:/c/cf-cli:/c/Program Files (x86)/NSIS:/c/Program Files/Mercurial:/c/hostedtoolcache/windows/Boost/1.69.0:/c/Program Files/dotnet:/c/mysql-5.7.21-winx64/bin:/c/Program Files/Java/zulu-8-azure-jdk_8.40.0.25-8.0.222-win_x64/bin:/c/SeleniumWebDrivers/GeckoDriver:/c/Program Files (x86)/sbt/bin:/c/Rust/.cargo/bin:/c/hostedtoolcache/windows/Ruby/2.5.7/x64/bin:/c/Go1.12.7/bin:/bin:/c/hostedtoolcache/windows/Python/3.6.8/x64/Scripts:/c/hostedtoolcache/windows/Python/3.6.8/x64:/c/npm/prefix:/c/Program Files (x86)/Microsoft SDKs/Azure/CLI2/wbin:/c/Program Files/Microsoft MPI/Bin:/c/windows/system32:/c/windows:/c/windows/System32/Wbem:/c/windows/System32/WindowsPowerShell/v1.0:/c/ProgramData/Chocolatey/bin:/c/Program Files/Docker:/c/Program Files/PowerShell/7:/c/Program Files/dotnet:/c/Program Files/Microsoft SQL Server/130/Tools/Binn:/c/Program Files (x86)/Microsoft SQL Server/110/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/120/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/130/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/140/DTS/Binn:/c/Program Files (x86)/Microsoft SQL Server/150/DTS/Binn:/c/Program Files (x86)/Windows Kits/10/Windows Performance Toolkit:/c/Program Files/Microsoft Service Fabric/bin/Fabric/Fabric.Code:/c/Program Files/Microsoft SDKs/Service Fabric/Tools/ServiceFabricLocalClusterManager:/c/Program Files/nodejs:/c/Strawberry/c/bin:/c/Strawberry/perl/site/bin:/c/Strawberry/perl/bin:/cmd:/mingw64/bin:/usr/bin:/c/tools/php:/c/Program Files (x86)/sbt/bin:/c/Program Files (x86)/Subversion/bin:/c/SeleniumWebDrivers/ChromeDriver:/c/SeleniumWebDrivers/EdgeDriver:/c/ProgramData/chocolatey/lib/maven/apache-maven-3.6.3/bin:/c/Program Files/CMake/bin:/c/Program Files/OpenSSL/bin:/c/Users/VssAdministrator/.dotnet/tools:/c/Program Files (x86)/Microsoft SQ)
2020-03-13T05:53:37.0042653Z tag is: MBP_TestTag6
2020-03-13T05:53:37.0043260Z Authorization: token [Edited: secret]
2020-03-13T05:53:37.0043852Z Repo: https://git.[company].com/api/v3/repos/[owner]/[MyTestRepo]
2020-03-13T05:53:37.0044723Z Tags url:  https://git.[company].com/api/v3/repos/[owner]/[MyTestRepo]/releases/tags/MBP_TestTag6
2020-03-13T05:53:37.0045218Z Validate token ...
2020-03-13T05:53:38.5703717Z Get api endpoints
2020-03-13T05:53:38.8038551Z API:  {
2020-03-13T05:53:38.8050142Z   "current_user_url": "https://git.[company].com/api/v3/user",
2020-03-13T05:53:38.8053206Z   "current_user_authorizations_html_url": "https://git.[company].com/settings/connections/applications{/client_id}",
2020-03-13T05:53:38.8060689Z   "authorizations_url": "https://git.[company].com/api/v3/authorizations",
2020-03-13T05:53:38.8064874Z   "code_search_url": "https://git.[company].com/api/v3/search/code?q={query}{&page,per_page,sort,order}",
2020-03-13T05:53:38.8071647Z   "commit_search_url": "https://git.[company].com/api/v3/search/commits?q={query}{&page,per_page,sort,order}",
2020-03-13T05:53:38.8073059Z   "emails_url": "https://git.[company].com/api/v3/user/emails",
2020-03-13T05:53:38.8074445Z   "emojis_url": "https://git.[company].com/api/v3/emojis",
2020-03-13T05:53:38.8075767Z   "events_url": "https://git.[company].com/api/v3/events",
2020-03-13T05:53:38.8077032Z   "feeds_url": "https://git.[company].com/api/v3/feeds",
2020-03-13T05:53:38.8078149Z   "followers_url": "https://git.[company].com/api/v3/user/followers",
2020-03-13T05:53:38.8079345Z   "following_url": "https://git.[company].com/api/v3/user/following{/target}",
2020-03-13T05:53:38.8080572Z   "gists_url": "https://git.[company].com/api/v3/gists{/gist_id}",
2020-03-13T05:53:38.8081548Z   "hub_url": "https://git.[company].com/api/v3/hub",
2020-03-13T05:53:38.8082418Z   "issue_search_url": "https://git.[company].com/api/v3/search/issues?q={query}{&page,per_page,sort,order}",
2020-03-13T05:53:38.8083431Z   "issues_url": "https://git.[company].com/api/v3/issues",
2020-03-13T05:53:38.8084430Z   "keys_url": "https://git.[company].com/api/v3/user/keys",
2020-03-13T05:53:38.8086272Z   "label_search_url": "https://git.[company].com/api/v3/search/labels?q={query}&repository_id={repository_id}{&page,per_page}",
2020-03-13T05:53:38.8091275Z   "notifications_url": "https://git.[company].com/api/v3/notifications",
2020-03-13T05:53:38.8092166Z   "organization_repositories_url": "https://git.[company].com/api/v3/orgs/{org}/repos{?type,page,per_page,sort}",
2020-03-13T05:53:38.8093095Z   "organization_url": "https://git.[company].com/api/v3/orgs/{org}",
2020-03-13T05:53:38.8096487Z   "public_gists_url": "https://git.[company].com/api/v3/gists/public",
2020-03-13T05:53:38.8097620Z   "rate_limit_url": "https://git.[company].com/api/v3/rate_limit",
2020-03-13T05:53:38.8098584Z   "repository_url": "https://git.[company].com/api/v3/repos/{owner}/{repo}",
2020-03-13T05:53:38.8100945Z   "repository_search_url": "https://git.[company].com/api/v3/search/repositories?q={query}{&page,per_page,sort,order}",
2020-03-13T05:53:38.8102239Z   "current_user_repositories_url": "https://git.[company].com/api/v3/user/repos{?type,page,per_page,sort}",
2020-03-13T05:53:38.8104230Z   "starred_url": "https://git.[company].com/api/v3/user/starred{/owner}{/repo}",
2020-03-13T05:53:38.8104831Z   "starred_gists_url": "https://git.[company].com/api/v3/gists/starred",
2020-03-13T05:53:38.8105328Z   "team_url": "https://git.[company].com/api/v3/teams",
2020-03-13T05:53:38.8105772Z   "user_url": "https://git.[company].com/api/v3/users/{user}",
2020-03-13T05:53:38.8106279Z   "user_organizations_url": "https://git.[company].com/api/v3/user/orgs",
2020-03-13T05:53:38.8106871Z   "user_repositories_url": "https://git.[company].com/api/v3/users/{user}/repos{?type,page,per_page,sort}",
2020-03-13T05:53:38.8107491Z   "user_search_url": "https://git.[company].com/api/v3/search/users?q={query}{&page,per_page,sort,order}"
2020-03-13T05:53:38.8108556Z }
2020-03-13T05:53:38.8109344Z Read asset tags: curl -sH Authorization: token [Edited: Secret] https://git.[company].com/api/v3/repos/[owner]/[MyTestRepo]/releases/tags/MBP_TestTag6
2020-03-13T05:53:39.0378180Z Response: {
2020-03-13T05:53:39.0379040Z   "message": "Not Found",
2020-03-13T05:53:39.0379599Z   "documentation_url": "https://developer.github.com/enterprise/2.20/v3/repos/releases/#get-a-release-by-tag-name"
2020-03-13T05:53:39.0380100Z }
2020-03-13T05:53:39.2992296Z 
2020-03-13T05:53:39.3060923Z ##[error]Bash exited with code '1'.
2020-03-13T05:53:39.3120104Z ##[section]Finishing: Bash Script

As you can see from the logs it always fails when calling the api to get the info of the supplied tag (or even latest tag).

Any idea why the system is trying to redirect me?

user5763204
  • 99
  • 1
  • 11
  • Maybe some header you are missing. What you could try is using your browser, go to the destination page with `Inspect mode` on network tab and see the request headers. – Mr.KoopaKiller Mar 13 '20 at 08:58

2 Answers2

0

You should make changes on this line:

GH_API="https://git.[company].com/api/v3"

To

GH_API="https://api.github.com"

enter image description here

Mengdi Liang
  • 17,577
  • 2
  • 28
  • 35
  • Sorry, but I am 99% certain that won't work since there is a very big difference between accessing the GitHub and GitHub Enterprise, the latter which I am working with here. https://git.[company].com/api/v3 is the url to use according to GitHub Enterprise documentation. This is according to the documentation at https://developer.github.com/enterprise/2.20/v3/enterprise-admin/ – user5763204 Mar 17 '20 at 09:44
  • @user5763204, did you ever notice this sentence in the link you shared? `In addition to the GitHub.com API, these GitHub Enterprise-specific endpoints are available`. – Mengdi Liang Mar 17 '20 at 11:11
  • Yes, I have seen that. Since the company I work for has GitHub Enterprise and all repositories are internal and only accessible via the Enterprise server, we can't access anything via GitHub itself. And yes, I have tested this. Querying the releases on my project via the github api (https://api.github.com) brings nothing back, but via the github enterprise api (https://git.[company].com/api/v3) I get the list of releases back. Same goes for authentication and other commands. – user5763204 Mar 17 '20 at 12:33
  • I am beginning to suspect that either the company disabled some api functionality, or there is a bug in the new api. 1) Querying .../releases brings back only pre-releases. 2) ...releases/latest just returns "Not Found" 3) .../releases/[id of pre-release] brings the data of the pre-relase back 4) .../releases/tags brings back a list of all 29 tags (each of which is a release) 5) .../releases/tags/latest brings back "Not Found". So, disabled or bug? I have no idea and nobody here (at the company) knows where to find out ... – user5763204 Mar 17 '20 at 12:45
0

The problem that I experienced has to do with the way Git handles releases. In this case there was a Pre-Release (Release name/Tag: v0.0.1), which is what I got back when querying for all releases. As soon as I deleted this pre-release I got back nothing, but then realized that none of the other 'releases' were ever published.

After publishing a couple of releases I then finally got back a list of published releases. All other commands then worked as expected as well.

So, no sinister problems or bugs. Just some minor mistakes from a DevOps pipeline newbie.

user5763204
  • 99
  • 1
  • 11