Shared Storage/Caching in Drone CI Runs

Question

I'm trying to dynamically generate a client library off an OpenAPI specification as a build artifact in my Drone CI/CD pipeline and publish it to an Artifactory registry. For numerous reasons this has to happen in the service repository that contains the API specification - I can't pop it in another repo that "listens" for changes in the API spec, for example.

I can successfully generate the client and all that jazz, but the issue I have is: My CI pipeline will generate & publish a "new" version of the client on every merge to master.

This is suboptimal, because the service API might not have really changed.

My build step that does the publishing is specified as follows:

  - name: publish-ts-library
    image: node
    environment:
      ARTIFACTORY_USERNAME:
        from_secret: ARTIFACTORY_USERNAME
      ARTIFACTORY_PASSWORD:
        from_secret: ARTIFACTORY_PASSWORD
      RC_RELEASE: false
    commands:
      - bash ./publish-ts-library.sh
    depends_on:
      - build-read-gateway-ts-client
    when:
      branch: master
      event:
        exclude:
        - pull_request

The shell script executed is:

#!/usr/bin/env bash

touch .npmrc
curl -u$ARTIFACTORY_USERNAME:$ARTIFACTORY_PASSWORD https://mycompany.jfrog.io/mycompany/api/npm/auth > .npmrc
echo "registry=https://mycompany.jfrog.io/mycompany/api/npm/npm" >> .npmrc
if [ $RC_RELEASE ]
then
    sed -i '/version/s/[^.]*$/'"0-dev${DRONE_BUILD_NUMBER}\",/" package.json
else
    sed -i '/version/s/[^.]*$/'"0.${DRONE_BUILD_NUMBER}\",/" package.json
fi
npm publish

I know it's pretty gross, apologies.

What's happening is pretty simple: we grab some auth credentials from the secrets in Drone, and pop them in a .npmrc file. Then, if it's an RC release we add dev to the version, else we just bump the version to whatever the drone build number is.

So, my initial thinking was to take a sha1sum of the OpenAPI spec yaml file that I use to generate the library, and compare it to the sha of the "last" generated client - if it matches then no-op instead of npm publish, but the kicker is I have no idea where I can store this SHA, and how to configure drone to have access to this storage.

I'm aware of plugins like http://plugins.drone.io/drillster/drone-volume-cache/ - but they seem decidedly ephemeral, I need to be assured that this SHA will be persisted across builds consistently, and not suddenly "vanish" because a cache gets cleared, thereby forcing a republish + bump of an otherwise identical generated client.

Any ideas?

score 2 · Answer 1 · answered Jun 25 '21 at 18:28

I'm not familiar with drone ci, so I'm making a couple of assumptions here:

git, awk, and wc are available in the context the script is executed.

If it is certain that the workflow will always be based on a git merge, then it is possible to use the diff between the merge's commit and the merge's first parent commit hashes to determine if changes where made to the relevant files.

#!/usr/bin/env bash

# Returns the commit sha of the first parent (merge target) of the last git merge
getMergeCommitBase () {
    git rev-parse HEAD | \
    git rev-list --parents --min-parents=2 -n1 --stdin | \
    awk '{print $2}'
}

exitIfTheseFilesHaveNotChanged () {
    local count
    count="$(git diff "$(getMergeCommitBase)" "$@" | wc -l | awk '{print $1}')"
    if [ "$count" == "0" ]
    then
        echo "no version changes detected"
        exit 1
    else
        echo "changes detected - publishing new version"
    fi
}

# If any of these files have changed, we want to publish a new version
exitIfTheseFilesHaveNotChanged path/to/file.json path/to/directory/

# the rest is your original script

touch .npmrc
curl -u$ARTIFACTORY_USERNAME:$ARTIFACTORY_PASSWORD https://mycompany.jfrog.io/mycompany/api/npm/auth > .npmrc
echo "registry=https://mycompany.jfrog.io/mycompany/api/npm/npm" >> .npmrc
if [ $RC_RELEASE ]
then
    sed -i '/version/s/[^.]*$/'"0-dev${DRONE_BUILD_NUMBER}\",/" package.json
else
    sed -i '/version/s/[^.]*$/'"0.${DRONE_BUILD_NUMBER}\",/" package.json
fi
npm publish

Hope the idea helps.

Shared Storage/Caching in Drone CI Runs

1 Answers1