Download single files from GitHub in CloudFormation template

Question

Within my CloudFormation template, I would like to download text files from GitHub. When I examine the downloaded files is encoded/encrypted content instead of plain text. It doesn't seem like it should matter, but I have files in both public and private repositories.

Here the code from my CloudFormation template that downloads the files:

           "AWS::CloudFormation::Authentication" : {
                "GithubAccessCredentials" : {
                    "type" : "basic",
                    "username" : { "Ref" : "GitHubLogin" },
                    "password" : { "Ref" : "GitHubPassword" },
                }
            },
            "AWS::CloudFormation::Init" : {
                "configSets" : {
                    "orderedConfig" : [ "yumRepoConfig", "mainConfig" ]
                },
                "yumRepoConfig" : {
                    "files" : {
                        "/etc/yum.repos.d/puppetlabs.repo" : {
                            "source"  : "https://github.com/MyOrganization/Repo/raw/master/provisioning/yum-repo/puppetlabs.repo",
                            "mode"    : "000644",
                            "owner"   : "root",
                            "group"   : "root",
                            "authentication" : "GithubAccessCredentials"
                        },
                        "/tmp/README" : {
                            "source"  : "https://github.com/puppetlabs/puppetlabs-apache/raw/master/README.md",
                            "mode"    : "000644",
                            "owner"   : "root",
                            "group"   : "root"
                        }
                    }
                },
            [...]

I am able to download the files fine using curl:

For the private repo

curl -u 'myGithubLogin' -L -O https://github.com/MyOrganization/Repo/raw/master/provisioning/yum-repo/puppetlabs.repo

For a public repo:

curl -L -O https://github.com/puppetlabs/puppetlabs-apache/raw/master/README.md

What is required to download a valid copy of the files? Also, what command is actually used by cloudFormation to download files (curl, wget,...)?

As mentionned in my message, the downloaded file is encoded/encrypted instead of a plain text. — PapelPincel, Dec 14 '12 at 17:32
I see, the statement that your files were corrupt was a bit buried in the question. Still, I suggest that you try and make your problem very obvious. A little repetition, of the core question is a good thing. — Zoredache, Dec 14 '12 at 18:06

score 2 · Accepted Answer · answered Dec 14 '12 at 17:58

There was an issue with cfn-init that was preventing from properly expanding downloads sent with Content-Encoding: gzip. Upgrading to the last version of cfn-init fixed the issue.

please see this thread : https://forums.aws.amazon.com/thread.jspa?threadID=111736&tstart=0

Download single files from GitHub in CloudFormation template

1 Answers1