API Gateway HTTP client request with IAM auth with Go

Question

Hello StackOverflow AWS Gophers,

I'm implementing a CLI with the excellent cobra/viper packages from spf13. We have an Athena database fronted by an API Gateway endpoint, which authenticates with IAM.

That is, in order to interact with its endpoints by using Postman, I have to define AWS Signature as Authorization method, define the corresponding AWS id/secret and then in the Headers there will be X-Amz-Security-Token and others. Nothing unusual, works as expected.

Since I'm new to Go, I was a bit shocked to see that there are no examples to do this simple HTTP GET request with the aws-sdk-go itself... I'm trying to use the shared credentials provider (~/.aws/credentials), as demonstrated for the S3 client Go code snippets from re:Invent 2015:

req := request.New(nil)

How can I accomplish this seemingly easy feat in 2019 without having to resort to self-cooked net/http and therefore having to manually read ~/.aws/credentials or worse, go with os.Getenv and other ugly hacks?

Any Go code samples interacting as client would be super helpful. No Golang Lambda/server examples, please, there's plenty of those out there.

Answer 1

The solution below uses aws-sdk-go-v2 https://github.com/aws/aws-sdk-go-v2

// A AWS SDK session is created because the HTTP API is secured using a
// IAM authorizer. As such, we need AWS client credentials and a
// session to properly sign the request.
cfg, err := external.LoadDefaultAWSConfig(
    external.WithSharedConfigProfile(profile),
)
if err != nil {
    fmt.Println("unable to create an AWS session for the provided profile")
    return
}


req, _ := http.NewRequest(http.MethodGet, "", nil)
req = req.WithContext(ctx)
signer := v4.NewSigner(cfg.Credentials)
_, err = signer.Sign(req, nil, "execute-api", cfg.Region, time.Now())
if err != nil {
    fmt.Printf("failed to sign request: (%v)\n", err)
    return
}

res, err := http.DefaultClient.Do(req)
if err != nil {
    fmt.Printf("failed to call remote service: (%v)\n", err)
    return
}

defer res.Body.Close()
if res.StatusCode != 200 {
    fmt.Printf("service returned a status not 200: (%d)\n", res.StatusCode)
    return
}

Answer 2

Unfortunately, it seems that the library has been updated since the accepted answer was written and the solution no longer is the same. After some trial and error, this appears to be the more current method of handling the signing (using https://pkg.go.dev/github.com/aws/aws-sdk-go-v2):

import (
    "context"
    "net/http"
    "time"

    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/aws/signer/v4"
)

func main() {
    // Context is not being used in this example.
    cfg, err := config.LoadDefaultConfig(context.TODO())

    if err != nil {
        // Handle error.
    }

    credentials, err := cfg.Credentials.Retrieve(context.TODO())

    if err != nil {
        // Handle error.
    }

    // The signer requires a payload hash. This hash is for an empty payload.
    hash := "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
    req, _ := http.NewRequest(http.MethodGet, "api-gw-url", nil)
    signer := v4.NewSigner()
    err = signer.SignHTTP(context.TODO(), credentials, req, hash, "execute-api", cfg.Region, time.Now())

    if err != nil {
        // Handle error.
    }

    // Use `req`
}

Answer 3

The first argument to request.New is aws.Config, where you can send credentials.

https://github.com/aws/aws-sdk-go/blob/master/aws/request/request.go#L99 https://docs.aws.amazon.com/sdk-for-go/api/aws/#Config

There are multiple ways to create credentials object: https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html

For example using static values:

creds:= credentials.NewStaticCredentials("AKID", "SECRET_KEY", "TOKEN")
req := request.New(aws.Config{Credentials: creds}, ...)

Answer 4

I'm pretty new to go myself (3rd day learning go) but from watching the video you posted with the S3 example and reading through the source code (for the s3 service and request module) here is my understanding (which I'm hoping helps).

If you look at the code for the s3.New() function aws-sdk-go/service/s3/service.go

func New(p client.ConfigProvider, cfgs ...*aws.Config) *S3 {
c := p.ClientConfig(EndpointsID, cfgs...)
return newClient(*c.Config, c.Handlers, c.Endpoint, c.SigningRegion, .SigningName) }

As opposed to request.New() function aws-sdk-go/aws/request/request.go

func New(cfg aws.Config, clientInfo metadata.ClientInfo, handlers Handlers,
retryer Retryer, operation *Operation, params interface{}, data interface{}) *Request { ...

As you can see in the s3 scenario the *aws.Config struct is a pointer, and so is probably initialized / populated elsewhere. As opposed to the request function where the aws.Config is a parameter. So I am guessing the request module is probably a very low level module which doesn't get the shared credentials automatically.

Now, seeing as you will be interacting with API gateway I had a look at that service specifically to see if there was something similar. I looked at aws-sdk-go/service/apigateway/service.go

func New(p client.ConfigProvider, cfgs ...*aws.Config) *APIGateway {
c := p.ClientConfig(EndpointsID, cfgs...)
return newClient(*c.Config, c.Handlers, c.Endpoint, c.SigningRegion, c.SigningName) }...

Which looks pretty much the same as the s3 client, so perhaps try using that and see how you go?