1

edit: Turns out the solution is in the docs. I had bog standard normal 'sam' installed but I needed what they call the 'public preview version' AKA 'sam-beta-cdk'. With this installed the API can be started locally with sam-betacdk start-api and works well. While I appreciate the answers which suggest that development should be done using purely TDD I feel there is also value in this more interactive, manual mode as it permits quicker exploration of the problem space.

I'm trying to build my first app with CDK + Typescript using API Gateway, Lambdas and DynamoDB. I have built a couple of Lambdas and deployed them and they work fine live on the web. However I don't want a minute long deploy cycle and various associated AWS costs as part of my workflow. What I want is to be able to test my API locally.

I have struggled to find docs on how to do this. Amazon seem to recommend using the SAM CLI here so that is what I've been trying.

The docs claim running sam local xyz runs cdk synth to make a "could assembly" in ./aws-sam/build but I see no evidence of this. Instead what I get is a complaint that sam could not find a 'template.yml'. So I manually run cdk synth > template.yml which creates one in the root folder. Then I run sam local start-api and it seems happy to start up.

Then I try and hit my test lambda using CURL: curl 'http://127.0.0.1:3000/test' I get {"message":"Internal server error"} and a huge ugly stack trace in the console that is running sam local start-api

The lambda is this...

exports.handler = async function() {
    console.log("WooHoo! Test handler ran")
    return {statusCode: 200, headers: {"Content-Type": "application/json"}, body: "Test handler ran!"}
}

Start of the huge ugly stack trace...

Mounting /home/user/code/image-cache/asset.beeaa749e012b5921018077f0a5e4fc3ab271ef1c191bd12a82aa9a92148782e as /var/task:ro,delegated inside runtime container
START RequestId: 99f53642-b294-4ce5-a1b4-8c967db80ce1 Version: $LATEST
2021-09-15T12:33:37.086Z    undefined   ERROR   Uncaught Exception  {"errorType":"Runtime.ImportModuleError","errorMessage":"Error: Cannot find module 'test'\nRequire stack:\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js","stack":["Runtime.ImportModuleError: Error: Cannot find module 'test'","Require stack:","- /var/runtime/UserFunction.js","- /var/runtime/index.js","    at _loadUserApp (/var/runtime/UserFunction.js:100:13)","    at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)",

The end of the huge ugly stack trace...

Invalid lambda response received: Lambda response must be valid json

So it would seem sam local start-api can't find test and throws and error which means the API gateway doesn't get a valid 'lambda response'. So far this has not helped me chase down the problem :/ It certainly seems aware that test is a route, as trying to hit other endpoints gives the classic {"message":"Missing Authentication Token"} but it chokes hard trying to fulfill it despite me having both functions/test.ts and the compiled functions/test.js present.

I have the test route and handler defined in my CDK stack definition like so...

    const testLambda = new lambda.Function(this, "testLambdaHandler", {
      runtime: lambda.Runtime.NODEJS_14_X,
      code: lambda.Code.fromAsset("functions"),
      handler: "test.handler"
    })

    api.root
      .resourceForPath("test")
      .addMethod("GET", new apigateway.LambdaIntegration(testLambda))

I considered posting my template.yml but that is even longer than the big ugly error message so I haven't.

So I have three questions (well actually a million but I don't want to be too cheeky!)

  1. Is this actually the canonical way of locally testing apps made with CDK
  2. If so, where am I going wrong?
  3. If not, what is the better/proper way?
Roger Heathcote
  • 3,091
  • 1
  • 33
  • 39
  • Hello, @Roger. When the `cdk synth` command is executed, could you post it here the `template.yaml` file so we can check it out the resources being created? – Mateus Arruda Sep 19 '21 at 12:21
  • @MateusArruda of course, it's all here: https://github.com/Roger-Heathcote/image-cache – Roger Heathcote Sep 20 '21 at 13:51
  • This is odd, but I have a guess about this: your `image-cache-stack.ts` is in `/lib` right? And in the `addLambda` function you are pointing out to a directory called "functions", but in the file you are right now there is no directory called `functions`. So I think you could do the following: 1. use `code: lambda.Code.fromAsset("../../functions")` or, as in [this documentation](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-lambda-readme.html) you set the root directory as a function. Let us know if this works out for you. – Mateus Arruda Sep 20 '21 at 17:43
  • Thanks Mateus. I just found the problem and it wasn't the code at all, I didn't have the most recent "public preview" version of sam installed. I was thrown by the examples at the top not using 'sam-beta-cdk'. I installed that an all is well. Thanks very much for your time! – Roger Heathcote Sep 21 '21 at 16:30
  • I'm glad to hear that! I'll stay tuned so that don't happen to me as well – Mateus Arruda Sep 21 '21 at 17:38

2 Answers2

0

You must be doing something wrong with your file directory. Where is your index.js located? If you generate the template.json, is the directory correct? Also in what directory do you execute the Sam local command?

The thing with testing your serverless application is you don't have to test your full application. You need to count on AWS that API gateway, dynamodb and lambda is perfectly working. The only thing you need to test is the logic you implemented.

In here you make sure your function prints out something and returns a 200. That's all you have to do. Look into 'jest' for testing js.

If you want to test cdk you should into https://docs.aws.amazon.com/cdk/latest/guide/testing.html

Also "running Aws locally" is not good practice. it's never the same as how it's running in real life aka the cloud. You use plugins for this, tools for that... Local is not the same as in the cloud.

If you have any more questions, feel free to ask.

Lucasz
  • 1,150
  • 9
  • 19
  • I have no index.js, my entry point is bin/image-cache.js. The generated template is in the root folder which is also where I am running sam. Project is here: https://github.com/Roger-Heathcote/image-cache thanks. – Roger Heathcote Sep 20 '21 at 13:56
0

Lambda handlers are just functions. They do not need any special environment to function - they are called at a specific point in the Lambda Invocation process, and provided an event (a json object) and a context (another json object)

You can (and should!) unit test them just like any other individual function in your language/testing framework.

As @Lucasz mentioned, you should rely on the fact that, if set up properly, API gateway and Lambda will interact the same way every time. Once you have run one end to end test and you know that the basics work, any further implementation can be done trough unit testing

There are numerous libraries for mocking AWS service calls in unit testing, and there are plenty of good practice work arounds for the services that are more difficult to mock (ie: its difficult to mock a Lambda call from inside another lambda - but if you wrap that lambda call in its own function, you can mock the function itself to return whatever you want it to - and this is good practice for testing as well!)

using jest, in a coded unit test, you can call the lambda handler, give it stubbed or mocked event json, as well as a context json (probably just blank as youre not using it) and the lambda handler will act just like any other function with two parameters you've ever written, including returning what you want it to return.

lynkfox
  • 2,003
  • 1
  • 8
  • 16
  • ```Lambda handlers are just functions. They do not need any special environment to function ``` This is not entirely true. They are containers, and it's possible to specify environment variables to pass on to them, for one. They can also depend on the `event` and `context` objects, which can be non-trivial to simulate locally. – gshpychka Sep 28 '21 at 13:56
  • 1
    *Lambdas* are containers. Lambda handlers are just functions. The Event and the Context are just JSON objects - And since you are programming the lambda yourself its no mystery what you need from them. You don't have to replicate the entire context object to run just the handler; you just need the keys out of it or the event you are using inside your handler. likewise, env are just env variables. Any language has a way to access them (os.environ in python) - overall, its very easy to run the *handler* to see if it acts as expected while unit testing and with proper mocks. – lynkfox Sep 29 '21 at 04:39
  • Sure, I'm only saying that lambda handlers do depend on their environment, and you need to be careful about replicating it in the right way. For example, you cannot simply unit test the lambda role to see whether it has the required permissions to do what it needs. The context object also has methods, it's not a json. – gshpychka Sep 29 '21 at 11:45
  • 1
    I hear you - and its important to remember that Unit Tests are not an end all be all, but one step/layer of a comprehensive testing strategy. They wouldn't be for testing the permissions for instance (though if you are using CDK and are familiar enough with cloudformation template structure, you can unit test that there) - but those kinds of changes are generally just a few deployments to get to work - where as the impression i got from OP's post was many small logic changes which are better suited for bare bones UTs – lynkfox Sep 30 '21 at 12:13