I'd like a lambda (transformer) to call another lambda (source) without going through lambda.invoke
, rather just requesting it by going through the (private) api gateway one more time. The background is that this allows a more straightforward debugging, testing & development (by not relying on aws facilities within the main functions). Running transfomer locally works fine (container can request the api gateway, in that case I replace the aws_api_root_uri
with http://host.docker.internal:3000
, which launches another container for the source lambda). However, trying the same after deployment on AWS I end up with
HTTPSConnectionPool(host='<id>.execute-api.eu-central-1.amazonaws.com', port=443): Max retries exceeded with url: /sandbox/sources/test (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f73ebda3790>: Failed to establish a new connection: [Errno -2] Name or service not known'))
This seems like a DNS problem accessing the private resource. Using the public internet on the same lambda works fine, i.e. google.com
can be requested fine. On the other hand, accessing the same URI that produces an error in the lambda, but on an EC2 instance in the same VPC, works fine as well.
This rules out a couple of things
- cannot be a general DNS protocol access issue
- the source lambda is available and can be requested on the network as expected
Do I need to add a specific policy that a VPC EC2 instance implicitely has, but a lambda on that VPC does not? If so, which policy would that be? Or can you think of another problem that is causing this issue?
expected response when calling GET on transfomer URI
{"message": "hello world", "location": "<ip>", "added": "value"}
actual response
{"message": "Internal server error"}
template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
SAM template
Globals:
Function:
Timeout: 10 # need a relatively long timeout for the local deployment - two container spin-ups in series needed
Parameters:
MyEndpointId:
Type: String
Description: The AWS endpoint ID for the API Gateway endpoint
Resources:
MyTestSourceFunction:
Type: AWS::Serverless::Function
Properties:
Runtime: python3.8
CodeUri: data_sources/test_source
Handler: service.lambda_handler.handle_lambda
Events:
MyTestSourceApi:
Type: Api
Properties:
RestApiId: !Ref MyApi
Path: /sources/test
Method: get
MyTestTransformerFunction:
Type: AWS::Serverless::Function
Properties:
Runtime: python3.8
CodeUri: data_transformers/test_transformer
Handler: service.lambda_handler.handle_lambda
Events:
MyTestTransformerApi:
Type: Api
Properties:
RestApiId: !Ref MyApi
Path: /transformers/test
Method: get
MyApi:
Type: AWS::Serverless::Api
Properties:
StageName: sandbox
MethodSettings:
- HttpMethod: '*'
ResourcePath: /*/*/*
LoggingLevel: ERROR
ThrottlingBurstLimit: 5000
ThrottlingRateLimit: 10000
EndpointConfiguration:
Type: PRIVATE
VPCEndpointIds:
- !Ref MyEndpointId
Auth:
ResourcePolicy:
CustomStatements: [{
"Effect": "Allow",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "execute-api:/*/*/*/*"
}]
data_sources/test_source/service/lambda_handler.py
import json
from .main import generate_test_data
def handle_lambda(event, context):
return {
"statusCode": 200,
"body": json.dumps(generate_test_data()),
}
data_sources/test_source/service/main.py
import requests
def generate_test_data():
try:
ip_address = requests.get("http://checkip.amazonaws.com/").text.replace("\n", "")
except requests.RequestException as exc:
print(exc)
raise exc
return {
"message": "hello world",
"location": ip_address
}
data_transformers/test_transformer/service/lambda_handler.py
import os
import json
from .main import load_and_transform
def handle_lambda(event, context):
aws_api_root_uri = "https://{}.execute-api.{}.amazonaws.com/{}/".format(
event['requestContext']['apiId'],
os.environ['AWS_REGION'],
event['requestContext']['stage']
)
return {
"statusCode": 200,
"body": json.dumps(load_and_transform(aws_api_root_uri)),
}
data_transformers/test_transformer/service/main.py
import urllib.parse
import requests
def load_and_transform(api_root_path):
test_data = {}
try:
# do we have DNS working?
print(requests.get("https://www.google.com/").text)
# can we receive data from another lambda?
uri = urllib.parse.urljoin(api_root_path, "sources/test")
print(f"test transfomer is sending query to {uri}")
test_data = requests.get(uri).json()
test_data["added"] = "value"
except Exception as exc:
print(exc)
raise exc
return test_data