I've been using CDK to regularly deploy my python-based lambdas to AWS for over a year and it's been working great. The build broke yesterday, and I noticed that the image I'm using was updated a couple of days ago: https://github.com/aws/aws-lambda-base-images/commits/python3.9/Dockerfile.python3.9
I'm now getting a build error 'Please make sure the libxml2 and libxslt development packages are installed.'
My functions and layers are build in a similar fashion, e.g.:
from aws_cdk import (
aws_lambda_python_alpha as _lambda,
DockerImage,
AssetHashType,
NestedStack,
RemovalPolicy,
)
def create_lambda_layer(
self, deploy_folder: str, layer_name: str, only_include_these=[]
) -> _lambda.PythonLayerVersion:
layer_folder = self.build_dependencies_layer(
deploy_folder, layer_name, only_include_these=only_include_these
)
print(layer_name, layer_folder, deploy_folder, only_include_these)
return _lambda.PythonLayerVersion(
self,
layer_name,
entry=layer_folder,
compatible_runtimes=[Runtime.PYTHON_3_9],
compatible_architectures=[Architecture.ARM_64],
bundling={
"image": DockerImage("public.ecr.aws/sam/build-python3.9:latest-arm64"),
"asset_hash_type": AssetHashType.SOURCE,
},
removal_policy=RemovalPolicy.RETAIN,
)
I have a particular layer which depends on the Zeep python library. I recall this being problematic to bundle/build on my Apple Silicon Mac. When building that layer, pip running within a Docker image fails with the following error:
Collecting lxml>=4.6.0
Downloading lxml-4.9.3.tar.gz (3.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.6/3.6 MB 8.7 MB/s eta 0:00:00
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'error'
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [3 lines of output]
Building lxml version 4.9.3.
Building without Cython.
Error: Please make sure the libxml2 and libxslt development packages are installed.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
The salient line from that is:
Please make sure the libxml2 and libxslt development packages are installed.
How do I make these available to the docker images doing the build? Do I build my own image, or do I need to run some install steps 'on'/'in' the docker image. Any suggestions?
[edit] I tried modifying the build to point at a Docker file and install the dependencies myself. I can see these binaries being installed in the Docker image, but pip within the container still failes
FROM public.ecr.aws/sam/build-python3.9:latest-arm64
RUN yum install -y libxml2 libxslt
[/edit]
[edit2] Got it working. I was yum installing libxml2 and libxslt, and not their DEVELOPMENT packages, which was what the build error said. So this was resolved by doing this in the Dockerfile:
FROM public.ecr.aws/sam/build-python3.9:latest-arm64
RUN yum install -y Cython libxml2-devel libxslt-devel
and then for building the layer I now do:
bundling={
'image': DockerImage.from_build(os.path.expanduser('~/workplace/cdk/stacks/')),
'asset_hash_type': AssetHashType.SOURCE
}
and it's working
[/edit2]
