ECS TaskDefinition creation fails with "Invalid request provided: Create TaskDefinition: Unknown volume 'null'."

Question

I am attempting to use the CDK to deploy an EC2 TaskDefinition for a DataDog agent which uses 3 bind mount volumes. The DD example task definition looks pretty simple.

The fragment of a Cloudformation template below has been generated by the AWS CDK. Deploying this raises the Unknown volume: 'null' error with all three volumes as is. It deploys successfully if no volumes are set.

I have tried using changesets through the cloudformation UI to add volumes one at a time. Adding the first volume by itself works normally, but adding two or three fails with the Unknown volume: 'null' error, regardless of doing it in stages or in one lump.

Editing the task definition manually through the ECS console does add a second volume successfully.

There doesn't appear to be any documentation on this particular error, and I can't understand how the ECS service is getting a null in the volumes array. I suspect I'm missing something obvious somewhere, so hoping someone has some insight into what's going on.

{
  "MonitoringTask9D49B4FA": {
    "Type": "AWS::ECS::TaskDefinition",
    "Properties": {
      "ContainerDefinitions": [
        {
          "Cpu": 10,
          "Environment": [
            {
              "Name": "DD_SITE",
              "Value": "datadoghq.com"
            }
          ],
          "Essential": true,
          "Image": "datadog/agent:latest",
          "LogConfiguration": {
            "LogDriver": "awslogs",
            "Options": {
              "awslogs-group": {
                "Ref": "MonitoringTaskDatadogAgentLogGroupC5828485"
              },
              "awslogs-region": "eu-west-1"
            }
          },
          "MemoryReservation": 256,
          "MountPoints": [
            {}
          ],
          "Name": "datadog"
        }
      ],
      "ExecutionRoleArn": {
        "Fn::GetAtt": [
          "MonitoringTaskExecutionRole3188D770",
          "Arn"
        ]
      },
      "Family": "datadog-agent-task",
      "NetworkMode": "bridge",
      "RequiresCompatibilities": [
        "EC2"
      ],
      "Tags": [
        {
          "Key": "Environment",
          "Value": "develop"
        },
      ],
      "TaskRoleArn": {
        "Fn::GetAtt": [
          "MonitoringTaskTaskRole70FF4D63",
          "Arn"
        ]
      },
      "Volumes": [
        {
          "Host": {
            "SourcePath": "/var/run/docker.sock"
          },
          "Name": "docker_sock"
        },
        {
          "Host": {
            "SourcePath": "/proc/"
          },
          "Name": "proc"
        },
        {
          "Host": {
            "SourcePath": "/sys/fs/cgroup/"
          },
          "Name": "cgroup"
        }
      ]
    },
    "Metadata": {
      "aws:cdk:path": "Stack/MonitoringTask/Resource"
    }
  }
}

score 1 · Accepted Answer · answered Nov 01 '21 at 08:42

Turns out I was misreading the CDK addMountPoints function.

When I was having success adding the volumes, I was also not adding any mount points, but I kept adding mount points while also adding new things, so masked the actual error. This is the relevant javascript from my original code.

    container.addMountPoints([
      {
        containerPath: "/var/run/docker.sock",
        sourceVolume: "docker_sock",
        readOnly: true
      },
      {
        containerPath: "/host/sys/fs/cgroup",
        sourceVolume: "cgroup",
        readOnly: true
      },
      {
        containerPath: "/host/proc",
        sourceVolume: "proc",
        readOnly: true
      },
    ])

It was generating a Cloudformation property that looked like this.

MountPoints:
  - {}

This is because I was giving the function an array of MountPoints. addMountPoints() like many other CDK functions uses a rest parameter (...mountPoints) so it was treating the array itself like a single MountPoint. Very frustrating, but hopefully this helps someone else.

ECS TaskDefinition creation fails with "Invalid request provided: Create TaskDefinition: Unknown volume 'null'."

1 Answers1