1

I'm using the Ruby SDK for AWS ECS to kick-off a task hosted in Fargate via run_task method. This all works fine with the defaults — I can kick off the task OK and can send along custom command parameters to my Docker container:

client = Aws::ECS::Client.new(region: region)
client.run_task({
  cluster: fargate_cluster_name,
  task_definition: task_definition,
  launch_type: 'FARGATE',
  network_configuration: {
    awsvpc_configuration: {
      subnets: my_subnets,
        security_groups: my_security_groups,
        assign_public_ip: 'ENABLED',
      },
  },
  overrides: {
    container_overrides: [
      {
        name: my_container_name,
        command: command_params
      }
    ]
  }
})

For some runs, I need a large disk size than the default & want to set this via the ephemeral_storage parameter in the SDK. I've tried this:

client = Aws::ECS::Client.new(region: region)
client.run_task({
  cluster: fargate_cluster_name,
  task_definition: task_definition,
  launch_type: 'FARGATE',
  network_configuration: {
    awsvpc_configuration: {
      subnets: my_subnets,
        security_groups: my_security_groups,
        assign_public_ip: 'ENABLED',
      },
  },
  overrides: {
    container_overrides: [
      {
        name: my_container_name,
        command: command_params
      }
    ],
    ephemeral_storage: {
      size_in_gi_b: 200
    }
  }
})

But it doesn't work (I'm running df -H within my entrypoint and can see that the storage doesn't increase).

  • I know that the ephermal_storage property is in the right place in my call, as if I move it the SDK gives an error.
  • I know that the size_in_gi_b value is correct, as if I change it (e.g. size_in_gib) I also get an error.
  • I am running with platform version 1.4.0
  • There are no errors or warnings in Cloudwatch that the property can't be set e.g. (the task runs fine, just then runs out of disk space).
  • I can't find anything in the docs for how to fix this.

How should ephemermal storage be set for ad hoc task execution via the Ruby SDK? (I'm looking for the SDK equivalent to this blogpost by AWS, rather than an alternative solution e.g. mounting EFS or running via EC2)

anotherdave
  • 6,656
  • 4
  • 34
  • 65
  • It will not reflect. As of now, You can increase ephemeral storage using 4 ways only, 1.) AWS Copilot CLI, 2.) CloudFormation, 3.) AWS SDK, 4.) AWS CLI Reference : https://aws.amazon.com/about-aws/whats-new/2021/04/amazon-ecs-aws-fargate-configure-size-ephemeral-storage-tasks/ – Ezhilisai May 25 '21 at 15:38
  • @Ezhilisai thanks for your comment. This is #3 — via the official AWS SDK for Ruby – anotherdave May 25 '21 at 15:43
  • You should first define 'ephemeral_storage' using register_task_definition, then you can override using run_task if you need to do so. – Ezhilisai May 25 '21 at 15:49
  • @Ezhilisai the Task Definition is already defined in AWS, I'm not using the Ruby SDK to define a new one. If I were though, from the docs, the [`register_task_definition`](https://docs.aws.amazon.com/sdk-for-ruby/v2/api/Aws/ECS/Client.html#register_task_definition-instance_method) method that you're talking about doesn't seem to take an `ephemeral_storage` property. Is this something that you've tried which worked, or are you making suggestions of where to look? If you do have something working, please add an answer with specifics & I'll check it out. – anotherdave May 26 '21 at 07:13
  • docs your are referring is old version, here is the latest version, https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/ECS/Client.html#register_task_definition-instance_method I have not used Ruby SDK, so i dont have any specifics to it. i used AWS Cli to update. Since it is a 1 time update of ephemeral_storage to task definition. why don't you try it from AWS CLI. – Ezhilisai Jun 02 '21 at 08:01
  • @Ezhilisai thanks, but, with respect, I'm not interested in alternative approaches like using the CLI (as I said in the question initially) — this isn't an XY problem, I'm trying to use a documented method in the SDK on `run_task`, and want to override the value at run-time. – anotherdave Jun 02 '21 at 08:12

1 Answers1

0

This was a bug of the SDK, now fixed (server-side, so doesn't require a library update).

The block of code in the question is the correct way for increasing ephemeral storage via the Ruby SDK:

client.run_task({
  # ...
  overrides: {
    # ...
    ephemeral_storage: {
      size_in_gi_b: 200
    }
  }
})
anotherdave
  • 6,656
  • 4
  • 34
  • 65