Get AWS resource cost based on particular tag

Question

I've created different AWS resources across EC2, Storage, Load Balancer, Networks etc. I've tagged each resource with GroupName as a key and value which is different for each group e.g. group1, group2, etc.

Now, I want to get cost of each group. I've written following code

    GroupDefinition groupDefinition =
        new GroupDefinition().withType(GroupDefinitionType.TAG).withKey("Cluster");

    GetCostAndUsageRequest costAndUsageRequest
        = new GetCostAndUsageRequest()
          .withGroupBy(groupDefinition)
          .withGranularity("MONTHLY")
          .withMetrics("UnblendedCost");

    GetCostAndUsageResult costAndUsage = 
         awsCostExplorer.getCostAndUsage(costAndUsageRequest);

Now, I expect the costAndUsage has groups based on each tag. But it's always giving me the total bill. I can give any random value to withKey but the result is always the same.

Maven dependency:

    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk</artifactId>
        <version>1.11.453</version>
    </dependency>

This is the response I'm getting (first resultByTime for brevity)

{
   "timePeriod":{
      "start":"2018-11-01",
      "end":"2018-12-01"
   },
   "total":{

   },
   "groups":[
      {
         "keys":[
            "Cluster$"
         ],
         "metrics":{
            "UnblendedCost":{
               "amount":"26712.9751185906",
               "unit":"USD"
            }
         }
      }
   ],
   "estimated":false
}

What I would like to have is

Cluster1 -> 1500 USD Cluster2 -> 1000 USD

Assuming I have a bunch of resources tagged with key Cluster and values Cluster1 and Cluster2.

There's no grouping by each actual tag value in the response. What am I missing here?

Which version of the API are you using? Can you share the documentation you looked at? — Dvir669, Dec 01 '18 at 22:02
Unfortunately there isn't much of documentation available for this. You just have to dig in the client source code for that. Here's the link: https://aws.amazon.com/sdk-for-java/ — Ganesh Satpute, Dec 03 '18 at 12:28
check AWS documentation on the response: https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_GetCostAndUsage.html — Dvir669, Dec 03 '18 at 16:54
As I'm debugging I can see the whole response. It's not the question of parsing it correctly, it's a problem where I'm getting the same output irrespective of what I pass as a input. For example, let's say I hav a tag with key `Cluster`. There are resources with values for this particular tag `Cluster1` `Cluster2`. What I expect is, `Cluster1 -> 1500 USD` `Cluster2 -> 1000 USD` What I'm getting is same value irrespective what I pass. If pass non-existent tag I get same values. — Ganesh Satpute, Dec 04 '18 at 05:42
So my only though would be that the tag is not enabled in the billing console and it treats it like a non-existent value. — Dvir669, Dec 04 '18 at 16:47
Yes. Those are not enabled it. We've already requested to enable those tags. But this seems quite confusing that the API is not rejecting those requests. — Ganesh Satpute, Dec 04 '18 at 18:20
You can do it yourself - Go to your "Billing" service, on the left side select "Cost Allocation Tags", mark the "Cluster" tag and press "Activate". The next report will contain your tags. — Dvir669, Dec 05 '18 at 01:17
Unfortunately, my account is a subaccount of some other account. So it needs to be done by the parent account owner. — Ganesh Satpute, Dec 05 '18 at 06:36

score 0 · Answer 1 · answered Dec 01 '18 at 22:09

0

I'm not sure what's your actual version of the SDK you're using, but I can show you a working code in Ruby (I'm also grouping by the AWS service):

credentials = Aws::SharedCredentials.new(profile_name: "REPLACE_profile_name_in_credentials_file")
client = Aws::CostExplorer::Client.new(region: "us-east-1", credentials: credentials)
resp = client.get_cost_and_usage({
      time_period: {
        start: Date.today.at_beginning_of_month.strftime("%Y-%m-%d"), 
        end: Date.tomorrow.strftime("%Y-%m-%d"), 
      },
      granularity: "MONTHLY", 
      metrics: ["UnblendedCost"],
      group_by: [
        {
          type: "DIMENSION", 
          key: "SERVICE",
        },
        {
          type: "TAG",
          key: "Cluster",  
        }
      ]
    })
resp.results_by_time.each do |result|
  #DO SOMETHING
end

It might give you a better direction on what you need to add \ edit. If you can share your documentation and the response you get it'll be helpful to edit and add more relevant information.

answered Dec 01 '18 at 22:09

Dvir669

6,577
1
20
21

Thanks for the answer @Dvir669. I don't want to aggregate on service. Would it be possible to check whether after removing `SERVICE` group by field, does it work? Also, if changing value to `Cluster1` do you get the same output? Also, have you enabled `Cluster` tag on your AWS billing portal? – Ganesh Satpute Dec 03 '18 at 12:39
Also, I tried the API similar way you've used. What I found is, irrespective whether I send the `TAG` field, I get the same output. – Ganesh Satpute Dec 03 '18 at 12:56
I don't see what you're doing with the response in your code - please note that any response from that API has a "ResultsByTime" -> "Groups" -> "Keys" that you need to access, and not the "Total" field. – Dvir669 Dec 03 '18 at 13:38
You can remove the "Service" clause and it'll still work. Please note you need to use the "Key" of the tag, and not the "Value" of the tag – Dvir669 Dec 03 '18 at 13:48
Yes. That is exactly what I'm doing (as per your last two comments). The problem is not with the response, the problem is the values I'm getting. They are same irrespective what I pass as a tag key. Logically this is what I want: `Cluster1 -> 1500 USD` `Cluster2 -> 1000 USD` Of course, here `Cluster1` and `Cluster2` are values of tag with name `Cluster`. – Ganesh Satpute Dec 04 '18 at 05:36
Also, have you enabled the Billing tags on cost and billing portal? – Ganesh Satpute Dec 04 '18 at 05:38
Hey Ganesh, you have to enable those tags in the billing console otherwise you won't be able to search by it. https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/activating-tags.html – Dvir669 Dec 04 '18 at 16:46

score 0 · Accepted Answer · answered Dec 17 '18 at 07:05

The code was working properly. I was not receiving properly because there are additional steps that needs to be carried out on parent payee account.

You need to

Go to the AWS billing portal
Select the tag for which you want to enable costing, in my case Cluster
Enable the cost allocation
It will take around 24 hours for this to affect.

Note: if you have AWS subaccount, your parent payee account will have permission to carry out the above steps.

See the documentation for more details.

After 24 hours, if you run the code will see the response as shown below

{
   "timePeriod":{
      "start":"2018-12-01",
      "end":"2018-12-31"
   },
   "total":{

   },
   "groups":[
      {
         "keys":[
            "Cluster$"
         ],
         "metrics":{
            "UnblendedCost":{
               "amount":"23434.5469766795",
               "unit":"USD"
            }
         }
      },
      {
         "keys":[
            "Cluster$cluster-1"
         ],
         "metrics":{
            "UnblendedCost":{
               "amount":"2343.3888413893",
               "unit":"USD"
            }
         }
      },
      {
         "keys":[
            "Cluster$cluster-2"
         ],
         "metrics":{
            "UnblendedCost":{
               "amount":"23464.8057597427",
               "unit":"USD"
            }
         }
      }
   ],
   "estimated":true
}

Get AWS resource cost based on particular tag

2 Answers2