5

I'm just starting out using Apache Beam on Google Cloud Dataflow. I have a project set up with a billing account. The only things I plan on using this project for are: 1. dataflow - for all data processing 2. pubsub - for exporting stackdriver logs to be consumed by Datadog

Right now, as I write this, I am not currently running any dataflow jobs.

Looking at the past month, I see ~$15 in dataflow costs and ~$18 in Stackdriver Monitor API costs. It looks as though Stackdriver Monitor API is close to a fixed $1.46/day.

I'm curious how to mitigate this. I do not believe I want or need Stackdriver Monitoring. Is it mandatory? Further, while I feel I have nothing running, I see this over the past hour:

enter image description here

So I suppose the questions are these: 1. what are these calls? 2. is it possible to disable Stackdriver Monitoring for dataflow or otherwise mitigate the cost?

drobert
  • 1,230
  • 8
  • 21
  • 3
    In the Stackdriver Monitoring API's "Metrics" console, I have graphed by credential, and found that these calls are coming from a third party app (in my case, Datadog). So, that mystery solved. – drobert Dec 02 '19 at 21:41
  • 1
    I recommend you post an answer to the question to say that graphing by credential is a good way to determine exactly where the usage is coming from. – Yuri Grinshteyn Dec 02 '19 at 22:47
  • I'd be interesting if datadog could be throttled to make less api requests (to stay within the free stackdriver quota of 1 million requests/month) – daniel-sc Apr 12 '21 at 08:50

2 Answers2

4

Per Yuri's suggestion, I found the culprit, and this is how (thanks to Google Support for walking me through this):

  • In GCP Cloud Console, navigate to 'APIs & Services' -> Library
  • Search for 'Strackdriver Monitoring Api' and click
  • Click 'Manage' on the next screen
  • Click 'Metrics' from the left-hand side menu
  • In the 'Select Graphs' dropdown, select "Traffic by Credential" and click 'OK'

This showed me a graph making it clear just about all of my requests were coming from a credential named datadog-metrics-collection, a service account I'd set up previously to collect GCP metrics and emit to Datadog.

drobert
  • 1,230
  • 8
  • 21
0

Considering the answer posted and question, If we think we do not need Stackdriver monitoring, we can disable stackdriver monitoring API using bellow steps:

  1. From the Cloud Console,go to APIs & Services.
  2. Select Stackdriver Monitoring API.
  3. Click Disable API.

In addition you can view Stackdriver usage by billing account and also can estimate cost using Stackdriver pricing calculator [a] [b].

View Stackdriver usage by billing account:

  1. From anywhere in the Cloud Console, click Navigation menu and select Billing.
  2. If you have more than one billing account, select Go to linked billing account to view the current project's billing account. To locate a different billing account, select Manage billing accounts and choose the account for which you'd like to get usage reports.
  3. Select Reports.

4.Select Group By > SKU. This menu might be hidden; you can access it by clicking Show Filters.

  1. From the SKUs drop-down list, make the following selections:
    1. Log Volume (Stackdriver Logging usage)
    2. Spans Ingested (Stackdriver Trace usage)
    3. Metric Volume and Monitoring API Requests (Stackdriver Monitoring usage)
  2. Your usage data, filtered by the SKUs you selected, will appear.

You can also select just one or some of these SKUs if you don't want to group your usage data.

Note: If your usage of any of these SKUs is 0, they don't appear in the Group By > SKU pull-down menu. For example, who use only the Cloud console might never generate API requests, so Monitoring API Requests doesn't appear in the list.

Use the Stackdriver pricing calculator [b]:

  1. Add your current or projected Monitoring usage data to the Metrics section and click Add to estimate.
  2. Add your current or projected Logging usage data to the Logs section and click Add to estimate.
  3. Add your current Trace usage data to the Trace spans section and click Add to estimate.
  4. Once you have input your usage data, click Estimate. Estimates of your future Stackdriver bills appear. You can also Email Estimate or Save Estimate.

[a] https://cloud.google.com/stackdriver/estimating-bills#billing-acct-usage

[b] https://cloud.google.com/products/calculator/#tab=google-stackdriver

Shanewaz
  • 191
  • 3
  • It does not appear to be acceptable to disable the Stackdriver Monitoring API for a Dataflow project. It would disable 'googleapis' which is presumably required for many background operations, such as autoscaling. – drobert Dec 04 '19 at 00:41
  • I was looking for the option for mitigating cost according to the requirement, though unable to get anything except:1. To disable Stackdriver monitoring API but if there is a dependency on backend services, in that case disabling the API is not a right move. 2. To modify application to stop sending metric data to Stackdriver. [c] https://cloud.google.com/monitoring/workspaces/guide#disabling-monitoring – Shanewaz Dec 04 '19 at 20:19