2

I am trying to read Crude Oil Production data from https://www.eia.gov/ using GET request. It is quite straightforward to read it in a single-line URL address. However, I can not use params within requests.get(api_url, params=params) command.

api_url = 'https://api.eia.gov/v2/steo/data/'

params = {
    "api_key": xxxxxx,
    "frequency": "annual",
    "data": [
        "value"
    ],
    "facets": {
        "seriesId": [
            "COPR_OPEC",
            "PAPR_WORLD"
        ]
    },
    "start": "2000",
    "end": "2023",
    "sort": [
        {
            "column": "period",
            "direction": "desc"
        }
    ],
    "offset": 0,
    "length": 5000,
    "api-version": "2.0.2"
}

try:
    (r := requests.get(api_url, params=params)).raise_for_status()
    print(r.text[0:1000])
except Exception as e:
    print(f'Data acquisition failed due to {e}')

The error I am getting is:

Error: 400 as data acquisition failed due to "Bad Request for URL":
https‍://api.eia.gov/v2/steo/data/?api_key=xxxxxx&frequency=annual&data=value&facets=seriesId&start=2000&end=2023&sort=column&sort=direction&offset=0&length=5000

I can fix the issue by removing "facets", "api-version", and "length" from "params"; but I need to filter the JSON data later on.

I wonder if I am not using the correct structure for "params". I couldn't find other Python sample code for using "params" on the EIA API v2 Dashboard.

Figure 1: EIA API v2 "params" settings

Brian Tompsett - 汤莱恩
  • 5,753
  • 72
  • 57
  • 129
Farid
  • 21
  • 2

2 Answers2

0

Don't know if you had solved it yet but I had the same problem, and solved it in a rather dummy way. You are right in not getting the correct structure for "params". It seems like they structured the faucet in a rather layered way. There doesn't seem to be any instruction online to directly tackle that as well so I've decided to use the link generator they provided:

https://www.eia.gov/opendata/browser/petroleum/crd/crpdn

and then just code the square bracket as strings into the header:

For example:

url1 = 'https://api.eia.gov/v2/petroleum/crd/crpdn/data/'
test = {
    "api_key": apikey,
    "frequency": "annual",
    "data[0]": "value",
    "facets[series][]": "MCRFPAKS1",
    "facets[series][]": "MCRFPAKS2",
    "start": "2000",
    "end": "2023",
    "sort[0][column]": "period",
    "sort[0][direction]": "desc",
    "offset": 0,
    "length": 5000,
    }
temp = requests.get(url1, params=test)

P.S., I think they changed the name from "seriesId" to "series", and I can't seem to find "COPR_OPEC" and "PAPR_WORLD" as well. But the codes above works. I've tested it.(make sure you get the API url right as well because I think they changed it from V1 to V2)

marcochu852
  • 1
  • 1
0

Found this post trouble shooting a similar problem with EIA API v2. I think the easier route is to pass all params except api_key in as a headers argument to the get request. You can include X-Params as a header and dump all the API filters into the header as opposed to trying to compose the url via params.

Your get call would look like this. You need to call json.dumps(header) to convert the python dict to a str.

api_url = 'https://api.eia.gov/v2/steo/data'

params = {"api_key": xxxxxx}

header = {
    "frequency": "annual",
    "data": [
        "value"
    ],
    "facets": {
        "seriesId": [
            "COPR_OPEC",
            "PAPR_WORLD"
        ]
    },
    "start": "2000",
    "end": "2023",
    "sort": [
        {
            "column": "period",
            "direction": "desc"
        }
    ],
    "offset": 0,
    "length": 5000
}

r = request.get(api_url, params=params, headers={"X-Params": json.dumps(header)})
John Rudolph
  • 1
  • 1