0

I'm trying to download data from CMIP6 climate projections ( https://cds.climate.copernicus.eu/cdsapp#!/dataset/projections-cmip6?tab=form ) with the cds.api extension on python.

The problem is that I don't get how to automatize the code in order to download successively the data from different existing models with a loop.


import __main__
import cdsapi
import sys
#import cdstoolbox 
import numpy as np


c = cdsapi.Client()

model=['cnrm_cm6_1','cmcc_cm2_sr5']
  
    
for k in range(len(model)):
        c.retrieve(
            'projections-cmip6',
            {
                'format': 'zip',
                'experiment': 'historical',
                'temporal_resolution': 'monthly',
                'variable': 'precipitation',
                'model': model[k],
                'year': '2010',
                'month': '01',
                'area': [
                    -2, 29, -3,
                    30,
                    ],
                },
                'download.zip')

The process takes place if I put only the name of a model outside the loop, instead of model[k]. But if I try to automate this query with a list, I am unable to get results.

Here is what I get in the Console:

2023-01-17 09:11:17,601 INFO Sending request 
to https://download-0000.copernicus-climate.eu/api/v2/resources/projections-cmip6
2023-01-17 09:11:17,607 WARNING Recovering from connection error [HTTPSConnectionPool
(host='download-0000.copernicus-climate.eu', port=443): 
Max retries exceeded with url: /api/v2/resources/projections-cmip6 
(Caused by NewConnectionError('<urllib3.connection.HTTPSConnection 
object at 0x00000222E751B3D0>: Failed to establish 
a new connection: [Errno 11001] getaddrinfo failed'))],
 attemps 0 of 500
2023-01-17 09:11:17,608 WARNING Retrying in 120 seconds
Max Arade
  • 3
  • 2
  • Apart from SSL warnings, this runs to normal completion on my system – DarkKnight Jan 17 '23 at 09:06
  • @Pingu And you use exactly the same code, with a list? – Max Arade Jan 17 '23 at 09:28
  • The fact that you're using a list is irrelevant. The problem stems from download-0000.copernicus-climate.eu which doesn't exist – DarkKnight Jan 17 '23 at 09:43
  • I read in others topics that it's supposed to work with server 0000, but here it seems it's not the case, do you know how to specify another server (0001,0002,...)? – Max Arade Jan 17 '23 at 10:16
  • You need to be in touch with the author(s) of the cdsapi module. This is not a Python issue *per se* – DarkKnight Jan 17 '23 at 10:19
  • @Pingu It seems it's under maintenance. By the past, it worked, and the request was sent. But with a list, the request was sent but did not finish, and the elements were therefore not downloadable on the copernicus site, but with only one element yes. Are you able to download the data on the site by using a loop with several models? – Max Arade Jan 17 '23 at 10:33

2 Answers2

0

Revised code that works with a list of models:

import cdsapi
import urllib3

urllib3.disable_warnings()

c = cdsapi.Client()

models = ['cnrm_cm6_1', 'cmcc_cm2_sr5']
params = {
    'format': 'zip',
    'experiment': 'historical',
    'temporal_resolution': 'monthly',
    'variable': 'precipitation',
    'year': '2010',
    'month': '01',
    'area': [-2, 29, -3, 30]
}


for model in models:
    params['model'] = model
    c.retrieve('projections-cmip6', params, f'{model}.zip')
DarkKnight
  • 19,739
  • 3
  • 6
  • 22
0

Thanks but the issue persists,

    models = ['cnrm_cm6_1', 'cmcc_cm2_sr5']

for model in models:
    params['model'] = model
    c.retrieve('projections-cmip6', params, f'{model}.zip')
2023-01-17 12:14:31,377 INFO Welcome to the CDS
2023-01-17 12:14:31,379 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/projections-cmip6
2023-01-17 12:14:31,462 INFO Request is completed
2023-01-17 12:14:31,463 INFO Downloading https://download-0014-clone.copernicus-climate.eu/cache-compute-0014/cache/data9/adaptor.esgf_wps.retrieve-1673946155.9144862-16441-14-5e78bb68-54bb-45c6-8624-7b97847b0541.zip to cnrm_cm6_1.zip (177.7K)
2023-01-17 12:14:31,843 INFO Download rate 469.9K/s
2023-01-17 12:14:31,942 INFO Welcome to the CDS
2023-01-17 12:14:31,943 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/projections-cmip6
2023-01-17 12:14:32,123 INFO Downloading https://download-0011-clone.copernicus-climate.eu/cache-compute-0011/cache/data8/adaptor.esgf_wps.retrieve-1673946169.8353872-1822-17-8196f65e-2a77-4ef4-a614-f50d3ea57296.zip to cmcc_cm2_sr5.zip (180K)
2023-01-17 12:14:32,666 INFO Download rate 332.6K/s

As you can see, there is no result. Do you have that?

If I run only with "cnrm_cm6_1":

2023-01-17 12:17:56,146 INFO Welcome to the CDS
2023-01-17 12:17:56,146 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/projections-cmip6
2023-01-17 12:17:56,220 INFO Downloading https://download-0014-clone.copernicus-climate.eu/cache-compute-0014/cache/data9/adaptor.esgf_wps.retrieve-1673946155.9144862-16441-14-5e78bb68-54bb-45c6-8624-7b97847b0541.zip to download.zip (177.7K)
2023-01-17 12:17:56,569 INFO Download rate 510.8K/s
Out[16]: Result(content_length=181969,content_type=application/zip,location=https://download-0014-clone.copernicus-climate.eu/cache-compute-0014/cache/data9/adaptor.esgf_wps.retrieve-1673946155.9144862-16441-14-5e78bb68-54bb-45c6-8624-7b97847b0541.zip)

And then it's downloadable

Max Arade
  • 3
  • 2