Exporting TIF Images from Google Earth Engine to Google Drive: Minimal Example in Python

Question

I thought exporting images from GEE should be quite straightforward, turns out I am facing difficulties, and I'm not satisfied with the answers given so far on this platform.

As a minimal example, I want to extract nightlights images at original scale for South Africa:

import ee
try:
        ee.Initialize()
except Exception as e:
        ee.Authenticate()
        ee.Initialize()

# Nightlights
viirs = ee.ImageCollection("NOAA/VIIRS/DNB/MONTHLY_V1/VCMSLCFG").select('avg_rad')

# Boundary of South Africa
sa = ee.FeatureCollection("FAO/GAUL/2015/level0").filter(ee.Filter.eq("ADM0_NAME", "South Africa"))

# Get date to add in file name
def get_date(img):
    return img.date().format().getInfo()

# Collection to list
viirs_list = viirs.toList(viirs.size())

# Iterate over images
for i in range(viirs_list.size().getInfo()):
    img = ee.Image(viirs_list.get(i))
    projection = img.projection().getInfo()
    d = get_date(img)[:7] # Data is monthly, so this gets year and month
    print(d)
    ee.batch.Export.image.toDrive(
            image = img,
            description = 'Download South Africa Nightlights',
            region = sa, 
            crs = projection["crs"],
            crsTransform = projection["transform"],
            maxPixels = 1e13,
            folder = "south_africa_viirs_dnb_nightlights",
            fileNamePrefix = 'south_africa_viirs_dnb_monthly_v1_vcmslcfg__' + d.replace('-', '_'),
            fileFormat = 'GeoTIFF').start()

This code runs perfectly, but in my drive I get something very odd:

In particular: why are there different versions of the same image (the constructed names are all unique, as evident from the printout), and what are these numbers appended to the file name?

Sebastian · Accepted Answer · 2023-03-24T09:49:20.160

So the problem with the above code is that it tries to clip an image to a non-rectangular shape file and save that as GeoTIFF, which strangely enough executes, and generates multiple files. But you cannot do that, the region argument of ee.batch.Export.image.toDrive() expects a rectangular boundary, created e.g. with ee.Geometry.Rectangle([minlon, minlat, maxlon, maxlat]). So the way to go about it is to save a rectangular region around the geometry you want to export, and then set all values outside your geometry to a no-data value using a mask. Full code for the South Africa Nightlights example:

import ee
try:
        ee.Initialize()
except Exception as e:
        ee.Authenticate()
        ee.Initialize()

# Nightlights
viirs = ee.ImageCollection("NOAA/VIIRS/DNB/MONTHLY_V1/VCMSLCFG").select('avg_rad')

# Get a list of all the image IDs (I'll use it to fetch images directly using ee.Image())
image_ids = viirs.aggregate_array("system:id").getInfo()

# Boundary of South Africa
sa_boundary = ee.FeatureCollection("FAO/GAUL/2015/level0").filter(ee.Filter.eq("ADM0_NAME", "South Africa"))

# Function to mask all values outside the geometry
def maskOutside(image, geometry):
    mask = ee.Image.constant(1).clip(geometry).mask() # add .not() to mask inside
    return image.updateMask(mask)

# Download each image
for image_id in image_ids:
    image = ee.Image(image_id).select('avg_rad')

    # Replace all values outside region with -9999
    image = maskOutside(image, sa_boundary).unmask(-9999)

    # Get the original CRS and geotransform of the image
    proj = image.projection().getInfo()

    # Create a filename for the downloaded image
    filename = image_id.split("/")[-1]

    # Export the image with the original CRS and geotransform
    task = ee.batch.Export.image.toDrive(
        image = image,
        region = sa_boundary.geometry().bounds(), # Or use custom ee.Geometry.Rectangle([minlon, minlat, maxlon, maxlat])
        description = filename,
        folder = "south_africa_viirs_dnb_nightlights_v1_vcmslcfg",
        crs = proj["crs"],
        crsTransform = proj["transform"],
        maxPixels = 1e13,
        fileFormat = "GeoTIFF"
    )
    task.start()

    print(f"Exporting {filename}...")

To see what you are doing before starting the download loop, you could generate a plot using e.g.

import geemap
Map = geemap.Map()
center = sa_boundary.geometry().centroid().getInfo()
Map.setCenter(center["coordinates"][0], center["coordinates"][1], 6) 
Map.addLayer(viirs.first().clip(sa_boundary), {}, "Layer")
Map.addLayer(sa_boundary.geometry().bounds(), {}, 'Rectangle')
Map

Exporting may take a while, you can go to https://code.earthengine.google.com/tasks to see the progress and any errors for your tasks on the server.

Even though the question may not have been that useful, I hope this answer is useful for all who want to bulk-download earth engine imagery for specific countries or regions at original resolution using the python API.

Exporting TIF Images from Google Earth Engine to Google Drive: Minimal Example in Python

1 Answers1