Choropleth: animation_frame does not working to create the graph on colab

Question

I am facing a problem with Cloropleth when I try to use the animation_frame and geojson together. If I run my code using the geojson on the colab, it generates the graph. However, if I try to animate it, the colab runs out of time and does not generate the graph. I ready tried the solution proposed here, but it does not work for me. I am creating my code based on this tutorial. Did someone have the same problem?

I am using this code:

import pandas as pd
import plotly.express as px
import json
from urllib.request import urlopen    

df_corona_brio_clean = pd.read_csv('https://raw.githubusercontent.com/Yuri-Nassar/public_datasets/master/coronavirus/source_brasil_io/caso_covid_brio_filtered.csv', sep=';')
df_corona_brio_clean['date_str'] = df_corona_brio_clean['date'].apply(lambda x: str(x))
df_corona_brio_clean['date'] = pd.to_datetime(df_corona_brio_clean['date'], format = '%Y-%m-%d')

df_agrupado_date_state = df_corona_brio_clean_2.groupby(['date_str','state_name'])['confirmed', 'deaths', 'estimated_population'].sum().reset_index()
df_agrupado_date_state.sort_values(by=['date_str','confirmed'], ascending=True)

with urlopen('https://raw.githubusercontent.com/codeforamerica/click_that_hood/master/public/data/brazil-states.geojson') as response:
  Brazil = json.load(response) # Javascrip object notation 

state_id_map = {}
for feature in Brazil ['features']:
  feature['id'] = feature['properties']['name']
  state_id_map[feature['properties']['sigla']] = feature['id']

fig = px.choropleth(df_agrupado_date_state,
                    locations='state_name',
                    geojson = Brazil,
                    color= df_agrupado_date_state['confirmed'],
                    color_continuous_scale= 'Reds',
                    hover_name='state_name',
                    hover_data=['confirmed', 'deaths'],
                    title='Number of cases over the time',
                    #animation_frame= 'date_str'
                    #animation_group= 'date_str'
                    
                    #animation_frame= df_agrupado_date_state['date'].dt.strftime('%Y-%m-%d'),
                    #animation_group= df_agrupado_date_state['date'].dt.strftime('%Y-%m-%d')
                  )
fig.update_geos(fitbounds = "locations", visible = False)
fig.show()

I tried both datetime and str type for the date variable. I got the following fig without using animate_frame:

However, if I try any of these types, they will not generate any graph, and the colab disconnects to restart. I got the following log on colab while it tries to build the graph:

Somebody can help me with any idea? Tkx

Answer 1

• there is a resource issue if you attempt to create animation frames for all dates.
  • Have limited to last dates using a constant
  • if you simplify the geometry the resource usage reduces significantly. Have used geopandas to do this
• have aggregated data using named aggregations
• have use featureidkey to eliminate need to manipulate geojson
• have added mapbox solution too. It's resource usage also depends highly on complexity of geometry

import pandas as pd
import plotly.express as px
import json
from urllib.request import urlopen

df_corona_brio_clean = pd.read_csv(
    "https://raw.githubusercontent.com/Yuri-Nassar/public_datasets/master/coronavirus/source_brasil_io/caso_covid_brio_filtered.csv",
    sep=";",
)
df_corona_brio_clean["date_str"] = df_corona_brio_clean["date"].apply(lambda x: str(x))
df_corona_brio_clean["date"] = pd.to_datetime(
    df_corona_brio_clean["date"], format="%Y-%m-%d"
)

with urlopen(
    "https://raw.githubusercontent.com/codeforamerica/click_that_hood/master/public/data/brazil-states.geojson"
) as response:
    Brazil = json.load(response)  # Javascrip object notation

SIMPLIFY=True
if SIMPLIFY:
    import geopandas as gpd
    gdf = gpd.GeoDataFrame.from_features(Brazil, crs="epsg:4326")
    gdf["geometry"] = gdf.to_crs(gdf.estimate_utm_crs()).simplify(5000).to_crs("epsg:4326")
    Brazil = gdf.__geo_interface__
    
df_agrupado_date_state = (
    df_corona_brio_clean.groupby(["date", "state"], as_index=False)
    .agg(
        **{
            **{c: (c, "first") for c in ["date_str", "state_name"]},
            **{c: (c, "sum") for c in ["confirmed", "deaths", "estimated_population"]},
        }
    )
    .sort_values(["date", "confirmed"])
)

MAX_DATES = 15
fig = px.choropleth(
    df_agrupado_date_state.loc[
        lambda d: d["date"].isin(d["date"].unique()[-MAX_DATES:])
    ],
    locations="state",
    geojson=Brazil,
    featureidkey="properties.sigla",
    color="confirmed",
    color_continuous_scale="Reds",
    hover_name="state_name",
    hover_data=["confirmed", "deaths"],
    title="Number of cases over the time",
    animation_frame="date_str",
)
fig.update_geos(fitbounds="locations", visible=False).update_layout(margin={"l":0,"r":0,"t":30,"b":0})

mapbox

fig = px.choropleth_mapbox(
    df_agrupado_date_state.loc[
        lambda d: d["date"].isin(d["date"].unique()[-MAX_DATES:])
    ],
    locations="state",
    geojson=Brazil,
    featureidkey="properties.sigla",
    color="confirmed",
    color_continuous_scale="Reds",
    hover_name="state_name",
    hover_data=["confirmed", "deaths"],
    title="Number of cases over the time",
    animation_frame="date_str",
)
fig.update_layout(
    margin={"l": 0, "r": 0, "t": 30, "b": 0},
    mapbox={
        "style": "carto-positron",
        "zoom": 2,
        "center": {"lat": -14.2350, "lon": -51.9253},
    },
)

data sourcing

• in comments you note you are not getting each state for each date
• clearly not your data source does not support this
• change to this source which has cases by state in Brazil https://github.com/wcota/covid19br

df_corona_brio_clean = pd.read_csv("https://raw.githubusercontent.com/wcota/covid19br/master/cases-brazil-states.csv")
df_corona_brio_clean = df_corona_brio_clean.loc[lambda d: d["state"].ne("TOTAL")]
df_corona_brio_clean["date"] = pd.to_datetime(df_corona_brio_clean["date"])
df_corona_brio_clean["date_str"] = df_corona_brio_clean["date"].dt.strftime("%Y-%m-%d")
df_agrupado_date_state = df_corona_brio_clean
MAX_DATES = 15
fig = px.choropleth(
    df_agrupado_date_state.loc[
        lambda d: d["date"].isin(d["date"].unique()[-MAX_DATES:])
    ],
    locations="state",
    geojson=Brazil,
    featureidkey="properties.sigla",
    color="newCases",
    color_continuous_scale="Reds",
    hover_name="state",
    hover_data=["newCases", "newDeaths"],
    title="Number of cases over the time",
    animation_frame="date_str",
)
fig.update_geos(fitbounds="locations", visible=False).update_layout(margin={"l":0,"r":0,"t":30,"b":0})