Weighted violinplot

Question

I have a Pandas dataframe of data on generator plant capacity (MW) by fuel type. I wanted to show the estimated distribution of plant capacity in two different ways: by plant (easy) and by MW (harder). Here's an example:

# import libraries
import pandas as pd
import numpy as np
import seaborn as sns

# generate empty dataframe
df = pd.DataFrame(data=None,columns=['Fuel','MW'])

# create and seed a randomstate object (to make #s repeatable below)
rnd = np.random.RandomState(7)

# generate fake data for each fuel type and append to df
for myfuel in ['Biomass','Coal','Hydro','Natural Gas','Oil','Solar','Wind','Other']:
    mymean = rnd.uniform(low=2.8,high=3.2)
    mysigma = rnd.uniform(low=0.6,high=1.0)
    df = df.append(
                   pd.DataFrame({'Fuel': myfuel,
                        'MW': np.array(rnd.lognormal(mean=mymean,sigma=mysigma,size=1000))
                       }),
                   ignore_index=True
                   )

# make violinplot
sns.violinplot(x = 'Fuel',
               y = 'MW',
               data=df,
               inner=None,
               scale='area',
               cut=0,
               linewidth=0.5
              )

And here's the plot of the estimated distributions of plant size by MW this code makes:

This violinplot is very deceptive, without more context. Because it is not weighted, the thin tail at the top of each category hides the fact that the relatively few plants in the tail contain a lot of (maybe even most of) the MWs of capacity. So I want a second plot with the distribution by MWs--basically a weighted version of this first violinplot.

I wanted to know if anyone has figured out an elegant way to make such a "weighted" violinplot, or if anyone has an idea about the most elegant way to do that.

I figured I could loop through each row of my plant-level dataframe and decompose the plant data (into a new dataframe) into MW-level data. For instance, for a row in the plant-level dataframe that shows a plant with 350 MW, I could decompose that into 3500 new rows of my new dataframe, each representing 100 kW of capacity. (I think I have to go to at least the 100 kW level of resolution, because some of these plants are pretty small, in the 100 kW range.) That new dataframe would be enormous, but I could then do a violinplot of that decomposed data. That seemed slightly brute force. Any better ideas for approach?

Update:

I implemented the brute force method described above. Here's what it looks like if anyone is interested. This is not the "answer" to this question, because I still would be interested if anyone knows a more elegant/simple/efficient way to do this. So please chime in if you know of such a way. Otherwise, I hope this brute force approach might be helpful to someone in the future.

So that it's easy to see that the weighted violinplot makes sense, I replaced the random data with a simple uniform series of numbers from 0 to 10. Under this new approach, the violinplot of df should look pretty uniform, and the violinplot of the weighted data (dfw) should get steadily wider towards the top of the violins. That's exactly what happens (see image of violinplots below).

# import libraries
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt

# generate empty dataframe
df = pd.DataFrame(data=None,columns=['Fuel','MW'])

# generate fake data for each fuel type and append to df
for myfuel in ['Biomass','Coal','Hydro','Natural Gas','Oil','Solar','Wind','Other']:
    df = df.append(
                   pd.DataFrame({'Fuel': myfuel,
                        # To make it easy to see that the violinplot of dfw (below)
                        # makes sense, here we'll just use a simple range list from
                        # 0 to 10
                        'MW': np.array(range(11))
                       }),
                   ignore_index=True
                   )

# I have to recast the data type here to avoid an error when using violinplot below
df.MW = df.MW.astype(float)

# create another empty dataframe
dfw = pd.DataFrame(data=None,columns=['Fuel','MW'])
# since dfw will be huge, specify data types (in particular, use "category" for Fuel to limit dfw size)
dfw = dfw.astype(dtype={'Fuel':'category', 'MW':'float'})

# Define the MW size by which to normalize all of the units
# Careful: too big -> loss of fidelity in data for small plants
#          too small -> dfw will need to store an enormous amount of data
norm = 0.1 # this is in MW, so 0.1 MW = 100 kW

# Define a var to represent (for each row) how many basic units
# of size = norm there are in each row
mynum = 0

# loop through rows of df
for index, row in df.iterrows():

    # calculate and store the number of norm MW there are within the MW of each plant
    mynum = int(round(row['MW']/norm))

    # insert mynum rows into dfw, each with Fuel = row['Fuel'] and MW = row['MW']
    dfw = dfw.append(
                   pd.DataFrame({'Fuel': row['Fuel'],
                                 'MW': np.array([row['MW']]*mynum,dtype='float')
                                 }),
                                 ignore_index=True
                    )


# Set up figure and axes
fig, (ax1, ax2) = plt.subplots(nrows=1, ncols=2, sharey='row')

# make violinplot
sns.violinplot(x = 'Fuel',
               y = 'MW',
               data=df,
               inner=None,
               scale='area',
               cut=0,
               linewidth=0.5,
               ax = ax1
              )   

# make violinplot
sns.violinplot(x = 'Fuel',
               y = 'MW',
               data=dfw,
               inner=None,
               scale='area',
               cut=0,
               linewidth=0.5,
               ax = ax2
              ) 

# loop through the set of tick labels for both axes
# set tick label size and rotation
for item in (ax1.get_xticklabels() + ax2.get_xticklabels()): 
    item.set_fontsize(8)
    item.set_rotation(30)
    item.set_horizontalalignment('right')

plt.show()

Weighted violinplot

0 Answers0