Trying a function to get data in separate ranges, tally it, and return a list of the tallies

Question

I have been trying to write a function that would take a value from a dictionary, check its range, then tally it in its range. Returning a list of the tallied values.

So given the following dictionary:

data={'P1': {'age': 'eighty two', 'salary': '96.0', 'suburb': 'Toorak', 'language': 'English'},
      'P2': {'age': '49', 'salary': '14.0', 'suburb': 'St. Kilda', 'language': 'Chinese'},
      'P3': {'age': '54', 'salary': '75.0', 'suburb': 'Neverland', 'language': 'Italian'}}

And the function code:

def wealth_distribution(data, n_bins, max_salary):
    count = 0
    sal_list = []
    bin_list = []
    bin_width = int(max_salary/n_bins)

    for bins in range(0, max_salary+1, bin_width):
        bin_list.append(bins)

        for val in data.values():
            if val['salary'] == None:
                continue
            for n in bin_list:
                if math.floor(n*bin_width)<=float(val['salary'])<math.floor((n+1)*bin_width):
                    count+= 1
            sal_list.append(count)
    return sal_list

Given n_bins = 5 and max_salary = 100, the desired output is [1,0,0,1,1].

But the function returns [0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 4, 4, 4, 5, 5, 5, 6, 6].

You are checking 3 salaries v 6 bins that end up being in `bin_list` that is why you have 18 results, could you maybe clarify what range and what values you want to know if they are in that range — vash_the_stampede, Oct 14 '18 at 16:06
Basically wanted to divide the range of 0 to max_salary by the number of n_bins so for the example given it would be 5 equal bins of size 20. e.g. 0-19,20-39 etc. then tally up the salaries that occur in each range, and output a list of the tallied bins. — Raquel Bisogno, Oct 14 '18 at 16:17

Stuart · Accepted Answer · 2018-10-14T18:20:18.983

First, you seem to have an indentation error - for val in data.values(): should not be nested within for bins in range(0, max_salary+1, bin_width): - that's why you are getting a longer list of values.

Second, your logic is a bit off in various ways - you keep a count variable that is only set to zero once, at the beginning of the function. for n in bin_list: loops through the values in bin_list, but you then multiply n by bin_width, which doesn't make sense. You could alter this using range(n_bins) to go through the indices of bin_lists, like this:

def wealth_distribution(data, n_bins, max_salary):
    sal_list = [0] * n_bins
    bin_list = []
    bin_width = int(max_salary/n_bins)
    for bins in range(0, max_salary+1, bin_width):
        bin_list.append(bins)
    for val in data.values():
        if val['salary'] == None:
            continue
        for i in range(n_bins):
            if math.floor(i*bin_width)<=float(val['salary'])<math.floor((i+1)*bin_width):
                sal_list[i] += 1
    return sal_list

But on closer inspection, bin_list is actually serving no purpose here. The function can be reduced to:

def wealth_distribution(data, n_bins, max_salary):
    sal_list = [0] * n_bins
    bin_width = max_salary/n_bins
    for val in data.values():
        if val['salary'] == None:
            continue
        bin_index = int(float(val["salary"]) / bin_width)
        if bin_index < n_bins:
            sal_list[bin_index] += 1
        else:  # salary = max_salary
            sal_list[n_bins-1] += 1
    return sal_list

The function above calculate the bin index rather than looping through the bins or indices. I also removed the math.floors as these seem unnecessary and could lead to some situations where a small rounding error would leave some salaries uncategorised.

You could simplify further using collections.Counter:

from collections import Counter
def wealth_distribution(data, n_bins, max_salary):
    bin_width = max_salary / n_bins
    bins = Counter(min(int(float(val["salary"]) // bin_width), n_bins-1)
               for val in data.values())
    return [bins[i] for i in range(n_bins)]

There is a histogram function in numpy that also does what you want, and as a bonus provides an array of the bin boundaries.

import numpy as np
salaries = [float(val["salary"]) for val in data.values()]
sal_list, bin_list = np.histogram(salaries, bins=5, range=(0, 100))

And if you want to use pandas... (might be useful for other operations on the same data)

import pandas as pd
def wealth_distribution(data, n_bins, max_salary):
    df = pd.DataFrame(data).transpose()
    bin_width = max_salary / n_bins
    df["salary_bin"] = (pd.to_numeric(df["salary"]) // bin_width).clip(upper=n_bins-1)
    counts = df["salary_bin"].value_counts()
    return counts.reindex(range(n_bins), fill_value=0).values

Thanks so much for your assistance. I can see where I was going wrong now. I'm only having one issue now. For the very last value in sal_list, the range doesn't include the maximum value of max_salary. — Raquel Bisogno, Oct 14 '18 at 17:07
Good point - I have edited the 2nd and 3rd versions so they work for salary = max_salary. The `numpy` function is a great option because it already takes care of this. — Stuart, Oct 14 '18 at 17:23

score 0 · Answer 2 · answered Oct 14 '18 at 16:26

Basically there were severally problems with the code that I fixed. A big issue, with what I believe you wanted, was your math if statement which I also fixed. This solution isn't the most efficient but works.

def wealth_distribution(data, n_bins, max_salary):
    count = 0
    bin_list = []
    bin_width = int(max_salary/n_bins)
    for bins in range(0, max_salary+1, bin_width):
        bin_list.append(bins)
    sal_list = [0]*len(bin_list)
    for val in data.values():
        if val['salary']:
            for index, n in enumerate(bin_list):
                if math.floor(n) <= float(val['salary']) < math.floor(n+bin_width):
                    sal_list[index] += 1
    return sal_list
print(wealth_distribution(data, 5, 100))

score 0 · Answer 3 · answered Oct 14 '18 at 17:03

I'm not sure exactly what's wrong with your code, except that it seems unnecessarily complicated.

Here's how I would do it:

from math import floor

def wealth_distribution(data, n_bins, max_salary):
    sal_list = [0 for _ in range(n_bins)]  # Pre-allocate salary counts.
    bin_width = max_salary // n_bins

    for item in data.values():
        salary = float(item['salary'])

        for i in range(n_bins):
            low = floor(float(i * bin_width))
            high = floor(float(low + bin_width))
            if (salary is not None) and (low <= salary < high):
                sal_list[i] += 1
                break

    return sal_list


data={
    'P1': {'age': 'eighty two', 'salary': '96.0', 'suburb': 'Toorak', 'language': 'English'},
    'P2': {'age': '49', 'salary': '14.0', 'suburb': 'St. Kilda', 'language': 'Chinese'},
    'P3': {'age': '54', 'salary': '75.0', 'suburb': 'Neverland', 'language': 'Italian'}
}


sal_list = wealth_distribution(data, 5, 100)
print(sal_list)  # -> [1, 0, 0, 1, 1]

score 0 · Answer 4 · answered Oct 14 '18 at 17:03

import pandas as pd
from pandas import DataFrame

def wealth_distribution(data, n_bins, max_salary):      

    sal_list = []
    bin_list = []
    bin_width = int(max_salary/n_bins)    
    for bins in range(0, max_salary+1, bin_width):
        bin_list.append(bins)
    sal_list = [0] * (len(bin_list) - 1)         
    df = pd.DataFrame(data)
    for sal in range(0,len(df) - 1):
        salary = float(df.loc['salary'][sal])
        for i in range(len(bin_list) - 1,-1,-1):
            if salary > bin_list[i]:
                sal_list[i] += 1                  
                break   

    return sal_list

Trying a function to get data in separate ranges, tally it, and return a list of the tallies

4 Answers4