0

I have a DataFrame containing 2 columns of ordered categorical data (of the same category). I want to construct another column that contains the categorical maximum of the first 2 columns. I set up the following.

import pandas as pd
from pandas.api.types import CategoricalDtype
import numpy as np

cats = CategoricalDtype(categories=['small', 'normal', 'large'], ordered=True)
data = {
    'A': ['normal', 'small', 'normal', 'large', np.nan],
    'B': ['small', 'normal', 'large', np.nan, 'small'],
    'desired max(A,B)': ['normal', 'normal', 'large', 'large', 'small']
}
df = pd.DataFrame(data).astype(cats)

The columns can be compared, although the np.nan items are problematic, as running the following code shows.

df['A'] > df['B']

The manual suggests that max() works on categorical data, so I try to define my new column as follows.

df[['A', 'B']].max(axis=1)

This yields a column of NaN. Why?

David de Wit
  • 23
  • 4

2 Answers2

0

The following code constructs the desired column using the comparability of the categorical columns. I still don't know why max() fails here.

dfA = df['A']
dfB = df['B']
conditions = [dfA.isna(), (dfB.isna() | (dfA >= dfB)), True]
cases = [dfB, dfA, dfB]
df['maxAB'] = np.select(conditions, cases)
David de Wit
  • 23
  • 4
-1

Columns A and B are string-types. So you gotta assign integer values to each of these categories first.

# size string -> integer value mapping
size2int_map = {
    'small': 0, 
    'normal': 1, 
    'large': 2
}

# integer value -> size string mapping
int2size_map = {
    0: 'small', 
    1: 'normal', 
    2: 'large'
}

# create columns containing the integer value for each size string
for c in df:
    df['%s_int' % c] = df[c].map(size2int_map)

# apply the int2size map back to get the string sizes back
print(df[['A_int', 'B_int']].max(axis=1).map(int2size_map))

and you should get

0    normal
1    normal
2     large
3     large
4     small
dtype: object
LogCapy
  • 447
  • 7
  • 20