Categorizing the data using pandas

Asked Oct 20 '18 at 10:48

Active Oct 20 '18 at 15:19

Viewed 86 times

enter image description hereI am trying to run a chi square test on a dataset and for that I need to use pd.cut() to formulate categories in the data set. However, I am getting this error

ufunc 'subtract' did not contain a loop with signature matching types dtype('

My Code:

import pandas as pd
import numpy as np
import scipy as sp
import math

data_main = pd.read_csv("sample_survey.csv")
data = data_main.iloc[:, [1,2]]

data["wrkstat"] = data["wrkstat"].astype(str)
data["marital"] = data["marital"].astype(str)
cols = ['wrkstat', 'marital']

cut_points = ['Divorced', 'Married', 'Never Married', 'Seperated','Widowed']
label_names = ['Divorced1', 'Married', 'Never Married', 
'Seperated','Widowed']
data["Marital_Categories"] = pd.cut(data["marital"], cut_points)

marital_by_wrkstat = data[['wrkstat', 'marital_categories']]
marital_by_wrkstat.head()

edited Oct 20 '18 at 15:19

asked Oct 20 '18 at 10:48

Aditya Shah

pd.cut uses continuous variables. In this case, your marital column are string types. You might want to consider using `.astype('categorical')` or explore more here https://pandas.pydata.org/pandas-docs/stable/categorical.html – BernardL Oct 20 '18 at 14:37
2

could you please add a sample from your data? – nimrodz Oct 20 '18 at 14:42
BernadL I tried but it says 'categorical' data type not understood – Aditya Shah Oct 20 '18 at 15:20
nimrodz sample added – Aditya Shah Oct 20 '18 at 15:20
Try it df[columnname] = pd.Categorical(df[columnname]) – dimension Oct 20 '18 at 15:44
what is your expected output for data["Marital_Categories"]? – nimrodz Oct 20 '18 at 15:47
@nimrodz and @BernardL, to convert to categorical data type, you need `.astype('category')`, not `.astype('categorical')`. – Peter Leimbigler Oct 20 '18 at 15:59
https://codingdisciple.com/chi-squared-python.html Nimrodz I want to categorize the data and have count as shown in the link – Aditya Shah Oct 20 '18 at 16:02
M.r_L still doesn't work – Aditya Shah Oct 20 '18 at 16:07
@Aditya Shah what is the difference between data["marital"] and data["Marital_Categories"]? you want that the categories would be `int` group number? – nimrodz Oct 20 '18 at 16:19
My goal is to get a distribution in such a way that I can have separate counts of each category based on their status in the wrkstat column. The idea is to perform the chi square test as mentioned in the link – Aditya Shah Oct 20 '18 at 16:40

Categorizing the data using pandas

0 Answers0