using DictVectorizer to convert strings

Question

satisfaction_level  last_evaluation number_project  average_montly_hours    time_spend_company  Work_accident   left    promotion_last_5years   dept    salary
0.38    0.53    2   157 3   0   1   0   TECHNICAL   low
0.8 0.86    5   262 6   0   1   0   HR  medium
0.11    0.88    7   272 4   0   1   0   HR  medium
0.72    0.87    5   223 5   0   1   0   FINANCE low
0.37    0.52    2   159 3   0   1   0   MARKETING   low
0.41    0.5 2   153 3   0   1   0   TECHNICAL   low
0.1 0.77    6   247 4   0   1   0   HR  low
0.92    0.85    5   259 5   0   1   0   FINANCE low
0.89    1   5   224 5   0   1   0   HR  low

I used the above data and tried to convert C using DictVectorizer. Code is given below

import pandas as pd
from sklearn.feature_extraction import DictVectorizer

dv=DictVectorizer() 
hr_data=pd.read_csv(r"C:\Users\IBM_ADMIN\Desktop\data\HR_comma_sep.csv")
dv.fit_transform(X=hr_data.dept)

But it threw error:

'str' object has no attribute 'items'

score 4 · Answer 1 · answered Nov 16 '17 at 16:36

Use LabelEncoder:

In [304]: df.dept
Out[304]:
0    TECHNICAL
1           HR
2           HR
3      FINANCE
4    MARKETING
5    TECHNICAL
6           HR
7      FINANCE
8           HR
Name: dept, dtype: object

In [305]: from sklearn.preprocessing import LabelEncoder

In [306]: le = LabelEncoder()

In [307]: df['dept'] = le.fit_transform(df['dept'])

In [308]: df.dept
Out[308]:
0    3
1    1
2    1
3    0
4    2
5    3
6    1
7    0
8    1
Name: dept, dtype: int64

In [309]: le.classes_
Out[309]: array(['FINANCE', 'HR', 'MARKETING', 'TECHNICAL'], dtype=object)

score 2 · Answer 2 · answered Nov 16 '17 at 16:39

2

Or we may using category

df.dept.astype('category').cat.codes
Out[925]: 
0    3
1    1
2    1
3    0
4    2
5    3
6    1
7    0
8    1
dtype: int8

df.dept.astype('category').cat.categories
Out[926]: Index(['FINANCE', 'HR', 'MARKETING', 'TECHNICAL'], dtype='object')

answered Nov 16 '17 at 16:39

BENY

317,841
20
164
234

Nice and simple. – cs95 Nov 16 '17 at 17:47

koras · Answer 3 · 2019-06-04T21:42:15.167

0

This is an old question but I had the same problem and solved it: First, change the data frame to the dictionary

data_dict = data.to_dict()

dv = DictVectorizer(sparse = False, dtype = int)
dv.fit_transform(data['column'])

Worked for me!

edited Jun 04 '19 at 21:42

answered Jun 04 '19 at 21:01

koras

15
5

1

Did you mean to put `dv.fit_transform(data_dict['column'])` ? You create the dictionary but never use it. – Kevin Jun 04 '19 at 21:24

score 0 · Answer 4 · answered Jun 04 '19 at 21:46

0

Actually, you should try this as well:

If the values are strings, then transform them into numerical categories first using LabelEncoder, then use DictVectorizer:

le = LabelEncoder()
data_le = le.fit_transform(data)

answered Jun 04 '19 at 21:46

koras

15
5

score 0 · Answer 5 · answered Mar 12 '20 at 14:41

It might be because DictVectorizer is a class intended for dictionaries while you are applying to a Series object (the result of accessing a column by dictionary or attribute style). For this case I would suggest the class OneHotEncoder from sklearn.preprocessing.

DictVectorizer works for dictionaries:

vec = DictVectorizer(sparse = False, dtype = int)
data = [{'character': 'Porthos'},{'character': 'Athos'},
        {'character': 'Aramis'}, {'character': 'Athos'},
        {'character': 'Aramis'}

vec.fit_transform(data)
array([[0, 0, 1],
       [0, 1, 0],
       [1, 0, 0],
       [0, 1, 0],
       [1, 0, 0]])

But not so much for Series or DataFrames:

data = pd.DataFrame(data)
vec.fit_transform(data)

*long error*
AttributeError: 'str' object has no attribute 'items'

For Pandas' DataFrames and Series use OneHotEncoder:

from sklearn.preprocessing import OneHotEncoder
vec = OneHotEncoder(sparse = False, dtype = int)
vec.fit_transform(data)
array([[0, 0, 1],
       [0, 1, 0],
       [1, 0, 0],
       [0, 1, 0],
       [1, 0, 0]])

using DictVectorizer to convert strings

5 Answers5

Linked