Stack Overflow
Stack Overflow

Questions tagged [one-hot-encoding]

One-Hot Encoding is a method to encode categorical variables to numerical data that Machine Learning algorithms can deal with. One-Hot encoding is most used during feature engineering for a ML Model. It converts categorical values into a new categorical column and assign a binary value of 1 or 0 to those columns.

Also known as Dummy Encoding, One-Hot Encoding is a method to encode categorical variables, where no such ordinal relationship exists, to numerical data that Machine Learning algorithms can deal with. One hot encoding is the most widespread approach, and it works very well unless your categorical variable takes on a large number of unique values. One hot encoding creates new, binary columns, indicating the presence of each possible value from the original data. These columns store ones and zeros for each row, indicating the categorical value of that row.

1224 questions
4
votes
1 answer

Python Pandas get_dummies() limitation. Doesnt convert all columns

I have 6 columns in my dataframe. 2 of them have about 3K unique values. When I use get_dummies() on the entire dataframe or just one those 2 columns what gets returned is the exact same column with 3k values. get_dummies fails to dummy-fy the…
cryp
  • 2,285
  • 3
  • 26
  • 33
4
votes
2 answers

How can I encode features with more than one value per column? MultiDictVectorizer needed?

I am vectorizing some features in sklearn, and I have run into a problem. DictVectorizer works well if your data can be encoded into one dict key per item. What if your items can have two or more values of the same column? For instance,…
rjurney
  • 4,824
  • 5
  • 41
  • 62
3
votes
5 answers

One-hot encoding when data is stored across multiple columns

Say I have a dataframe primary_color secondary_color tertiary_color red blue green yellow red NA and i want this to encode by checking if the color exists across any of the three columns (1) or none of the 3 columns (0). So, it should…
user276238
  • 107
  • 6
3
votes
1 answer

Create One hot labels, one Hot encoding based on multiple condition

For example i have the following: [1,2,3,5] and I want to hot encode it. It usually looks like this: [1,0,0,0,0] [0,1,0,0,0] [0,0,1,0,0] [0,0,0,0,1] But instead of that, I want to have a conditional one hot encoding and only two classes. All values…
cocojambo
  • 63
  • 2
  • 5
3
votes
3 answers

How to One-Hot Encoding stacked columns in R

I have data that look like this +---+-------+ | | col1 | +---+-------+ | 1 | A | | 2 | A,B | | 3 | B,C | | 4 | B | | 5 | A,B,C | +---+-------+ Expected Output +---+-----------+ | | A | B | C | +---+-----------+ |1 | 1 | 0 | 0 | |2…
BERKz
  • 45
  • 4
3
votes
2 answers

pivot long form categorical data by group and dummy code categorical variables

For the following dataframe, I am trying to pivot the categorical variable ('purchase_item') into wide format and dummy code them as 1/0 - based on whether or not a customer purchased it in each of the 4 quarters within 2016. I would like to…
veg2020
  • 956
  • 10
  • 27
3
votes
1 answer

What should be the format of one-hot-encoded features for scikit-learn?

I am trying to use the regressor/classifiers of scikit-learn library. I am a bit confused about the format of the one-hot-encoded features since I can send dataframe or numpy arrays to the model. Say I have categorical features named 'a', 'b' and…
MehmedB
  • 1,059
  • 1
  • 16
  • 42
3
votes
1 answer

Scikit-learn Column Transformer does not return back feature names

I'm trying to use Column Transformer with OneHotEncoder to transform my categorical data : A quick look at my data : I want to do one-hot-encoding for 3 features : 'sex' , 'smoker' , 'region', so I use Column Transformer by scikit-learn. ( I don't…
Miku
  • 79
  • 1
  • 7
3
votes
2 answers

How to convert one (comma split) column into multiple columns in R?

For example, I have this data: data <- data.frame(person=paste0("person_", 1:5), keyword=sapply(1:5, function(x) paste0(sample(letters, sample(1:5, 1)), collapse = ",")) ) > data person keyword 1 person_1…
achai
  • 199
  • 1
  • 7
3
votes
1 answer

Convert list to binary values using one-hot encoding

I have one column in CSV file. Each cell in the column has multiple values in a list. For e.g. one cell would contain ['A', 'B', 'C'] and the other ['B', 'D']. I want to apply one-hot encoding to this column to convert to binary values to use for…
Raghav
  • 33
  • 4
3
votes
1 answer

How can I prevent TextVectorization in Tensorflow creating values for Unknown and blank strings?

I am looking to one hot encode string tensor as part of my dataset pipeline. It seems to me this can be achieved using TextVectorization to get an integer representation of the string tensor and then one_hot to convert to achieve the encoded 2d…
DataJack
  • 341
  • 2
  • 13
3
votes
1 answer

How do I one hot encode along a specific dimension using PyTorch?

I have a tensor of size [3, 15, 136], where: 3 is batch size 15 - sequence length and 136 is tokens I want to one-hot my tensor using the probabilities in the tokens dimension (136). To do so I want to extract the tokens dimension for each letter…
julliet
  • 147
  • 10
3
votes
2 answers

Python DataFrame: One-Hot Encode Rows Containing a Specific Substring

I have a DataFrame containing strings. I would like to create another DataFrame that indicates whether the string contains a specific month through one-hot encoding. Using the below as an example: data = { 'User': ['1', '2', '3', '4'] 'Months':…
student214
  • 95
  • 9
3
votes
1 answer

One hot coding in Train Validation and Test set (Production data)

For example I have below train set. name values 0 Tony 100 1 Smith 110 2 Sam 120 3 Shane 130 4 Sam 140 5 Ram 160 After one hot encoding it becomes values 0 1 2 3 4 0 100 1 …
vsb
  • 428
  • 6
  • 15
3
votes
3 answers

How to include a OneHot in an ONNX coming from PyTorch

I'm using PyTorch to train neural-net and output them into ONNX. I use these models in a Vespa index, which loads ONNXs through TensorRT. I need one-hot-encoding for some features but this is really hard to achieve within the Vespa framework. Is it…
fweber
  • 355
  • 1
  • 2
  • 10
1 2 3
…
81 82