0

I'd like to make Backpropagation algorithm for Titanic (Kaggle's competition).

It's very easy to do but I have one problem. Backpropagation algorithm is about numbers. But we have string types in Titanic. For example,

Kaggle's Titanic

We have a column "Sex" (male or female), "Embarked" (S, C or Q). I will not use "Name" and "Ticket". I don't know what I can do with column "Sex" and "Embarked". How can I use these things for Backpropagation algorithm.

My idea how to resolve this problem I think I can use number int=1 for male and number int=2 for female. Also I can use number 1 for S and 2 for C and 3 for Q.

What do you think about my idea? How can I made Backpropagation algorithm for Titanic? (maybe my idea is not very good).

Community
  • 1
  • 1
vjg
  • 131
  • 1
  • 13

1 Answers1

2

My idea how to resolve this problem I think I can use number int=1 for male and number int=2 for female. Also I can use number 1 for S and 2 for C and 3 for Q.

This is almost correct. For features that only have two values encoding with 0 and 1 is fine. However, if you have categorical feature with more than 2 possible values you should never encode them as a single number. This kind of encoding can be used only for ordinal features, meaning, that despite being "not numbers" they do have order. For example if you have "small, big, huge" then you can encode them as 1, 2, 3 but if your values are completely unorderable you should one hot encode them, meaning that something that has K possible values becomes K bits in your representation, and you simply put 1 in a given position (leading to (red, blue, green) encoded as [1 0 0], [0 1 0], [0 0 1].

What happens if you do not follow this rule? Lets say that your feature is a name, and you have fours possibilities "James, Mark, Eve, Kate" which you encode as 1, 2, 3, 4. Then, if your model learns that James and Eve get good scores, while Kate does not, it will (or at least there is a decent probability it will) deduce that Mark has good scores as well, due to the fact that from models perspective there are no names, just arbitrary feature, and someone with value 1 is good, with value 3 is good, and with 4 is bad, so assuming that "2" is good as well is a decent reasoning. Notice, that it is just a result of encoding, and if you just order names differently - you get different results. This will not happen with one-hot encoding.

This is what is usually done in ML, if you want to use a method that is defined only on R^d spaces. Of course if one knows more about the feature you can sometimes come up with better encoding, for example colors can be decomposed to their RGB representation so you encode a similarity between red, orange and yellow directly. In general - one hot encoding is the safest approach, any other introduces some assumptions which can misguide your model (if they are false).

lejlot
  • 64,777
  • 8
  • 131
  • 164