Prediction of dependent variable with Keras shows the same result for the entire array & Confusion Matrix throws error

Question

I am building a ANN with Keras. The shape of my df is (120000,18). My goal is to predict based on 17 independent variables(X's) what my dependent variable(Y) will be. I have 2 questions which I added below. Here is my code:

Creating ANN

Question 1: How can all my values for y_pred_train for my Training set be the same value? Also, the predictions should show a binary result, meaning 0 or 1, meaning if prediction will be true vs false. Why am I getting 0.41542563?

Data Preprocessing
# Splitting the dataset into the Training set and Test set
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, random_state = 0)

# Feature Scaling
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)

#ANN
import keras
from keras.models import Sequential
from keras.layers import Dense

# Initialising the ANN
classifier = Sequential()
# Adding the input layer and the first hidden layer
classifier.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'relu', input_dim = 17))
# Adding the second hidden layer
classifier.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'relu'))
# Adding the output layer
classifier.add(Dense(units = 1, kernel_initializer = 'uniform', activation = 'sigmoid'))
# Compiling the ANN
classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])

# Fitting the ANN to the Training set
classifier.fit(X_train, y_train, batch_size = 10, epochs = 100)

Snippet of epoch at point of convergence: 7210/80030 [=>............................] - ETA: 8s - loss: 0.6822 - acc: 0.7046

# Classifying the Train set results
y_pred_train = classifier.predict(X_train)
y_pred_train

Out[50]: 
array([[0.41542563],
       [0.41542563],
       [0.41542563],
       ...,
       [0.41542563],
       [0.41542563],
       [0.41542563]], dtype=float32)

Creating the Confusion Matrix

Question 2: When I try to execute a Confusion matrix, I get an error.

from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred)

ValueError-Traceback (most recent call last)
<command-207679> in <module>()
      1 # Making the Confusion Matrix
      2 from sklearn.metrics import confusion_matrix
----> 3 cm = confusion_matrix(y_test, y_pred)

/databricks/python/lib/python3.5/site-packages/sklearn/metrics/classification.py in confusion_matrix(y_true, y_pred, labels, sample_weight)
    238 
    239     """
--> 240     y_type, y_true, y_pred = _check_targets(y_true, y_pred)
    241     if y_type not in ("binary", "multiclass"):
    242         raise ValueError("%s is not supported" % y_type)

/databricks/python/lib/python3.5/site-packages/sklearn/metrics/classification.py in _check_targets(y_true, y_pred)
     70     y_pred : array or indicator matrix
     71     """
---> 72     check_consistent_length(y_true, y_pred)
     73     type_true = type_of_target(y_true)
     74     type_pred = type_of_target(y_pred)

/databricks/python/lib/python3.5/site-packages/sklearn/utils/validation.py in check_consistent_length(*arrays)
    179     if len(uniques) > 1:
    180         raise ValueError("Found input variables with inconsistent numbers of"
--> 181                          " samples: %r" % [int(l) for l in lengths])
    182 
    183 

ValueError: Found input variables with inconsistent numbers of samples: [34299, 22866]

Your dense layers only have 1 unit each. They are not learning at all and just passing the values along. Hence, all the outputs are same. — Raunaq Jain, Aug 19 '18 at 19:25
@RaunaqJain Thank you but when I change units=10 for example, I get an error saying "ValueError: Error when checking target: expected dense_21 to have shape (10,) but got array with shape (1,)". I checked my shape of the df and it is (114329, 18). Why is it recognizing shape (1,)? — LaLaTi, Aug 19 '18 at 19:42
From what I can see, the problem is occurring because of the batch_size argument in the call to fit. You may read it up more over here https://stackoverflow.com/questions/44747343/keras-input-explanation-input-shape-units-batch-size-dim-etc/44748370 — Raunaq Jain, Aug 19 '18 at 19:51
What should I change my batch_size to? What do you recommend? — LaLaTi, Aug 19 '18 at 20:01
I am not used to Keras but setting `batch_size = None` should work. — Raunaq Jain, Aug 19 '18 at 20:04
Unfortunately, I get the same error. Maybe someone else knows? Also, do you have any idea about Question2? The confusion matrix shows error. Would you know why it is? — LaLaTi, Aug 19 '18 at 20:07
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/178311/discussion-between-raunaq-jain-and-tina). — Raunaq Jain, Aug 19 '18 at 20:11

score 0 · Answer 1 · answered Aug 19 '18 at 19:38

0

Your dense layers only have 1 unit each. Except the first layer, rest of the layers are just passing the values along. Hence, all the outputs are same.

As for your second question, the activation function on the last layer is sigmoid, which gives an output in the range of 0-1. You will have to convert them based on a threshold. For example,

y_pred = [1 if x>0.5 else 0 for x in y_pred], in which 0.5 is the threshold.

answered Aug 19 '18 at 19:38

Raunaq Jain

917
7
13

I just commented back on your answer. For the 1st problem of unit=1, I now changed it to unit=10 and get an error: ValueError: Error when checking target: expected dense_21 to have shape (10,) but got array with shape (1,). Why is it receiving shape (1,)? – LaLaTi Aug 19 '18 at 19:46
Also, for the second question, why do you pick 0.5 as the threshold? Is that random or does it mean something? – LaLaTi Aug 19 '18 at 19:48
Threshold may be anything. It is up to you. Generally, 0.5 is taken as a threshold because the values represent probabilities. Hence, if a value is greater than 0.5, it means that the probability of it being 1 is more than 50% – Raunaq Jain Aug 19 '18 at 19:54
perfect, thanks. And what will solve the issue with the confusion matrix? – LaLaTi Aug 19 '18 at 20:02

Prediction of dependent variable with Keras shows the same result for the entire array & Confusion Matrix throws error

Creating ANN

Creating the Confusion Matrix

1 Answers1