24

I am new to Tensorflow and deep leaning. I am trying to see how the loss decreases over 10 epochs in my RNN model that I created to read a dataset from kaggle which contains credit card fraud data. I am trying to classify the transactions as fraud(1) and not fraud(0). When I try to run the below code I keep getting the below error:

> 2018-07-30 14:59:33.237749: W
> tensorflow/core/kernels/queue_base.cc:277]
> _1_shuffle_batch/random_shuffle_queue: Skipping cancelled enqueue attempt with queue not closed Traceback (most recent call last):  
> File
> "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py",
> line 1322, in _do_call
>     return fn(*args)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py",
> line 1307, in _run_fn
>     options, feed_dict, fetch_list, target_list, run_metadata)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py",
> line 1409, in _call_tf_sessionrun
>     run_metadata) tensorflow.python.framework.errors_impl.InvalidArgumentError: logits
> and labels must be broadcastable: logits_size=[1,2] labels_size=[1,24]
>    [[Node: softmax_cross_entropy_with_logits_sg =
> SoftmaxCrossEntropyWithLogits[T=DT_FLOAT,
> _device="/job:localhost/replica:0/task:0/device:CPU:0"](add, softmax_cross_entropy_with_logits_sg/Reshape_1)]]
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):   File
> "/home/suleka/Documents/untitled1/RNN_CrediCard.py", line 96, in
> <module>
>     train_neural_network(x)   File "/home/suleka/Documents/untitled1/RNN_CrediCard.py", line 79, in
> train_neural_network
>     _, c = sess.run([optimizer, cost], feed_dict={x: feature_batch, y: label_batch})   File
> "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py",
> line 900, in run
>     run_metadata_ptr)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py",
> line 1135, in _run
>     feed_dict_tensor, options, run_metadata)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py",
> line 1316, in _do_run
>     run_metadata)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py",
> line 1335, in _do_call
>     raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: logits
> and labels must be broadcastable: logits_size=[1,2] labels_size=[1,24]
>    [[Node: softmax_cross_entropy_with_logits_sg =
> SoftmaxCrossEntropyWithLogits[T=DT_FLOAT,
> _device="/job:localhost/replica:0/task:0/device:CPU:0"](add, softmax_cross_entropy_with_logits_sg/Reshape_1)]]
> 
> Caused by op 'softmax_cross_entropy_with_logits_sg', defined at:  
> File "/home/suleka/Documents/untitled1/RNN_CrediCard.py", line 96, in
> <module>
>     train_neural_network(x)   File "/home/suleka/Documents/untitled1/RNN_CrediCard.py", line 63, in
> train_neural_network
>     cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=prediction,
> labels=y))   File
> "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py",
> line 250, in new_func
>     return func(*args, **kwargs)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py",
> line 1968, in softmax_cross_entropy_with_logits
>     labels=labels, logits=logits, dim=dim, name=name)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py",
> line 1879, in softmax_cross_entropy_with_logits_v2
>     precise_logits, labels, name=name)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_nn_ops.py",
> line 7205, in softmax_cross_entropy_with_logits
>     name=name)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py",
> line 787, in _apply_op_helper
>     op_def=op_def)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py",
> line 3414, in create_op
>     op_def=op_def)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py",
> line 1740, in __init__
>     self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access
> 
> InvalidArgumentError (see above for traceback): logits and labels must
> be broadcastable: logits_size=[1,2] labels_size=[1,24]     [[Node:
> softmax_cross_entropy_with_logits_sg =
> SoftmaxCrossEntropyWithLogits[T=DT_FLOAT,
> _device="/job:localhost/replica:0/task:0/device:CPU:0"](add, softmax_cross_entropy_with_logits_sg/Reshape_1)]]

Can anyone point out what I am doing wrong in my code and also any problem in my code if possible. Thank you in advance.

Shown below is my code:


import tensorflow as tf
from tensorflow.contrib import rnn



# cycles of feed forward and backprop
hm_epochs = 10
n_classes = 2
rnn_size = 128
col_size = 30
batch_size = 24
try_epochs = 1
fileName = "creditcard.csv"

def create_file_reader_ops(filename_queue):
    reader = tf.TextLineReader(skip_header_lines=1)
    _, csv_row = reader.read(filename_queue)
    record_defaults = [[1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1]]
    col1, col2, col3, col4, col5, col6, col7, col8, col9, col10, col11, col12, col13, col14, col15, col16, col17, col18, col19, col20, col21, col22, col23, col24, col25, col26, col27, col28, col29, col30, col31 = tf.decode_csv(csv_row, record_defaults=record_defaults)
    features = tf.stack([col1, col2, col3, col4, col5, col6, col7, col8, col9, col10, col11, col12, col13, col14, col15, col16, col17, col18, col19, col20, col21, col22, col23, col24, col25, col26, col27, col28, col29, col30])
    return features, col31


def input_pipeline(fName, batch_size, num_epochs=None):
    # this refers to multiple files, not line items within files
    filename_queue = tf.train.string_input_producer([fName], shuffle=True, num_epochs=num_epochs)
    features, label = create_file_reader_ops(filename_queue)
    min_after_dequeue = 10000 # min of where to start loading into memory
    capacity = min_after_dequeue + 3 * batch_size # max of how much to load into memory
    # this packs the above lines into a batch of size you specify:
    feature_batch, label_batch = tf.train.shuffle_batch(
        [features, label],
        batch_size=batch_size,
        capacity=capacity,
        min_after_dequeue=min_after_dequeue)
    return feature_batch, label_batch


creditCard_data, creditCard_label = input_pipeline(fileName, batch_size, try_epochs)


x = tf.placeholder('float',[None,col_size])
y = tf.placeholder('float')


def recurrent_neural_network_model(x):
    #giving the weights and biases random values
    layer ={ 'weights': tf.Variable(tf.random_normal([rnn_size, n_classes])),
            'bias': tf.Variable(tf.random_normal([n_classes]))}

    x = tf.split(x, 24, 0)
    print(x)

    lstm_cell = rnn.BasicLSTMCell(rnn_size)
    outputs, states = rnn.static_rnn(lstm_cell, x, dtype=tf.float32 )
    output = tf.matmul(outputs[-1], layer['weights']) + layer['bias']

    return output

def train_neural_network(x):
    prediction = recurrent_neural_network_model(x)
    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=prediction, labels=y))
    optimizer = tf.train.AdamOptimizer().minimize(cost)


    with tf.Session() as sess:

        gInit = tf.global_variables_initializer().run()
        lInit = tf.local_variables_initializer().run()
        coord = tf.train.Coordinator()
        threads = tf.train.start_queue_runners(coord=coord)
        for epoch in range(hm_epochs):
            epoch_loss = 0

            for counter in range(101):
                    feature_batch, label_batch = sess.run([creditCard_data, creditCard_label])
                    print(label_batch.shape)
                    _, c = sess.run([optimizer, cost], feed_dict={x: feature_batch, y: label_batch})
                    epoch_loss += c
            print('Epoch', epoch, 'compleated out of', hm_epochs, 'loss:', epoch_loss)



train_neural_network(x)
PatXio
  • 129
  • 2
  • 16
Suleka_28
  • 2,761
  • 4
  • 27
  • 43

8 Answers8

19

Make sure that the number of labels in the final classification layer is equal to the number of classes you have in your dataset. InvalidArgumentError (see above for traceback): logits and labels must be broadcastable: logits_size=[1,2] labels_size=[1,24] as shown in your question might suggest that you are have just two classes in your final classification layer while you actually need 24.

In my case, I had 7 classes in my dataset, but I mistakenly used 4 labels in the final classification layer. Therefore, I had to change from

tf.keras.layers.Dense(4, activation="softmax")

to

tf.keras.layers.Dense(7, activation="softmax")

Anas
  • 751
  • 8
  • 6
7

When you say

    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=prediction, labels=y))

the prediction and labels have incompatible shapes. You need to change how the predictions are computed to get one per example in your minibatch.

Alexandre Passos
  • 5,186
  • 1
  • 14
  • 19
  • Yes, that was also one of the problems in my code and also I had made several more mistakes. I have been trying to use softmax_cross_entropy_with_logits when it is a binary classification where I should have used a sigmoid_cross_entropy_with_logits with n_classes variable set to 1. – Suleka_28 Aug 20 '18 at 08:06
  • 2
    Hi, could you please let me know how did you manage to compute predictions per example for a given minibatch. I am facing the same error as you. logits and labels must be broadcastable: logits_size=[32,31] labels_size=[32,199]. – prb_cm Aug 19 '20 at 01:24
4

This error occurs because there is a mismatch in the count of Predicted class and the input. I copied a code and encountered this error

This was the original code where output is 5 classes

model.add(Dense(5, activation = "softmax"))

In my case, I had 30 classes and correcting the classes count fixed it

model.add(Dense(30, activation = "softmax"))
Ashish Tripathi
  • 301
  • 3
  • 3
2

A little late to the party but I had the same error with a CNN, I messed around with different types of cross entropy and the error was resolved by using sparce_softmax_cross_entropy_with_logits().

cost = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=prediction, labels=y))
Nhowboh
  • 21
  • 2
2

This happened to me when using Tensorflow DataGenerator with small batch size. Try increasing your batch size.

Akihero3
  • 365
  • 5
  • 12
  • I was using batch size of 32 in my generator function and was getting this error. Changing it to 128 worked. Also I have defined a custom loss function for my use case. But still couldn't figure out the reason for this error. Anyways Thanks. – Gurpreet Mar 01 '23 at 12:58
1

I met the similar problems when using the CNN, the problem occurs because I changed the labels data type as np.uint8 in the Generator function while did nothing for the labels type in the rest of my code. I solved the problem by changing the labels type to uint8 in all of my code.

Mingming Qiu
  • 333
  • 4
  • 9
0

Check the input shape in the first convolutional layer and the input shape fed into the network using data.shape. There is high chance you have a different shape fed into your CNN.

Mousa Moradi
  • 1
  • 1
  • 1
    As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Mar 22 '22 at 16:45
0

in my case, my training dataset labels and my validation dataset labels count are not equal. when I try using the same count of labels it works.

Training data details should be like this

validation data details should be like this

pasindu sandeepa
  • 21
  • 3
  • 1
    Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – xlmaster Feb 24 '23 at 12:21