0

I'm trying to use ctc_loss in order to predict some labels, but it's the first time I use this function and I got an error.

def bi_rnn(x_bi):
    # Prepare data shape to match `rnn` function requirements
    # Current data input shape: (batch_size, timesteps, n_input)
    # Required shape: 'timesteps' tensors list of shape (batch_size, num_input)
    # Unstack to get a list of 'timesteps' tensors of shape (batch_size, num_input)
    x_bi_shape = x_bi.get_shape().as_list()
    x_bi = tf.unstack(x_bi, x_bi_shape[2], 2)
    # Define lstm cells with tensorflow
    # Forward direction cell

    lstm_fw_cell = rnn.BasicLSTMCell(num_neurons, forget_bias=1.0)
    # Backward direction cell
    lstm_bw_cell = rnn.BasicLSTMCell(num_neurons, forget_bias=1.0)


    bi_outputs, _, _ = rnn.static_bidirectional_rnn(lstm_fw_cell, lstm_bw_cell, x_bi, dtype=tf.float32)

    w_bi = init_weights([num_neurons * 2, num_classes])
    b_bi = init_bias([num_classes])

    bi_outputs = tf.squeeze(input=bi_outputs, axis=1)

    return tf.matmul(bi_outputs, w_bi) + b_bi


def sparse_tuple_from(sequences, dtype=np.int32):
    """Create a sparse representention of x.
    Args:
        sequences: a list of lists of type dtype where each element is a sequence
    Returns:
        A tuple with (indices, values, shape)
    """
    indices = []
    values = []

    for n, seq in enumerate(sequences):
        indices.extend(zip([n] * len(seq), range(len(seq))))
        values.extend(seq)

    indices = np.asarray(indices, dtype=np.int64)
    values = np.asarray(values, dtype=dtype)
    shape = np.asarray([len(sequences), np.asarray(indices).max(0)[1] + 1], dtype=np.int64)

    return indices, values, shape


bi_output_1 = bi_rnn(vectors)
targets = tf.sparse_placeholder(tf.int32, name="targets_sparse_tensor")
bi_output_1_shape = bi_output_1.get_shape().as_list()
global_step = tf.Variable(0, trainable=False)
bi_output_1 = tf.reshape(bi_output_1, [bi_output_1_shape[0], batch_size, bi_output_1_shape[1]])
# 1d array of size [batch_size]
seq_len = tf.placeholder(tf.int32, [None])
loss = tf.nn.ctc_loss(labels=targets, inputs=bi_output_1, sequence_length=seq_len)

cost = tf.reduce_mean(loss)
optimizer = tf.train.MomentumOptimizer(learning_rate=0.001,  momentum=0.9).minimize(cost, global_step=global_step)

here is how I train the net:

with tf.Session() as sess:
    sess.run(init)
    for i in range(1, epochs):
        loss = 0
        start = 0
        for cont in range(int(n_samples / batch_size)):
            batch_x, batch_y = feature_set.next_batch_training_set(batch_size, start)
            batch_y = sparse_tuple_from(batch_y)
            seq_len_training = np.ones(1) * 7
            _, c = sess.run([optimizer, cost], feed_dict={x: batch_x, targets: batch_y, seq_len: seq_len_training})
            loss += c
        print('Epoch:', i, ' loss:', loss)

with:

batch_size = 1
bi_output_1 = Tensor("Reshape:0", shape=(25, 1, 37), dtype=float32)
batch_x has shape : (1, 20, 100, 1)
batch_y has shape : (1, 7) (before the function sparse_tuple_from())

I got this error:

2017-11-23 18:50:14.033220: W C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\util\ctc\ctc_loss_calculator.cc:144] No valid path found.

and I don't know how to solve this. I tried to follow this example, but I got no help.

Yes92
  • 301
  • 4
  • 15
  • 1
    The answer to https://stackoverflow.com/questions/45130184/ctc-loss-error-no-valid-path-found might help you. Also, the code that generates your warning has the comment `// It is possible that no valid path is found if the activations for the targets are zero.` Incidentally, the link you posted is a bit messed up. – Stephen Nov 24 '17 at 21:12
  • Thank you, I'll take a look to that question. I also corrected the link. – Yes92 Nov 25 '17 at 09:35
  • I can't solve my problem... – Yes92 Jan 23 '18 at 17:01
  • could you provide the data sample (text, RNN output length) for which this warning occurs? It might be that the text can't be encoded in the RNN sequence, e.g. if the RNN length is 2 and the ground-truth text is "aa". There is no way to encode "aa" in a length 2 sequence. – Harry Feb 28 '18 at 10:34
  • Possible duplicate of [ctc\_loss error "No valid path found."](https://stackoverflow.com/questions/45130184/ctc-loss-error-no-valid-path-found) – Anaphory Dec 06 '18 at 20:53

0 Answers0