Questions tagged [conv-neural-network]

A convolutional neural network (CNN, or ConvNet) is a class of deep, feed-forward artificial neural networks that has successfully been applied to analyzing visual imagery. It falls under the [deep-learning] tag.

Convolutional Neural Networks (CNN) are a type of s used in many problems. This architecture allows sharing the set of weights between different spatial regions in the input. Training a CNN for solving a machine learning problem such as object recognition involves finding the decision boundaries for classification as well as learning the kernels of the filters that generate the features without prior manual tuning.

In a typical CNN the input is convolved with multiple filters to produce feature maps. A pooling layer selects which responses to route further into the network. Non-linear transformations are applied to responses similar to non-linearities found in multi-layer perceptron.

The idea behind CNNs came from recognizing that low-level image statistics appear everywhere in an image and are not necessarily limited to a particular location.

Additional resources:

10589 questions
248
votes
13 answers

How to verify CuDNN installation?

I have searched many places but ALL I get is HOW to install it, not how to verify that it is installed. I can verify my NVIDIA driver is installed, and that CUDA is installed, but I don't know how to verify CuDNN is installed. Help will be much…
alfredox
  • 4,082
  • 6
  • 21
  • 29
238
votes
8 answers

Ordering of batch normalization and dropout?

The original question was in regard to TensorFlow implementations specifically. However, the answers are for implementations in general. This general answer is also the correct answer for TensorFlow. When using batch normalization and dropout in…
golmschenk
  • 11,736
  • 20
  • 78
  • 137
215
votes
12 answers

Why binary_crossentropy and categorical_crossentropy give different performances for the same problem?

I'm trying to train a CNN to categorize text by topic. When I use binary cross-entropy I get ~80% accuracy, with categorical cross-entropy I get ~50% accuracy. I don't understand why this is. It's a multiclass problem, doesn't that mean that I have…
195
votes
4 answers

Intuitive understanding of 1D, 2D, and 3D convolutions in convolutional neural networks

Can anyone please clearly explain the difference between 1D, 2D, and 3D convolutions in convolutional neural networks (in deep learning) with the use of examples?
126
votes
4 answers

Tensorflow Strides Argument

I am trying to understand the strides argument in tf.nn.avg_pool, tf.nn.max_pool, tf.nn.conv2d. The documentation repeatedly says strides: A list of ints that has length >= 4. The stride of the sliding window for each dimension of the input…
117
votes
2 answers

Which parameters should be used for early stopping?

I'm training a neural network for my project using Keras. Keras has provided a function for early stopping. May I know what parameters should be observed to avoid my neural network from overfitting by using early stopping?
AizuddinAzman
  • 1,307
  • 2
  • 9
  • 5
93
votes
5 answers

Calculate the output size in convolution layer

How do I calculate the output size in a convolution layer? For example, I have a 2D convolution layer that takes a 3x128x128 input and has 40 filters of size 5x5.
Monk247uk
  • 1,170
  • 1
  • 8
  • 15
91
votes
7 answers

How to tell Keras stop training based on loss value?

Currently I use the following code: callbacks = [ EarlyStopping(monitor='val_loss', patience=2, verbose=0), ModelCheckpoint(kfold_weights_path, monitor='val_loss', save_best_only=True, verbose=0), ] model.fit(X_train.astype('float32'),…
ZFTurbo
  • 3,652
  • 3
  • 22
  • 27
90
votes
4 answers

Batch Normalization in Convolutional Neural Network

I am newbie in convolutional neural networks and just have idea about feature maps and how convolution is done on images to extract features. I would be glad to know some details on applying batch normalisation in CNN. I read this paper…
83
votes
3 answers

How to calculate the number of parameters for convolutional neural network?

I'm using Lasagne to create a CNN for the MNIST dataset. I'm following closely to this example: Convolutional Neural Networks and Feature Extraction with Python. The CNN architecture I have at the moment, which doesn't include any dropout layers,…
Waddas
  • 1,353
  • 2
  • 16
  • 28
82
votes
4 answers

Instance Normalisation vs Batch normalisation

I understand that Batch Normalisation helps in faster training by turning the activation towards unit Gaussian distribution and thus tackling vanishing gradients problem. Batch norm acts is applied differently at training(use mean/var from each…
78
votes
2 answers

What does the parameter retain_graph mean in the Variable's backward() method?

I'm going through the neural transfer pytorch tutorial and am confused about the use of retain_variable(deprecated, now referred to as retain_graph). The code example show: class ContentLoss(nn.Module): def __init__(self, target, weight): …
68
votes
5 answers

Dimension of shape in conv1D

I have tried to build a CNN with one layer, but I have some problem with it. Indeed, the compilator says me that ValueError: Error when checking model input: expected conv1d_1_input to have 3 dimensions, but got array with shape (569, 30) This is…
protti
  • 871
  • 1
  • 9
  • 12
68
votes
7 answers

Convert Keras model to C++

I am using Keras (with Theano) to train my CNN model. Does anyone has idea how can I use it in my C++ application? Does anyone tried something similar? I have idea to write some python code that will generate a c++ code with network functions - any…
pplonski
  • 5,023
  • 1
  • 30
  • 34
66
votes
4 answers

2-D convolution as a matrix-matrix multiplication

I know that, in the 1D case, the convolution between two vectors, a and b, can be computed as conv(a, b), but also as the product between the T_a and b, where T_a is the corresponding Toeplitz matrix for a. Is it possible to extend this idea to…
1
2 3
99 100