Highway network

In machine learning, the Highway Network was the first working very deep feedforward neural network with hundreds of layers, much deeper than previous artificial neural networks. It uses skip connections modulated by learned gating mechanisms to regulate information flow, inspired by Long Short-Term Memory (LSTM) recurrent neural networks. The advantage of a Highway Network over the common deep neural networks is that it solves or partially prevents the vanishing gradient problem, thus leading to easier to optimize neural networks. The gating mechanisms facilitate information flow across many layers ("information highways").

This article is about a technique used in machine learning. For other uses, see Highway.

Highway Networks have been used as part of text sequence labeling and speech recognition tasks. An open-gated or gateless Highway Network variant called Residual neural network was used to win the ImageNet 2015 competition. This has become the most cited neural network of the 21st century.

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.