Understanding Convolutional Neural
Networks for NLP
Convolutional Neural Network - answer In machine learning, a convolutional neural
network (CNN, or Convent) is a class of deep, feed-forward artificial neural networks
that has successfully been applied to analyzing visual imagery.
CNNs use a variation of multilayer perceptron’s designed to require minimal
preprocessing. They are also known as shift invariant or space invariant artificial neural
networks (SIANN), based on their shared-weights architecture and translation
invariance characteristics.
Computer Vision - answer Computer vision is an interdisciplinary field that deals with
how computers can be made for gaining high-level understanding from digital images or
videos. From the perspective of engineering, it seeks to automate tasks that the human
visual system can do.
Natural Language Processing - answerA field of computer science, artificial intelligence
concerned with the interactions between computers and human (natural) languages,
and, in particular, concerned with programming computers to fruitfully process large
natural language data.
Challenges in natural language processing frequently involve speech recognition,
natural language understanding, and natural language generation.
Convolution - answerThe for me easiest way to understand a convolution is by thinking
of it as a sliding window function applied to a matrix.
The sliding window is called a kernel, filter, or feature detector. Say we use a 3×3 filter,
multiply its values element-wise with the original matrix, then sum them up.
To get the full convolution we do this for each element by sliding the filter over the whole
matrix.
This emulates the response of an individual neuron to visual stimuli.
Kernel - answerSliding window used in a convolution.
Filter - answerSliding window used in a convolution.
Feature Detector - answerSliding window used in a convolution.
, Hadamard Product - answerMultiply Values Element-Wise: A binary operation that
takes two matrices of the same dimensions, and produces another matrix where each
element ij is the product of elements ij of the original two matrices. It should not be
confused with the more common matrix product.
This product is associative and distributive, and unlike the matrix product it is also
commutative.
Non-Linear Activation Functions - answerIn computational networks, this function of a
node defines the output of that node given an input or set of inputs.
However, only nonlinear activation functions allow such networks to compute nontrivial
problems using only a small number of nodes. In artificial neural networks this function
is also called the transfer function.
Ex: Sigmoid(bounded), Tanh(similar to Sigmoid), ReLU(0,inf)
Feedforward Neural Network - answerAn artificial neural network wherein connections
between the units do not form a cycle.
The neural network was the first and simplest type of artificial neural network devised. In
this network, the information moves in only one direction, forward, from the input nodes,
through the hidden nodes (if any) and to the output nodes. There are no cycles or loops
in the network.
Fully Connected Layer - answerLayers connect every neuron in one layer to every
neuron in another layer. It is in principle the same as the traditional multi-layer
perceptron neural network.
Affine Layer - answerFully Connected Layer
Pooling Layers - answerTypically applied after the convolutional layers. These layers
subsample their input.
Ex: Max, Average
Subsampling Layers - answerPooling Layers
Invariance - answerThe property of remaining unchanged regardless of changes in the
conditions of measurement. For example, the area of a surface remains unchanged if
the surface is rotated in space; thus the area exhibits this property.
Principle of Compositionality - answerIn mathematics, semantics, and philosophy of
language, this principle is the principle that the meaning of a complex expression is
determined by the meanings of its constituent expressions and the rules used to
combine them.
Networks for NLP
Convolutional Neural Network - answer In machine learning, a convolutional neural
network (CNN, or Convent) is a class of deep, feed-forward artificial neural networks
that has successfully been applied to analyzing visual imagery.
CNNs use a variation of multilayer perceptron’s designed to require minimal
preprocessing. They are also known as shift invariant or space invariant artificial neural
networks (SIANN), based on their shared-weights architecture and translation
invariance characteristics.
Computer Vision - answer Computer vision is an interdisciplinary field that deals with
how computers can be made for gaining high-level understanding from digital images or
videos. From the perspective of engineering, it seeks to automate tasks that the human
visual system can do.
Natural Language Processing - answerA field of computer science, artificial intelligence
concerned with the interactions between computers and human (natural) languages,
and, in particular, concerned with programming computers to fruitfully process large
natural language data.
Challenges in natural language processing frequently involve speech recognition,
natural language understanding, and natural language generation.
Convolution - answerThe for me easiest way to understand a convolution is by thinking
of it as a sliding window function applied to a matrix.
The sliding window is called a kernel, filter, or feature detector. Say we use a 3×3 filter,
multiply its values element-wise with the original matrix, then sum them up.
To get the full convolution we do this for each element by sliding the filter over the whole
matrix.
This emulates the response of an individual neuron to visual stimuli.
Kernel - answerSliding window used in a convolution.
Filter - answerSliding window used in a convolution.
Feature Detector - answerSliding window used in a convolution.
, Hadamard Product - answerMultiply Values Element-Wise: A binary operation that
takes two matrices of the same dimensions, and produces another matrix where each
element ij is the product of elements ij of the original two matrices. It should not be
confused with the more common matrix product.
This product is associative and distributive, and unlike the matrix product it is also
commutative.
Non-Linear Activation Functions - answerIn computational networks, this function of a
node defines the output of that node given an input or set of inputs.
However, only nonlinear activation functions allow such networks to compute nontrivial
problems using only a small number of nodes. In artificial neural networks this function
is also called the transfer function.
Ex: Sigmoid(bounded), Tanh(similar to Sigmoid), ReLU(0,inf)
Feedforward Neural Network - answerAn artificial neural network wherein connections
between the units do not form a cycle.
The neural network was the first and simplest type of artificial neural network devised. In
this network, the information moves in only one direction, forward, from the input nodes,
through the hidden nodes (if any) and to the output nodes. There are no cycles or loops
in the network.
Fully Connected Layer - answerLayers connect every neuron in one layer to every
neuron in another layer. It is in principle the same as the traditional multi-layer
perceptron neural network.
Affine Layer - answerFully Connected Layer
Pooling Layers - answerTypically applied after the convolutional layers. These layers
subsample their input.
Ex: Max, Average
Subsampling Layers - answerPooling Layers
Invariance - answerThe property of remaining unchanged regardless of changes in the
conditions of measurement. For example, the area of a surface remains unchanged if
the surface is rotated in space; thus the area exhibits this property.
Principle of Compositionality - answerIn mathematics, semantics, and philosophy of
language, this principle is the principle that the meaning of a complex expression is
determined by the meanings of its constituent expressions and the rules used to
combine them.
