DEPARTMENT OF ARTIFICIAL INTELLIGENCE AND
MACHINE LEARNING
AM3403 Machine Learning: Concepts and
Application
LECTURE NOTES
UNIT-3
, UNIT-3
Syllabus
NEURAL NETWORKS
Artificial Neural Networks: Introduction, Biological motivation, ANN representation,
appropriate problem for ANN learning, Perceptron, multilayer networks and the back
propagation algorithm, Popular CNN Architectures, RNNs, LSTM, BERT, GANS and
Generative Models
3. Artificial Neural Networks
3.1 Introduction
Artificial Neural Networks (ANNs) are a subset of machine learning inspired by the structure and
functionality of the human brain. They consist of layers of interconnected nodes (neurons) that
process and transform data to recognize patterns, make predictions, or perform classification tasks.
NNs learn relationship between cause and effect or organize large volumes of data into orderly and
informative patterns
Definition of ANN
Tsoukalas & Uhrig, 1997 defined ANN as
Data processing system consisting of a large number of simple, highly interconnected
processing elements (artificial neuron) in an architecture inspired by structure of the
cerebral cortex of the brain
3.2 Biological motivation
Artificial Neural Networks (ANNs) are inspired by the structure and functioning of the human
brain. The goal of ANNs is to mimic the way biological neurons process and transmit information,
enabling machines to learn from data and make intelligent decisions
How do our brains works?
The Brain is A massively parallel information processing system.Our brains are a huge network of
processing elements. A typical brain contains a network of 10 billion neurons.
,A biological neuron has three t y p e s o f m a i n components; dendrites, soma (or
cell body) and axon.Dendrites receives signals from other nerves. The soma, sums the
incoming signals. When sufficient input is received, the cell fires; that is it transmit a signal over
its axon to other cell
Researchers Warren McCullock and Walter Pitts published their first concept of simplified brain
cell in 1943.This was called McCullock-Pitts (MCP) neuron. They described such a nerve cell as a
simple logic gate with binary outputs
Biological vs. Artificial Neuron
3.3 ANN representation
An ANN is a network of nodes (units) called Artificial Neurons connected by 'edges'. Edge
typically has a weight that can be adjusted. The weight increases or decreases the strength of the
connection. The output of each Artificial Neuron is typically computed by some non-linear
function of the sum of its inputs if their sums exceed some threshold. Neurons can potentially
fire in parallell but may also be temporally constrained. It resembles the brain in two respects:
Knowledge is acquired by the network from its environment through a learning process
Synaptic connection strengths among neurons are used to store the acquired knowledge.
, Artificial Neurons are aggregated into layers. Different layers may perform different kinds of
transformations on the inputs. There are several different architectures for ANNs, each with
their own strengths and weaknesses. Some of the most common architectures include:
Feed forward Neural Networks: This is the simplest type of ANN architecture, where the
information flows in one direction from input to output. The layers are fully connected,
meaning each neuron in a layer is connected to all the neurons in the next layer.
Recurrent Neural Networks (RNNs): These networks have a ―memory‖ component, where
information can flow in cycles through the network. This allows the network to process
sequences of data, such as time series or speech.
Convolutional Neural Networks (CNNs): These networks are designed to process data with a
grid-like topology, such as images. The layers consist of convolutional layers, which learn to
detect specific features in the data, and pooling layers, which reduce the spatial dimensions of
the data.
Auto encoders: These are neural networks that are used for unsupervised learning. They
consist of an encoder that maps the input data to a lower-dimensional representation and a
decoder that maps the representation back to the original data.
Generative Adversarial Networks (GANs): These are neural networks that are used for
generative modeling. They consist of two parts: a generator that learns to generate new data
samples, and a discriminator that learns to distinguish between real and generated data.
Basic types of neuron connection architecture :
1. Single-layer feed-forward network
In this type of network, we have only two layers input layer and the output layer but the input
layer does not count because no computation is performed in this layer. The output layer is
formed when different weights are applied to input nodes and the cumulative effect per node is
taken. After this, the neurons collectively give the output layer to compute the output signals.
MACHINE LEARNING
AM3403 Machine Learning: Concepts and
Application
LECTURE NOTES
UNIT-3
, UNIT-3
Syllabus
NEURAL NETWORKS
Artificial Neural Networks: Introduction, Biological motivation, ANN representation,
appropriate problem for ANN learning, Perceptron, multilayer networks and the back
propagation algorithm, Popular CNN Architectures, RNNs, LSTM, BERT, GANS and
Generative Models
3. Artificial Neural Networks
3.1 Introduction
Artificial Neural Networks (ANNs) are a subset of machine learning inspired by the structure and
functionality of the human brain. They consist of layers of interconnected nodes (neurons) that
process and transform data to recognize patterns, make predictions, or perform classification tasks.
NNs learn relationship between cause and effect or organize large volumes of data into orderly and
informative patterns
Definition of ANN
Tsoukalas & Uhrig, 1997 defined ANN as
Data processing system consisting of a large number of simple, highly interconnected
processing elements (artificial neuron) in an architecture inspired by structure of the
cerebral cortex of the brain
3.2 Biological motivation
Artificial Neural Networks (ANNs) are inspired by the structure and functioning of the human
brain. The goal of ANNs is to mimic the way biological neurons process and transmit information,
enabling machines to learn from data and make intelligent decisions
How do our brains works?
The Brain is A massively parallel information processing system.Our brains are a huge network of
processing elements. A typical brain contains a network of 10 billion neurons.
,A biological neuron has three t y p e s o f m a i n components; dendrites, soma (or
cell body) and axon.Dendrites receives signals from other nerves. The soma, sums the
incoming signals. When sufficient input is received, the cell fires; that is it transmit a signal over
its axon to other cell
Researchers Warren McCullock and Walter Pitts published their first concept of simplified brain
cell in 1943.This was called McCullock-Pitts (MCP) neuron. They described such a nerve cell as a
simple logic gate with binary outputs
Biological vs. Artificial Neuron
3.3 ANN representation
An ANN is a network of nodes (units) called Artificial Neurons connected by 'edges'. Edge
typically has a weight that can be adjusted. The weight increases or decreases the strength of the
connection. The output of each Artificial Neuron is typically computed by some non-linear
function of the sum of its inputs if their sums exceed some threshold. Neurons can potentially
fire in parallell but may also be temporally constrained. It resembles the brain in two respects:
Knowledge is acquired by the network from its environment through a learning process
Synaptic connection strengths among neurons are used to store the acquired knowledge.
, Artificial Neurons are aggregated into layers. Different layers may perform different kinds of
transformations on the inputs. There are several different architectures for ANNs, each with
their own strengths and weaknesses. Some of the most common architectures include:
Feed forward Neural Networks: This is the simplest type of ANN architecture, where the
information flows in one direction from input to output. The layers are fully connected,
meaning each neuron in a layer is connected to all the neurons in the next layer.
Recurrent Neural Networks (RNNs): These networks have a ―memory‖ component, where
information can flow in cycles through the network. This allows the network to process
sequences of data, such as time series or speech.
Convolutional Neural Networks (CNNs): These networks are designed to process data with a
grid-like topology, such as images. The layers consist of convolutional layers, which learn to
detect specific features in the data, and pooling layers, which reduce the spatial dimensions of
the data.
Auto encoders: These are neural networks that are used for unsupervised learning. They
consist of an encoder that maps the input data to a lower-dimensional representation and a
decoder that maps the representation back to the original data.
Generative Adversarial Networks (GANs): These are neural networks that are used for
generative modeling. They consist of two parts: a generator that learns to generate new data
samples, and a discriminator that learns to distinguish between real and generated data.
Basic types of neuron connection architecture :
1. Single-layer feed-forward network
In this type of network, we have only two layers input layer and the output layer but the input
layer does not count because no computation is performed in this layer. The output layer is
formed when different weights are applied to input nodes and the cumulative effect per node is
taken. After this, the neurons collectively give the output layer to compute the output signals.
