UNIT V NEURAL NETWORKS 9
Perceptron - Multilayer perceptron, activation functions, network training – gradient descent
optimization – stochastic gradient descent, error backpropagation, from shallow networks to deep
networks –Unit saturation (aka the vanishing gradient problem) – ReLU, hyperparameter tuning,
batch normalization, regularization, dropout.
Perceptron
What is the Perceptron model in Machine Learning?
Perceptron is Machine Learning algorithm for supervised learning of various binary classification
tasks. Further, Perceptron is also understood as an Artificial Neuron or neural network unit that
helps to detect certain input data computations in business intelligence.
Perceptron model is also treated as one of the best and simplest types of Artificial Neural networks.
However, it is a supervised learning algorithm of binary classifiers. Hence, we can consider it as a
single-layer neural network with four main parameters, i.e., input values, weights and Bias, net
sum, and an activation function.
What is Binary classifier in Machine Learning?
In Machine Learning, binary classifiers are defined as the function that helps in deciding whether
input data can be represented as vectors of numbers and belongs to some specific class.
Binary classifiers can be considered as linear classifiers. We can understand it as a classification
algorithm that can predict linear predictor function in terms of weight and feature vectors.
Basic Components of Perceptron
Three main components. These are as follows:
, o Input Nodes or Input Layer:
This is the primary component of Perceptron which accepts the initial data into the system for further
processing. Each input node contains a real numerical value.
o Weight and Bias:
Weight parameter represents the strength of the connection between units. This is another most
important parameter of Perceptron components.
Weight is directly proportional to the strength of the associated input neuron in deciding the output.
Further, Bias can be considered as the line of intercept in a linear equation.
o Activation Function:
These are the final and important components that help to determine whether the neuron will fire or
not. Activation Function can be considered primarily as a step function.
Types of Activation functions:
o Sign function
o Step function, and
o Sigmoid function
The data scientist uses the activation function to take a subjective decision based on various problem
statements and forms the desired outputs.
Activation function may differ (e.g., Sign, Step, and Sigmoid) in perceptron models by checking
whether the learning process is slow or has vanishing or exploding gradients.
, How does Perceptron work?
Perceptron is considered as a single-layer neural network that consists of four main parameters
named input values (Input nodes), weights and Bias, net sum, and an activation function.
The perceptron model begins with the multiplication of all input values and their weights, then adds
these values together to create the weighted sum.
Then this weighted sum is applied to the activation function 'f' to obtain the desired output. This
activation function is also known as the step function and is represented by 'f'.
This step function or Activation function plays a vital role in ensuring that output is mapped between
required values (0,1) or (-1,1). It is important to note that the weight of input is indicative of the
strength of a node. Similarly, an input's bias value gives the ability to shift the activation function
curve up or down.
Perceptron model works in two important steps as follows:
Step-1
Multiply all input values with corresponding weight values and then add them to determine the
weighted sum. Mathematically, we can calculate the weighted sum as follows:
∑wi*xi = x1*w1 + x2*w2 +…wn*xn
Add a special term called bias 'b' to this weighted sum to improve the model's performance.
Perceptron - Multilayer perceptron, activation functions, network training – gradient descent
optimization – stochastic gradient descent, error backpropagation, from shallow networks to deep
networks –Unit saturation (aka the vanishing gradient problem) – ReLU, hyperparameter tuning,
batch normalization, regularization, dropout.
Perceptron
What is the Perceptron model in Machine Learning?
Perceptron is Machine Learning algorithm for supervised learning of various binary classification
tasks. Further, Perceptron is also understood as an Artificial Neuron or neural network unit that
helps to detect certain input data computations in business intelligence.
Perceptron model is also treated as one of the best and simplest types of Artificial Neural networks.
However, it is a supervised learning algorithm of binary classifiers. Hence, we can consider it as a
single-layer neural network with four main parameters, i.e., input values, weights and Bias, net
sum, and an activation function.
What is Binary classifier in Machine Learning?
In Machine Learning, binary classifiers are defined as the function that helps in deciding whether
input data can be represented as vectors of numbers and belongs to some specific class.
Binary classifiers can be considered as linear classifiers. We can understand it as a classification
algorithm that can predict linear predictor function in terms of weight and feature vectors.
Basic Components of Perceptron
Three main components. These are as follows:
, o Input Nodes or Input Layer:
This is the primary component of Perceptron which accepts the initial data into the system for further
processing. Each input node contains a real numerical value.
o Weight and Bias:
Weight parameter represents the strength of the connection between units. This is another most
important parameter of Perceptron components.
Weight is directly proportional to the strength of the associated input neuron in deciding the output.
Further, Bias can be considered as the line of intercept in a linear equation.
o Activation Function:
These are the final and important components that help to determine whether the neuron will fire or
not. Activation Function can be considered primarily as a step function.
Types of Activation functions:
o Sign function
o Step function, and
o Sigmoid function
The data scientist uses the activation function to take a subjective decision based on various problem
statements and forms the desired outputs.
Activation function may differ (e.g., Sign, Step, and Sigmoid) in perceptron models by checking
whether the learning process is slow or has vanishing or exploding gradients.
, How does Perceptron work?
Perceptron is considered as a single-layer neural network that consists of four main parameters
named input values (Input nodes), weights and Bias, net sum, and an activation function.
The perceptron model begins with the multiplication of all input values and their weights, then adds
these values together to create the weighted sum.
Then this weighted sum is applied to the activation function 'f' to obtain the desired output. This
activation function is also known as the step function and is represented by 'f'.
This step function or Activation function plays a vital role in ensuring that output is mapped between
required values (0,1) or (-1,1). It is important to note that the weight of input is indicative of the
strength of a node. Similarly, an input's bias value gives the ability to shift the activation function
curve up or down.
Perceptron model works in two important steps as follows:
Step-1
Multiply all input values with corresponding weight values and then add them to determine the
weighted sum. Mathematically, we can calculate the weighted sum as follows:
∑wi*xi = x1*w1 + x2*w2 +…wn*xn
Add a special term called bias 'b' to this weighted sum to improve the model's performance.
