---

Perceptron Networks

---

The perceptron was first introduced by Mr. Frank Rosenblatt in 1957.

A Perceptron is an algorithm for supervised learning of binary classifiers.

There are two types of Perceptrons: Single layer and Multilayer.

• Single layer - Single layer perceptrons can learn only linearly separable patterns
• Multilayer - Multilayer perceptrons or feedforward neural networks with two or more layers have the greater processing power

The Perceptron algorithm learns the weights for the input signals in order to draw a linear decision boundary.

This enables you to distinguish between the two linearly separable classes +1 and -1.

Perceptron algorithm (Single layer)

Step 0: initialize the weights and the bias(for easy calculation they can be set to zero). also initialize the learning rate α(0, α, 1) for simpicity α is set to 1.

Step 1: Perform step 2 to 6 until the final stopping condition is false.

Step 2: Perform step 3 to 5 each training pari indicated by s:t

Step 3: The input layer containing input units is applied with identity activation function: x_i = s_i

Step 4: calculate the ourput of the network. to do so first obtain the net input:

$$y_{in}=\underset{i}{\overset{n}{\sum <!--\; \sum \; -->}}{x}_i.w_i+\mathrm{b}$$

where n is the number of inputs neurons in the input layer. Then apply activation over the net input calculated to obrain the output

$$f(y_{in})=\{\begin{array}{ll}1& if{y}_{in}>\mathrm{\theta <!--\; \theta \; -->}\\ 0& if-<!--\; -\; -->\mathrm{\theta <!--\; \theta \; -->}⩽<!--\; ⩽\; -->y_{in}⩽<!--\; ⩽\; -->\mathrm{\theta <!--\; \theta \; -->}\\ -<!--\; -\; -->1& if{y}_{in}<-<!--\; -\; -->\mathrm{\theta <!--\; \theta \; -->}\end{array}$$

Step 5: weight and bias adjustment: compare the value of the actual (calculated) output and desire(targer) output

if y ≠ t, then
      w_i(new) = w_i(old)+ αtx_i
      b(new) = b(old)+ αt
else we have
      w_i(new) = w_i(old)
      b(new) = b(old)

Step 6: Train the network until there is no weight change. This is the stopping condition for the network. If this condition is not met, then start again form step 2.

Perceptron algorithm (Multiple layer)

Step 0: Initialize the weights, biases and learning rate suitably.

Step 1: Check for stopping condition; if it is false, perform Steps 2–6.

Step 2: Perform Steps 3–5 for each bipolar or binary training vector pair s : t.

Step 3: Set activation (identity) of each input unit i = 1 to n: x_i = s_i

Step 4: Calculate output response of each output unit j = 1 to m: First, the net input is calculated as $$y_{inj}=\sum _i^n{x}_i.w_{ij}+b_j$$
Then activations are applied over the net input to calculate the output response: $$f(y_{in})=\{\begin{array}{ll}1& if{y}_{in}>\theta \\ 0& if-\theta ⩽y_{in}⩽\theta \\ -1& if{y}_{in}<-\theta \end{array}$$

Step 5: Make adjustment in weights and bias for j = l to m and i = l to n.

if y_j ≠ t_j, then
      w_ij(new) = w_ij(old)+ αt_jx_i
      b_j(new) = b_j(old)+ αt_j
else we have
      w_ij(new) = w_ij(old)
      b_j(new) = b_j(old)

Step 6: Test for the stopping condition, i.e., if there is no change in weights then stop the training process, else start again from Step 2.

Example of Single layer Perceptron

We need to understand that the output of an AND gate is 1 only if both inputs (in this case, x1 and x2) are 1.

Truth table for AND function with bipolar inputs and targets.

x1	x2	Target
1	1	1
1	-1	-1
-1	1	-1
-1	-1	-1

Row-1

initializing w₁, w₂, and b as 0, α=1, and θ=0.

we get;

x₁(0)+x₂(0)+0

Passing the first row of the AND logic table (x1=1, x2=1), we get;

1*0+1*0+0 = 0

y_in = 0

Y = f(y_in) => f(0) => 0

check Y is equal to t or not, that is 0 ≠ 1, Hence weights change is required.

w_i(new) = w_i(old)+ αtx_i
b(new) = b(old)+ αt

w₁(new) = w₁(old)+ αtx₁

              = 0+ 1*1*1 = 1

w₂(new) = w₂(old)+ αtx₂

              = 0+ 1*1*1 = 1

b(new) = 0+ 1*1 = 1

new updated weightes are w1=w2=b=1

y_in = 1*1 + 1*1 + 1 = 3

Y = f(y_in) => f(3) => 1

check Y is equal to t or not, that is 1 = 1, Hence weights change is Not required.

Row-2

w₁, w₂, and b as 1, α=1, and θ=0.

we get;

x₁(1)+x₂(1)+1

Passing the second row of the AND logic table (x1=1, x2=-1), we get;

1*1+-1*1+1 = 1

y_in = 1

Y = f(y_in) => f(1) => 1

check Y is equal to t or not, that is 1 ≠ -1, Hence weights change is required.

w_i(new) = w_i(old)+ αtx_i
b(new) = b(old)+ αt

w₁(new) = w₁(old)+ αtx₁

              = 1+ 1*-1*1 = 0

w₂(new) = w₂(old)+ αtx₂

              = 1+ 1*-1*-1 = 2

b(new) = 1+ 1*-1 = 0

new updated weightes are w1=0, w2= 2, b=0

y_in = 1*0 + -1*2 + 0 = -2

Y = f(y_in) => f(-2) => -1

check Y is equal to t or not, that is -1 = -1, Hence weights change is Not required.

Row-3

w₁=0, w₂= 2, b=0, α=1, and θ=0.

we get;

x₁(0)+x₂(2)+0

Passing the third row of the AND logic table (x1=-1, x2=1), we get;

-1*0+1*2+0 = 2

y_in = 2

Y = f(y_in) => f(2) => 1

check Y is equal to t or not, that is 1 ≠ -1, Hence weights change is required.

w_i(new) = w_i(old)+ αtx_i
b(new) = b(old)+ αt

w₁(new) = w₁(old)+ αtx₁

              = 0+ 1*-1*-1 = 1

w₂(new) = w₂(old)+ αtx₂

              = 2+ 1*-1*1 = 1

b(new) = 0+ 1*-1 = -1

new updated weightes are w1=1, w2= 1, b=-1

y_in = -1*1 + 1*1 + -1 = -1

Y = f(y_in) => f(-1) => -1

check Y is equal to t or not, that is -1 = -1, Hence weights change is Not required.

Row-4

w₁=1, w₂= 1, b=-1, α=1, and θ=0.

we get;

x₁(1)+x₂(1)+-1

Passing the fourth row of the AND logic table (x1=-1, x2=-1), we get;

-1*1+-1*1+(-1) = -3

y_in = -3

Y = f(y_in) => f(-3) => -1

check Y is equal to t or not, that is -1 = -1, Hence weights change is not required.

---

Next Topic :Adaptive Linear Neuron