Introduction
This project is about number recognition with multi layer perceptron and there are some new ways to extract features from pictures in this project.
This project get some datas (that are numbers here) and learn with neural network mlp (multi layer perceptron) and after it learned, we test other numbers and our network says what the number is.
Background
- neural network
- artificial intelligence
- matlab
Using the Code
In this project, we have 10 different font of numbers for leaning and we extract 26 feature vector form pictures of numbers.
Feature vectors include:
- Density of colors in 4 sides of pictures
- The ratio of black to white in 4 sides of pictures
- Determine the horizontal and vertical lines
- Determine hole in photo
- Determine number of up, down, left and right ways in writing a number with chain code
- Convert the number to 9segment number
- Determine tangent of total angle of number of black pixels in each row and column
These are our feature vectors that we want to learn it to our neural network.
This project includes the Main script and "pref" "Extract Number" "Extract Features" and "chain code" functions that do a special job that we will explain about. We also use the data from "datas" and target from "target" that we extracted from the "input data" function.
Input datas()
This function gets number 1 in an argument to run. So this function gives the address of pictures and sends them to "Extract Features " and saves the features. Finally, collect all features in one matrix and set it to output.
Datas
This data includes all feature vectors that we save from "input data", that we didn’t necessary extract again in each running and it's good for saving the time.
We have two different types of datas in it:
num: that includes feature vectors of 10 different fonts of numbers (0 to 9) and totally numbers are 100Num_half: that includes half of "num" feature vectors
Target
Our target includes four different types in "binary" type and "one by one" type:
Target100: 10 different fonts divided by "one by one"target100: 10 different fonts divided by binary type with 4 bitTarget50: 5 different fonts divided by "one by one"target50: 5 different fonts divided by binary type with 4 bit
Tip: Attention to capital and small alphabet in targets name.
If we want to learn 10 fonts to our neural network, we have to use "Target100" or "target100" for our target argument and if we want to learn 5 (half) fonts to our neural network, we have to use "Target50" or "target50" for our target argument.
Chain Code
This is the function that gets binary pictures and simulates the handwriting in two types "8 ways" or "4 ways". In other words, if we use 4 ways, chain code exports the numbers between 1 to 4, that means:
- 1 is move up
- 2 is move right
- 3 is move down
- 4 is move left
With this function, we simulate handwriting and we count the number of up, down, left and right ways that we went in drawing the number on paper .
Extract Features
This function gets the address in input argument and exports all features vectors of pictures that we talk about in output argument. In the following, we explain each code.
First of all, in line 2 to 9, we convert the pictures to grayscale and binary, then we reshape and compress it.
I = imread(address);
%%chenge to gray
Igray = rgb2gray(I);
%%chenge to bw
Ibw = im2bw(Igray,graythresh(Igray));
T=reshape(Ibw,10000,1);
Ibw_comp=resizem(Ibw,0.25);
Then, in line 13 to 56, we divide picture to 4 parts, then we find the Density of colors and ratio of black to white in each part. (8 the first feature was created)
k=0;
j=0;
for i=1:2500
if(T(i,1)== 1)
k=k+1;%white
else
j=j+1;%black
end
end
ch1=j/2500;
w_b1=j/k;
j=0;
k=0;
for i=2500:5000
if(T(i,1)== 1)
k=k+1;%white
else
j=j+1;%black
end
end
ch2=j/2500;
w_b2=j/k;
j=0;
k=0;
for i=5000:7500
if(T(i,1)== 1)
k=k+1;%white
else
j=j+1;%black
end
end
ch3=j/2500;
w_b3=j/k;
j=0;
k=0;
for i=7500:10000
if(T(i,1)== 1)
k=k+1;%white
else
j=j+1;%black
end
end
ch4=j/2500;
w_b4=j/k;
In line 59 to 92, we count the number of black pixels in each column and row, then we find the arc-tangent of the angle. After that, we add all angles and finally, we tangent the angle for finding the line slope. (2 other feature vectors were created).
k=0;
j=0;
for i=1:25
for j=1:25
if (Ibw_comp(i,j)==0)
k=k+1;
end
end
teta(i,1)=k;
k=0;
end
k=0;
for i=1:25
for j=1:25
if (Ibw_comp(j,i)==0)
k=k+1;
end
end
teta(i,2)=k;
k=0;
end
for i=1:25
teta(i,3)=(teta(i,2)/teta(i,1));
teta(i,4)=atan(teta(i,3))
end
for i=1:25
for j=1:4
if (isnan(teta(i,j))==1)
teta(i,j)=0;
end
end
end
teta(26,4)=sum(teta(:,4));
teta(27,4)=tan(teta(26,4))
From line 96 to 114, we count the black pixels in each column and if the number was more than our threshold, we understand there was a vertical line in this picture and we say it happened by setting number "1" as feature vector.
Tip: '0' means there is no vertical line and '1' means there is a vertical line in the picture.
n=0;
j=1;
for i=1:100
for j=1:100
if(Ibw(j,i)==0)
n=n+1;
t(i)=n;
end
end
n=0;
end
n=max(t);
if(n>79)
ver=1;
else
ver=0;
end
From line 118 to 137 again, we count the black pixels in each row and if the number was more than our threshold , we understand there was a horizontal line in this picture and we say it happened by setting number "1" as feature vector.
Tip: It is the same as the previous tip '0' which means there is no horizontal line and '1' means there is a horizontal line in picture.
For example, number 1 has a vertical line and number 4 has a horizontal line.
(2 other feature vectors were created.)
t=0;
n=0;
j=1;
for i=1:100
for j=1:100
if(Ibw(i,j)==0)
n=n+1;
t(i)=n;
end
end
n=0;
end
n=max(t);
if(n>40)
har=1;
else
har=0;
end
From line 140 to 167, we simulate the handwriting by chain code and we export the number of total moving in each way as feature vector. (Two other feature vectors were created).
B = bwboundaries(Ibw,4); % find the boundaries of all objects
CC = cell(1, length(B)); % pre-allocate
for k = 1:length(B)
CC{k} = chaincode(B{k},1); % chain code for the k'th object
end
up=0;
down=0;
left=0;
right=0;
for i=1:length(B)
t=max(size(CC{1,i}.code));
for j=1:t
if(CC{1,i}.code(j,1)==0)
right=right+1;
else if(CC{1,i}.code(j,1)==2)
up=up+1;
else if(CC{1,i}.code(j,1)==4)
left=left+1;
else if(CC{1,i}.code(j,1)==6)
down=down+1;
end
end
end
end
end
end
From line 171 to 292, we change the number to our 9 segment number.
We divide the picture to 9 parts, then in each part if the Density of black pixels are more than the threshold (that here is 150), the segment will be black and set number 1.
Do this in each 9 segments, then we change the number to look like 7 segments numbers and here 9 other feature vectors are created.
alfa=150;
k=0;
for i=1:33
for j=1:33
if(Ibw(i,j)==0)
k=k+1;
end
end
end
if(k>alfa)
seg1=1;
else
seg1=0;
end
k=0;
%%------------------------
for i=1:33
for j=33:66
if(Ibw(i,j)==0)
k=k+1;
end
end
end
if(k>alfa)
seg2=1;
else
seg2=0;
end
k=0;
for i=1:33
for j=66:99
if(Ibw(i,j)==0)
k=k+1;
end
end
end
if(k>alfa)
seg3=1;
else
seg3=0;
end
k=0;
for i=33:66
for j=1:33
if(Ibw(i,j)==0)
k=k+1;
end
end
end
if(k>alfa)
seg4=1;
else
seg4=0;
end
k=0;
for i=33:66
for j=33:66
if(Ibw(i,j)==0)
k=k+1;
end
end
end
if(k>alfa)
seg5=1;
else
seg5=0;
end
k=0;
for i=33:66
for j=66:99
if(Ibw(i,j)==0)
k=k+1;
end
end
end
if(k>alfa)
seg6=1;
else
seg6=0;
end
k=0;
for i=66:99
for j=1:33
if(Ibw(i,j)==0)
k=k+1;
end
end
end
if(k>alfa)
seg7=1;
else
seg7=0;
end
k=0;
for i=66:99
for j=33:66
if(Ibw(i,j)==0)
k=k+1;
end
end
end
if(k>alfa)
seg8=1;
else
seg8=0;
end
k=0;
for i=66:99
for j=66:99
if(Ibw(i,j)==0)
k=k+1;
end
end
end
if(k>alfa)
seg9=1;
else
seg9=0;
end
seg10=[seg1 seg2 seg3
seg4 seg5 seg6
seg7 seg8 seg9];
In line 295, we find that our picture has a hole or not and say it by setting number 1 and 0 as feature vector. (1 other feature vector was created).
Tip: 1 means there is a hole in the picture and 0 means there is no hole in the picture. (For example, number 8 has a hole but number 1 does not.)
hole=bweuler(Ibw);
Finally in line 298, we collect all feature vectors in one matrix and export it to the output argument.
chall=[ch1 ch2 ch3 ch4 w_b1 w_b2 w_b3 w_b4 ver har hole up down left right
seg1 seg2 seg3 seg4 seg5 seg6 seg7 seg8 seg9 teta(26,4) teta(27,4)];
Main:
This is the main part of our code and we run our project here.
In line 6 and 8, we get the inputs and targets.
% load data
load('datas.mat');
% load targets
load ('targets.mat');
input=num;
target=Target100;
As I mentioned, we have 4 types of target and we choose it by "style" parameter.
Tip: If style=0, our target is "one by one" type and if style=1, our target will have to be "binary" type.
type=1;
In the following, we create our neural network with name "newff" and train our datas and targets with this network.
newff=feedforwardnet([10 10],'trainlm');
newff=train(newff,input,target);
In line 29, we test the number 1. If our network says the number is 1, it works well.
r= newff(num1); %<==Number for test
So we test it, but network exports the number between 0 and 1 and we should normalize it by threshold 0.5 till we find our answer.
Lines 31 till 39 show how we normalize the exported number if our type was "one by one" type (the maximum similarity is answer).
if(type==1)
max=max (r(:,1));
for i=1:10
if(max==(r(i,1)))
disp('your number is:')
disp(i)
end
end
end
%%check the answer:
% i=10 =>number is 0 % i=5 =>number is 5
% i=1 =>number is 1 % i=6 =>number is 6
% i=2 =>number is 2 % i=7 =>number is 7
% i=3 =>number is 3 % i=8 =>number is 8
% i=4 =>number is 4 % i=9 =>number is 9
Lines 45 till 56 shows how we normalize the exported number if our type was "binary" type (if number is more than 0.5(threshold), change it to 1, else change it to number 0). After that, we send the number to "findNumber" function and it says what is the number.
if(type==0)
for i=1:4
if(r(i,1)>0.5)
r(i,1)=1;
else
r(i,1)=0;
end
end
FindNumber(r)
end
In lines from 60 to 111, we show the performance for one font (10 numbers from 0 to 9).
disp('performance by test num from 0 to 9 (10 number)')
per=0;
r= newff(num0);
r=ExtractNumber(1,r);
if(r==10)
per=per+1;
end
r= newff(num1);
r=ExtractNumber(1,r);
if(r==1)
per=per+1;
end
r= newff(num2);
r=ExtractNumber(1,r);
if(r==2)
per=per+1;
end
r= newff(num3);
r=ExtractNumber(1,r);
if(r==3)
per=per+1;
end
r= newff(num4);
r=ExtractNumber(1,r);
if(r==4)
per=per+1;
end
r= newff(num5);
r=ExtractNumber(1,r);
if(r==5)
per=per+1;
end
r= newff(num6);
r=ExtractNumber(1,r);
if(r==6)
per=per+1;
end
r= newff(num7);
r=ExtractNumber(1,r);
if(r==7)
per=per+1;
end
r= newff(num8);
r=ExtractNumber(1,r);
if(r==8)
per=per+1;
end
r= newff(num9);
r=ExtractNumber(1,r);
if(r==9)
per=per+1;
end
For finding the performance of our code, we should test all of our datas so we create a function "ExtractNumber" for using it for all datas and we create function "pref" to find the performance
Pref tests all input datas and counts the true and wrong answers and tells the performance of our code.
You can test your own number in this project. Just write the address of your picture of the number in line 29 in the main script, then you can see the answer of the network
In our test, we get the performance more than 70% and if we increase fonts for learning to 200 fonts or more, we will get better performance.
There is a document in the attached source code.
History
- 23rd May, 2017: Initial version
