Introduction
This article is a continuation of my current look at neural Networks and to see if there is an easier way to achieve the goals that I have written previously in my articles on Neural Networks. These articles are based on my own dissatisfaction with Neural Networks and the nagging feeling I still have that there must be a better way to do what Neural networks do. These articles are not an attempt to diminish or disparage Neural Networks in any way, they are merely and expression of my curiosity and are meant to be looked at strictly from a programmer's "How do I implement this" point of view rather than from a mathematician's or from the point of view of someone who is trying to replicate the way the brain actually functions.
One thing that I have come across a lot in my research into artificial intelligence is the idea of rules. In order for a Neural Network to work at all, it must decide what rules it is going to use. It is normally helped along this path through the use of something called supervised training which in effect means the network is given the test questions with the answers and has to work out how to derive the answers from the test data. Basically, it has to work out the rules that it is going to use and then memorize them in some way. There are unsupervised ways of training Neural Networks, but seeing as this article is looking at the Back Propagation Network, I can comfortably ignore these for now.
Perhaps, unsurprisingly, when I started to look into Fuzzy Logic, the problems of rules raised its head again to the point where I came up with the ideas I talk about in the Fuzzy Logic Vs Adaline Neural Network article. In that article, I also express my own doubts about whether what I am doing can even be called Fuzzy Logic anymore. In his book Fuzzy Thinking, Bart Kosko says: "Fuzzy Systems Store dozens or hundreds or thousands of these common-sense fuzzy rules. Each new piece of data activates all the fuzzy rules to some degree (most to zero degree). The Fuzzy System then blends together the outputs and produces a final output or answer", which puts me on more solid ground than I previously thought I was. Admittedly, at the moment, the examples are simple and therefore the decisions taken over which rules to use and the rules themselves are fairly simple. But as I proceed along these lines, everything is going to get more complicated.
The BackPropagation Neural Network
The idea behind the code for the Back Propagation Network is that it is an improvement over the Adaline Network in that it addresses some of the issues that the Adaline Neural Network is unable to resolve. For a full description of the Back Propagation Network, see Neural Dot Net Pt 7 The BackPropagation Network. However, a brief description is that the Back Propagation network allows the Neural Network to deal with solutions such as logical OR, for example, if you have two input numbers of the values 0 and 1, then you can get the Back Propagation network to output a 1 if one but not both of the numbers are equal to 1. This is done mainly through the use of a tolerance value that allows you to set the level of correctness for the network. The tolerance level is entirely up to the person running the network, but for example, if you set the tolerance level to 0.4, it would mean that all the values returned by the network within the range of 0 to 0.4 would be classed as equal to 0 and all the values returned in the range of 1 to 0.6 would be considered to be equal to 1. This allows the person running the network to allow for a certain amount of leeway, and of course, the lower the value of the tolerance level, the longer it takes to train the system.
The Fuzzy Way To Do It.
Once again, the task of the Neural Network at this point is a matter of rules, and as with the previous look at the Adaline Network, I stated that even though the Neural Network technically works out what the rules are for itself. These rules are in fact known to the programmer before hand and just as with the Adaline, the Back Propagation is trained with the answers to the questions, which means that the training for the network is nothing more than working out how to get the given answer from the given input. Once again, it strikes me that a simple rules based mechanism can achieve the same result in a lot less time with a lot less work.
As with the previous example, I start with an interface that describes the rules that apply to the program. The interface for this code looks like:
public interface IBackPropagationRules
{
bool IsOneVarEqualOne
{
get;
set;
}
bool IsOneVarEqualZero
{
get;
set;
}
bool AreBothVarsEqualOne
{
get;
set;
}
bool AreBothVarsEqualZero
{
get;
set;
}
}
This interface is more specific to the Back Propagation Network example than the interface face used in the previous Adaline example, and for this reason, is included in the demonstration code file rather than as part of a library. The basic idea behind it is to define the rules that we wish to use within the code without actually specifying the way in which those rules should be implemented. The reason for this being that within projects, the rules for one part of the project may well be the same as the rules for a different part of the project although the implementation of those rules may be completely different. It is just a way of saying that these are the rules to be implemented at this point, how you implement them is of no concern to me but these are the rules to use.
At this point, the class implementation is set up and this is another separation from the previous example in that the class is defined specifically as a rules class for implementing the rules within the program and is not defined or implemented as a part of the class.
class Rules : IBackPropagationRules
{
private bool bOneVarEqualsOne;
private bool bOneVarEqualsZero;
private bool bBothEqualOne;
private bool bBothEqualZero;
public Rules()
{
bOneVarEqualsOne = false;
bOneVarEqualsZero = false;
bBothEqualOne = false;
bBothEqualZero = false;
}
The Rules class inherits from the interface and implements the rules defined in the interface. The class has four member variables that are used to track which rule is being used at any given time. This is because one of the essential ideas behind this method is that the rules are flexible in that certain requirements within the code or class can require the use of different rules for each object within the class.
The Interface is implemented as:
#region IBackPropagationRules Members
public bool IsOneVarEqualOne
{
get
{
return bOneVarEqualsOne;
}
set
{
bOneVarEqualsOne = value;
}
}
public bool IsOneVarEqualZero
{
get
{
return bOneVarEqualsZero;
}
set
{
bOneVarEqualsZero = value;
}
}
public bool AreBothVarsEqualOne
{
get
{
return bBothEqualOne;
}
set
{
bBothEqualOne = value;
}
}
public bool AreBothVarsEqualZero
{
get
{
return bBothEqualZero;
}
set
{
bBothEqualZero = value;
}
}
#endregion
As you can see, the implementation of the interface is nothing more than a standard setting of the values to say which rule is implemented and which rule isn't. It is in the Compare function of the Rules class that the rules are implemented.
public int Compare( FuzzyInteger nOne, FuzzyInteger nTwo )
{
if( IsOneVarEqualOne == true )
{
if( nOne.Number == 1 && nTwo.Number == 0
|| nTwo.Number == 1 && nOne.Number == 0 )
return 1;
else
return 0;
}
else if( IsOneVarEqualZero == true )
{
if( nOne.Number == 0 && nTwo.Number == 1
|| nTwo.Number == 0 && nOne.Number == 1 )
return 1;
else
return 0;
}
else if( AreBothVarsEqualOne == true )
{
if( nOne.Number == 1 && nTwo.Number == 1 )
return 1;
else
return 0;
}
else if( AreBothVarsEqualZero == true )
{
if( nOne.Number == 0 && nTwo.Number == 0 )
return 1;
else
return 0;
}
else
return 0;
}
All the Compare function really does is check which rule is being enforced at the moment, the implementation of the rules is really simple.
The loop for the code is:
for( int i=0; i<patterns.Count; i++ )
{
leftValue = ( ( FuzzyIntegerPattern )patterns[ i ] ).InputValue( 0 );
rightValue = ( ( FuzzyIntegerPattern )patterns[ i ] ).InputValue( 1 );
Rules rule = new Rules();
if( this.oneVarCheckBox.Checked == true )
{
rule.IsOneVarEqualOne = true;
rule.IsOneVarEqualZero = false;
rule.AreBothVarsEqualZero = false;
rule.AreBothVarsEqualOne = false;
outputBox.AppendText( "One Var is Equal to One Rule Used\n" );
}
else if( this.zeroVarCheckBox.Checked == true )
{
rule.IsOneVarEqualZero = true;
rule.IsOneVarEqualOne = false;
rule.AreBothVarsEqualOne = false;
rule.AreBothVarsEqualZero = false;
outputBox.AppendText( "One Var is Equal to Zero Rule Used\n" );
}
else if( this.allOneCheckBox.Checked == true )
{
rule.AreBothVarsEqualOne = true;
rule.IsOneVarEqualOne = false;
rule.IsOneVarEqualZero = false;
rule.AreBothVarsEqualZero = false;
outputBox.AppendText( "Both Vars are Equal to One Rule Used\n" );
}
else
{
rule.AreBothVarsEqualZero = true;
rule.IsOneVarEqualOne = false;
rule.IsOneVarEqualZero = false;
rule.AreBothVarsEqualOne = false;
outputBox.AppendText( "Both Vars are Equal to Zero Rule Used\n" );
}
switch( rule.Compare( leftValue, rightValue ) )
{
case 0:
{
outputBox.AppendText( "Zero returned when comparing, left value :- "
+ leftValue.Number.ToString() + ", and right value :- "
+ rightValue.Number.ToString() + "\n" );
}break;
case 1:
{
outputBox.AppendText( "One returned when comparing, left value :- "
+ leftValue.Number.ToString() + ", and right value :- "
+ rightValue.Number.ToString() + "\n" );
}break;
default: outputBox.AppendText( "Arrgghh Vomit Stains\n" ); break;
}
}
The main loop for the code simply checks which check box on the rules page has been checked and then implements the rule. If you are really fast or have an extremely slow computer, you might be able to change the rules while the code is running, but the purpose of this example is just to show how to do it. If you change the rules and simply run the code, the new rule will apply.
Finally
So far, this rules concept has not been tested on anything serious or even slightly complicated, and it will be interesting to see if like so many good ideas, it can stand up to some real development beyond simple examples, or will it just disappear in a haze of confusion. Sometimes, I think that it will just end up merging into the background of a large project and not provide any real benefit, and at other times, I think it will provide a better easier to understand structure, but in the end, there is only one way to find out and that is to shut up and do some work.
