Developing AI chatbots

Introduction

Artificial Intelligence (AI) technology provides techniques for developing computer programs that carry out a variety of tasks, simulating the intelligent way of problem-solving by humans. The problems that humans solve in their day-to-day lives are of a wide variety and in different domains. Although the domains are different and so are the methods, AI technology provides a set of formalisms to represent the problems as well as techniques for solving them. What AI technology provides us is what is described in the above sentences. Based on this, it is very difficult to precisely define the term "artificial intelligence."

Chatbots

As you know, the chatbot is a chatting robot that can understand what you are saying, analyze it and give you a suitable response. It's considered to be a serious branch in AI development, as the purpose of programming a chatbot is to help people obtain information. Examples include:

• Selling chatbots: These help people to know item prices and offers.
• Supporting chatbots: You may find this kind of chatbot on websites that offer products and services.
• Help desk (information desk) chatbots: You may find these in large libraries, websites or programs.
• Entertaining chatbots: These are made for fun and chatting.

In 1966, Eliza, the first chatbot, was created by Joseph Weizenbaum. In this article, we are going to discuss how to build your chatbot, how to increase its ability to understand, how to make your 'bot able to do operations that help you in your daily life and, finally, we'll also discuss some tips and tricks to make your chatbot seem almost human. We are going to focus on the logics only, so any programmer can program the logics into any programming language.

Building your chatbot

Before building your chatbot, you have to understand that any chatbot consists of three main parts:

The most important part is the sending button, as it contains all of the processes that analyze your input before copying the result to the output. For this reason, I will call the sending button the Artificial Intelligence Base (AI Base).

The AI Base principles

Any dialogue has a beginning and ending, like saying "Hello"-"Hey"-"Good morning"-"Hi, how are you?" etc. We always hear these sentences when two people start talking. There are also other situations that don't need these sentences to begin, like "Excuse me, what is the time?" etc. On the other hand, we hear other sentences like, "I have to go now" - "bye, see you later" - "I'll call you later" - "looking forward to see you," etc. These sentences are usually used at the end of conversations.

As you can see, there are countless possibilities for inputs that your chatbot may not understand. For this reason, we have to classify any dialogue to a number of subjects and expressions. We also need to create files for each subject and each expression containing all of the related words. See Figure 1. Now your chatbot has the database with which to understand user inputs and give suitable outputs. However, it's not entirely accurate because some sentences have similar words but very different meanings.

Example:

"I hate you." This will make your chatbot angry.

"I hate myself." This may also make your chatbot angry, as he is looking for the word "hate." This is wrong because the phrase is does not convey hate towards the chatbot; rather, the user is directing it at himself. So the chatbot should be sad or surprised, not angry.

You must therefore build search engines to look for user inputs in different ways, to make sure that your chatbot will understand user inputs well and give suitable responses. We are going to discuss these search engines, which I call Special Search Engines and Matrix Search Engines.

Special and Matrix Search Engines

The Special Engine compares between sentences stored in the database and user inputs, while the Matrix Engine compares between words stored in the database and user inputs. Each file has two types, Matrix Database and Special Database. Each type of database uses its engine.

File: "Laugh"

...to Special "Laugh" Database

...to Matrix "Laugh" Database

The following example shows the difference between the two engines.

As you see in Figure 2, the Matrix Engine failed to give the correct answer because it was looking for words. You may think that the Matrix Engine is useless, but it is actually not. As I said before, you can't fill the AI Base with all possibilities and so the Matrix Engine is very important to cover more possibilities than the Special Engine.

Figure 3 shows the importance of the Matrix Engine in understanding three different sentences that have the same meaning. The match is with only one record in the Matrix Database. That means that the Special and Matrix Engines complete each other. One last thing you should know about this engines is that the Special Engine is a part of (belongs to) the Matrix engine and so you should arrange the steps of searching as follows:

• 1st search in: Special Engine. If a result is found, then give the output. Else search using the Matrix Engine.
• 2nd search in: Matrix Engine. If a result is found, then give the output. Else search in the One Word File.

If you follow these steps in searching, you will not face the problem of the Matrix Engine in Figure 2 or the problem of the Special Engine in Figure 3. You may wonder what the One Word File is after both the Matrix Engine and the Special Engine failed to find any matches. One Word File is a database that contains only one word that may be found in user inputs. So, your chatbot may reply with something related to the user input, but with less accuracy.

The accuracy of the Special Engine, Matrix Engine and One Word File:

• If the result is found in the Special Database, its accuracy in producing suitable output is roughly 95%.
• If the result is found in the Matrix Database, its accuracy in producing suitable output is roughly 65%.
• If the result is found in the One Word Database, its accuracy in producing suitable output is roughly 15%.

You can increase the accuracy of your chatbot's response by increasing the records of Matrix and Special Databases for each file and One Word Database.

Chatbot response

I'm now going to discuss how the chatbot responds and gives suitable answers depending on user inputs. I will also examine how to make your chatbot give more than one answer for the same input. This will make your chatbot more real, human-like and more reasonable. Each file has two databases: Special and Matrix. Each database also contains two fields:

• 1st field: "Request" is where the user inputs are compared with data recorded in this field. See Figures 2 and 3.
• 2nd field: "Respond" is where the chatbot outputs are taken from data recorded in this field.

In Figure 4, there are two outputs for the same input. Also, one row can contain more than one request and respond value. This decreases the number of records, as it reduces the repeated (related) sentences in one row.

How to make your chatbot respond by different ways

To do this, you have to create a variable with an initial value equal one. Every time your chatbot gives a response, add 1 to the variable value. When the Request value matches with the user input, the output equals the sentence that has the number of the variable. If the variable is greater than the last sentence number, make this variable equal to 1 and let the output equal the 1st sentence. Otherwise, add 1 to the variable value.

Figure 5 shows how to merge related records into one record. This also leads to merging responses into one record, which makes your chatbot respond in different ways.

Misunderstanding

Assume that, after trying both the Matrix and Special Search Engines, your chatbot didn't find any records similar to the user input. This means that your chatbot can't understand the user input. Perhaps the user mistyped the input or perhaps you simply have poor AI databases. You need to create a database that your chatbot can use to generate sentences about misunderstandings.

Example:

  I don't understand. 
  Please explain.

Also, you can improve the database by inserting sentences that don't refer to understanding or misunderstanding. These sentences leave an impression on the user, as they make your chatbot appear to understand.

 

Example:

  Wait, I got a phone call. 

  Oh, my keyboard hung again.

Other engines

Previous Events Engine

 

When you ask anyone about something and then you ask them about it again, they are going to say, "I told you that …etc." That means that his/her mind recorded a previous conversation about the same subject and that made him/her say this sentence. The Previous Events Engine function simulates the same thing with your chatbot.

Figure 6 shows how the Previous Events Engine works. You may have noticed that there are two databases for this engine: the Request Database, which records user inputs if there is no repeated input, and the Respond Database, from which your chatbot gets a random sentence that expresses a repeated input.

Auto Talk Engine

This engine makes your chatbot respond automatically if the user hasn't chatted in a specific period of time. You can also improve this engine by making your chatbot wait if the user is typing a message. Assume that two minutes is the defined period of time before this engine works. If the time is up, your chatbot sends a response from the Auto Talk Database.

Example: 

  chatbot: Where did you go?

  chatbot: Are you there?

  chatbot: I'm bored from talking to my self.

AM-ARE Functions Engine

 

This engine is considered to embody the infrastructure of any chatbot. The AM-ARE Functions Engine's main task is to recognize the pronunciation in the user's sentence input. It should also recognise some verbs and come with their pronunciations in all tenses, such as the verbs of "to be," "to have" and "to do," as well as modal verbs. It then uses them in the response sentence. To understand what I mean, let's take an example:

The user says, "You are suspicious." If we look at this phrase, we will find the two important components: 1- Pronounce you, 2- Verb. At first, the 'bot recognizes and excludes them from the sentence said to him. Then it begins to replace some pronunciations in the sentence like I/We, You, My/Our, Your, Me/Us, You, Myself, Yourself, Ourselves, Yourselves, etc. It also replaces their verbs like Am, Are, Was, Were, etc. You can also add a small part to the sentence like, "Why do you think that?" which will make the final output: "Why do you think that I am a suspicious?" You can also convert the sentence to a question, such as, "Am I a suspicious?"

Convincing chatbots

Proverbs

The advantage of adding proverbs is that they can be convenient responses to a lot of sentences. It also makes your chatbot seem wiser. Say the user is talking about trouble. The 'bot can say, "Don't trouble trouble until trouble troubles you." If he is talking about being greedy, "A bird in the hand is worth two in the cloche." As you know, there are many proverbs that are suitable for many subjects.

Increasing your chatbot's intelligence

Your chatbot's intelligence depends on two factors: good logic and databases that contain a huge number of records. Since the most important factor is the database, I should program something to increase my chatbot's database while I'm chatting with him. I will call this the Learn Engine. The Learn Engine's purpose is to make your chatbot understand information given by user, record it in his database and then recall it when the user asks about it. For instance...

"WHAT IS THE COLOR OF THE SKY?

OF COURSE, THE SKY COLOR IS BLUE  BUT SOMETIMES
IT SEEMS TO WHITE AS THERE IS SOME CLOUDS

In the previous example, the chatbot replied with the right answer. Example: (when your chatbot doesn't know the answer)

HOW MANY FINGERS IN YOUR  HAND?

WHY YOU DON'T ANSWER  THIS DIFFICULT QUESTION,
I'D LIKE TO HEAR THE ANSWER FROM YOU?

YOU HAVE 5 FINGERS IN  ONE HAND

BEFORE GOING TO SAVE  THAT IN MY MIND, IS THAT
THE ANSWER OF YOUR QUESTION?

YES

I APPRECIATE TEACHING  ME, SAVE APPROVED.

HOW MANY FINGERS IN YOUR  HAND?

YOU HAVE 5 FINGERS IN  ONE HAND

Also, your chatbot can recognize the behavior of any user while chatting with him/her. It can tell whether he/she is lying or not by analyzing previous user inputs, recording it and comparing it with the current conversation.

Tips and tricks to improve your chatbot's abilities

Improving your chatbot's abilities to do specific tasks is another important part of AI development. Examples of such AI operations include the ability to play chess while chatting, the ability to organize your schedule and the ability to control your PC. I'm now going to discuss some ideas for operations that are programmed into our chatbot, RoboMatic X1.

System Operations

Here is how to make your chatbot able to carry out certain System Operations, such as opening and closing CD-ROMs, browsing, copying folders, etc. Please remember that this is a dangerous area. Any fault in performing system operations may not only make user calls to chatbot erroneous, but may also make it so that the chatbot program doesn't open again. In the case of dangerous operations like deleting files, the chatbot must warn the user before performing them. Maybe the chatbot doesn't understand correctly or doesn't exclude the path or name of the deleting file correctly from the sentence said to him.

Operations don't consider an isolated working area without caring about other chatbot engines. So, we have to treat the operation sentences like ordinary sentences first. What we do when we make operations work depending on the Matrix Engine is to make the chatbot know the meaning of sentences and to choose the convenient answer to them.

There is a little difference in the Operations database, as it doesn't contain request and respond sentences. Rather, it contains some information about the operation that the chatbot should carry out. We do this a simple way by defining an ID number for every operation. When the chatbot finds the equivalent ID for the Operation Sentence, he does the operation that carries the same ID. There are two main parts of operations: Limited (direct) Operations and Extended Operations.

The Limited Operations are the commands that chatbot carries out directly without caring about the next response of the user. Examples would be opening a CD-ROM, opening a file, minimizing explorer widows, etc. These wouldn't need a sentence to warn the user that he/she is about to open the file, etc. The chatbot will open file straightaway. In this type of operation, the difficulty is found only in having the code of the desired operation in the language you program your chatbot in.

The Extended Operations are the commands that chatbot carries out with concern about the next response of the user, which may specify more details about it and how it should be done. The difficulty in this kind of operation is not only in the code, but also in how to relate between the command sentence and the next sentences wherein the user will inform the chatbot of more required information regarding the running operation.

In this part, efficiency in doing the task depends on the chatbot's intelligence. So, we will use the Previous Events Engine, whose main task is relating between sentences. To see what I mean, let's take an example and look at the following flow chart.

Figure 7 shows the copying of some files from a path (E:\) to a definite destination (F:\) where there are some files that have the same names of the copying files. Let's see how the chatbot will do an operation like that.

1. After the chatbot receives the sentence containing the command to copy files, it will be analyzed by the Matrix Engine which recognizes the command. It will choose a suitable response and make the user determine the files and destination they should be copied to.
2. The chatbot will begin to copy files to the destination where some files have the same name as the copying files. So, the chatbot will ask the user what to do in the message, "The paste destination already contains the file (name of the file). Do you want to overwrite it ? Yes, Yes to all, No, No to all." This sentence will be stored in the Previous Event Engine, in addition to the current process (copying). This allows the chatbot to relate between the question he asked and the user reply.
3. The user reply will pass through the AI engines until it reaches Previous Event. This will recognize that the previous event recorded for the copying operation hasn't finished yet. This reply belongs to this event, but there will be five possibilities for replying: Yes, Yes to all, No, No to all or a reply that has no relation to the chatbot's question.

Random calling name

This feature will make the dialogue with your chatbot seem more reliable, as it makes the chatbot add a user name in the beginning or at the end of the replied sentence. For example, assume that the user name is John.

John: Hi bot.

Bot: Hi john, how are you? <OR>
John, I'm glad to see you.  <OR>
Hello, john.

There are 3 ways to make your chatbot say the user name. The first way is to make the chatbot say the user name randomly, by adding it to the replied sentence if the engine chooses to add. The second way is by putting a symbol in your database respond records like the /N symbol, so when the chatbot finds it in the respond sentence he will replace it with user name. This way is not preferred, as it makes the chatbot always call the user name in the same position of the sentence and repeat it because he always finds the symbol (/N) in the sentence.

The third way is by combining the two previous ways. Let's take an example, the user saying, "I'm sure you can swim." The 'bot searches the Matrix Engine in the database records and then finds the response, "Why do you think that I can do this, (/N)?" It found that there is the mark /N in the sentence then randomly chose whether to replace the /N symbol with the user name or not. If not, the /N symbol will be removed from the sentence.

Math Engine

This engine enables your chatbot to calculate numbers, including the addition, subtraction and division operations.If you want to calculate something by chatting with your chatbot, you should improve upon this engine to search for the equation in the sentences. The following figure shows how to do that and how to make your engine accept numbers written in letters.

Colors/capitals databases

You can increase your chatbot's knowledge by teaching him the colors and capitals of countries. First, create two databases. The first one contains the sentences that the user can say to ask about the color/capital of the country like, "What is the color/capital of ...?" or, "Do you know the color/capital of ...?" The other database contains the countries and capitals. When the user says, "What is the capital of the USA?" the bot will search in the first database, where he will find this question and its response, "(/Cap) is the capital of (/Count)."

Next, the 'bot will exclude the name of the country from the question (USA) and replace it with the (/Count) symbol. Finally, the 'bot will search the second database to find the capital of this country (Washington) and then replace the (/Cap) symbol with it. The output will ultimately be, "Washington is the capital of USA." We can do the color database in the same way. Figure 9 shows how it works.

Reminders and notifications

To put this feature into your chatbot, you should create a database that contains all of the events that happen on a specific date, time or date and time in the records. After creating this database, make your chatbot always check the current time and date and compare it with the records in the database. If the current time, date or both equals one of the records then your chatbot should say what is recorded in the matched records.

 

Finally, you did it!

We hope that you enjoyed our simple article as a guide to building your own chatbot. You can use any programming language you want. We have discussed the basics of building a chatbot and ahve used real examples from our chatbot, Robomatic X1. You can download the chatbot as a reference. We'd also like to recommend some useful links to help you build your own chatbot:

• Personality Forge
• Alicebot
• Pandorabots

If you have anything you need to discuss with us, please fell free to contact us.

History

• 26 November, 2005 -- Original version posted
• 11 June, 2007 -- Article edited and moved to the main CodeProject.com article base