Overview
In this article, I will be discussing about autonomous intelligent Home Assistant. It is a virtual Robot for you and your home. It will take care of all your home needs. It will check your household groceries and order for you. It will speak to you and you can speak to the virtual robot. The robot will be able to recognize your commands and order whatever products you want to order.
It is helpful for all families in bringing the intelligent systems in their lives. There is no need to go to shopping malls and it is a new way of ordering things. The intelligent desktop app will do all the things for you. You can order directly or else it will order intelligently according to your needs.
App Specifications
You may have heard about Richard Wallace Chat Bot, the AIML based chatting application. The same AIML (Artificial Intelligence Markup Language) technology is used for background intelligence of virtual Robot. The App will have a speaking robot as GUI and also it can work as a background task. It will use Google's speech engine for converting your speech into Text or Microsoft speech Engine. The AIML and WebServices are used for taking your order. A wide angle bar code scanner which can scan 4 to 5 items at a time can be used to scan the predefined tags on grocery bottles. The motor connected to scanner will scan all the grocery bottles and identify the quantity of product is less than 20 % or not. The scanner is Microsoft Kinect sensor itself. Whenever your household items especially groceries are less than 20%, then it will automatically order for the new products.
Remaining Products
There are many other products which are quantity based like toothpaste, hair oil, facial creams, soaps and perfumes which are mostly constant on a monthly basis. It will order for you every month depending on your usage. And also, if you need to order something else, you can order directly by your voice or by using scanning. Daily orders like milk, curry leaves, and other items will be ordered on a daily basis.
Example for Voice Order
The voice order will be processed like this example if you need a cold drink.
Human: Get me a cold drink
Robot: I am ordering for you a Coke from Walmart. Is it ok?
Explanation: The process will happen like this. It will take cold drink as a keyword from your sentence and it will pick your most frequent order, i:e., Coke in this example and order from the nearest retailer shop Walmart as it is the nearest store and it will deduct from your account and place an order. Then if you say yes or ok, it will get it for you, or else, it will ask questions depending on your criteria and the order will be made.
Also, if you want to order a particular item from some particular shop, then you are welcome.
Human: Get me pepsi from Reliance Mart.
Robot: Yes Sir, should I order?
Depending on your needs, it will constantly be monitoring you and ordering for you and make you feel happy.
Some Valuables
If you need some other items like dress or jewellery, you can dress virtually using Kinect sensor. It will measure your body measurements and give you a variety of options to choose from and also change colors accordingly if you fit into the clothes. You can virtually wear jewellery and check from home itself. This is new style of shopping the things. The Artificial intelligence and Powerful Computing Machine will give you a new style of living. You can even monitor your home or children from your office by pinging to your robot address and you can virtually play with children and look after them.
When any product is completed, you have a chance to renew new things by making voice commands or else scan and throw your waste wrappers it will order for you.
About the Uploaded Files
The uploaded files are an intelligent chat system image, and a Robot which can stream video and audio and also it will have intelligent speech with humans which I am currently developing.
Category and Platform Details
My category goes into Retail (the way we shop). According to category, my platform goes to the tablet. But since I have to do video processing and Kinect barcode scanning, I think the platform should be All in One (for good amount of run time memory). So I request for All in One.
Approach
- The first step is to scan bar code using kine-ct sensor and get the bar code and process it to get the details on the bar code.
- Then from the back end, the submission form of order will be filled with defaults depending on the users interest (He usually takes 1kilogram quantity from Walmart store). The form will be filled accordingly and the order will be processed.
- And whatever is going on should be notified to the user at billing time.
- The second step is getting voice commands from the user. Every time, it will listen to you depending on the amplitude of your voice.
- If you are in conversation with others, the application will listen for 10 seconds and it will delete the conversation.
- If you want to start a conversation with the robot, you can ping it in two ways. Come into field view of robot and speak to robot, or you can start making a conversation by specifying it like "Robot what are you doing?"
- Then robot will be taken as keyword and it will give an answer according to your question.
- The third step is to order taking from your commands. You say robo get me ice cream.
- The Robot will see the pattern of the sentence if pattern like "get me ***" take the star as keyword and search in the product list and its manufacturing list and then retailers list and order for you.
- It will order chocolate ice cream from vanilla brand from nearby retail store, and that toi depends on users' previous orders.
- The fourth step is video streaming remotely. Talk to your robot remotely and monitor your house and also through video streaming, you can entertain your children.
Coding Languages
- AIML
- C#
- Kinect SDK
- XML
- JavaScript
- HTML, CSS, JSP
- Zxing library for bar code scanning
- and other utils
Currently, I am working on AndroHumanoid Robot which is a replica of virtual home assistant. Now, I am working on real robot. The same robot which can be made virtually for intelligent homes as discussed above.
You can watch the video below:
History
- 11th September, 2015: Initial version
