Introduction
Sikuli is a visual technology and a simple way to automate and test graphical user interfaces using screenshot images. Sikuli originally started at the User Interface Design Group at MIT as an open-source research project. . The first release of Sikuli contains Sikuli Script, a visual scripting API for Python, and Sikuli IDE, an integrated development environment for writing visual scripts. Sikuli Script automates anything you see on the screen without internal API support. Thus, it is possible to control a web page, a desktop application running on Windows*/Linux*/Mac* OS X, or even an Android* application running via virtual network computing (VNC).
There are a number of applications for configuring a VNC server on Android devices. I can run a VNC server in Android and use a VNC viewer on my Windows machine, but I prefer to use the MyPhoneExplorer app for Android, and then run the Sikuli program on my PC. As a result, I can see the Android HOME screen on my Desktop.
In this article I explain how to validate applications on Android devices with the help of Sikuli IDE, a simple and effective scripting language.
Sikuli IDE Installation
First of all, there are some special steps for installing Sikuli. I will describe the process of installing it on Windows.
On the official site, you will find the 32-bit version of Sikuli. Don’t worry if you have a 64-bit Windows system (for 64-bit systems launch Sikuli-IDE-w.bat and don’t use Sikuli-IDE.exe)—it will work on both of these systems.
Before installing Sikuli, be sure that you have installed Java Runtime Environment (JRE) and have uninstalled any previous version of Sikuli.
- Download and install Sikuli: Sikuli-X-1.0rc3 (r905)-win32.exe. After installation is complete, a folder named Sikuli X should have been created on your system.
- Important Note: Do not start using Sikuli X now because it has some bugs. Download the zip file Sikuli X r930 which contains important bug fixes.
- Open the downloaded zip file and locate the folder called SIKULI-IDE.
- Copy the content in SIKULI-IDE to Sikuli X. The purpose of this step is to replace the files associated with r905 (the buggy version) with the files associated with r930 that have the most recent bug fixes.
Launch Android in Windows
I will use a Lenovo K900 for testing games and, in particular, I will show how easy it is to validate Angry Birds* on this device. Now let’s connect the device to Windows with MyPhoneExplorer.
- Install the MyPhoneExplorer on an Android device and PC.
- Connect the Lenovo K900 with a USB [File -> Settings -> connect via USB Cable].
- Launch MyPhoneExplorer in Windows.
- Extras -> Control Phone/Load screenshot -> check refresh automatically
Figure 1: Lenovo K900 with MyPhoneExplorer
Now we can manage an Android device with our PC.
Creating a Bot for Angry Birds on Android Device
Launch Sikuli IDE editor in Windows.
Figure 2: Sikuli IDE
Then let’s see it on the Angry Birds app.
Figure 3: Angry Birds on Lenovo K900
Be sure to click on the button with the white triangle to run the game. There is a Click() command in Sikuli for pushing onto any graphical object. Use the Sleep() instruction for waiting to refresh the scene.
We have the following code:
click("1400666128574.png")
sleep(20)
Click the button with the white triangle (1400666128574.png is the image for this button) and then wait 20 seconds until the scene changes.
When you launch the app the first time, you will see the advice. Sikuli likes any script language that has conditions and cycles. I will use condition "if" for handling this event.
Figure 4: Lenovo K900: 1 level of Angry Birds
if (exists("1400672718307.png")):
click("1400672718307.png")
sleep(20)
This code snippet checks to see if the button shows up on the screen.
Now it’s time to show how to launch the bird. For this, I use the dragDrop (img1, img2) command:
dragDrop("1400171376311.png", "1400253224137.png")
This is how we launched the bird at a certain angle and shot down all the pigs. So far, we have developed a bot which plays the first level of Angry Birds and, as you can see, it is very easy because we didn’t use difficult constructions in this script.
click("1400666128574.png")
sleep(20)
click("1400666165817.png")
sleep(20)
click("1400666191490.png")
sleep(60)
if (exists("1400672718307.png")):
click("1400672718307.png")
sleep(20)
dragDrop("1400171376311.png", "1400253224137.png")
sleep(60)
click("KQY.png")
On the next screenshot, you will see what this code looks like in Sikuli IDE.
Figure 5: Sikuli IDE code
Now we can use this script for validation of the first level of Angry Birds. It is possible to validate any application with the same ease.
I hope that my blog helps you to validate your own app without any difficulties.
About the Author
Vitaliy Kalinin works in the Software & Service Group at Intel Corporation. He is a PhD student of Lobachevsky State University in Nizhni Novgorod, Russia in the economics department. His specialty is applied mathematics and informatics in economics. His main interest is mobile technologies and game development on PC’s and smartphones.
Related Articles and Resources:
Notices
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
UNLESS OTHERWISE AGREED IN WRITING BY INTEL, THE INTEL PRODUCTS ARE NOT DESIGNED NOR INTENDED FOR ANY APPLICATION IN WHICH THE FAILURE OF THE INTEL PRODUCT COULD CREATE A SITUATION WHERE PERSONAL INJURY OR DEATH MAY OCCUR.
Intel may make changes to specifications and product descriptions at any time, without notice. Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined." Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them. The information here is subject to change without notice. Do not finalize a design with this information.
The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.
Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.
Copies of documents which have an order number and are referenced in this document or other Intel literature may be obtained by calling 1-800-548-4725 or going to: http://www.intel.com/design/literature.htm
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark* and MobileMark*, are measured using specific computer systems, components, software, operations, and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products.
Any software source code reprinted in this document is furnished under a software license and may only be used or copied in accordance with the terms of that license.
Intel and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries.
Copyright © 2014 Intel Corporation. All rights reserved.
*Other names and brands may be claimed as the property of others.
