Intel® Developer Zone offers tools and how-to information for cross-platform app development, platform and technology information, code samples, and peer expertise to help developers innovate and succeed. Join our communities for Android, Internet of Things, Intel® RealSense™ Technology, and Windows to download tools, access dev kits, share ideas with like-minded developers, and participate in hackathon’s, contests, roadshows, and local events.
Overview
Multi-window is a feature in the Android* OS that can differentiate your apps. Many OEM and ODMs such as Samsung, Ramos, and Huawei use the feature to promote their products and make them stand out from the rest. In this article, we will introduce the multi-window function and show you how to implement it in your apps.
Figure 1. Multi-window use cases
Introduction
In June, 2012, the first open source multi-tasking framework, called Cornerstone, was developed. In August 2012, Samsung launched the first multi-window commercial product. And from 2013 until now, there has been an explosion of multi-window solutions on the market (see Figure 2).
Figure 2. Multi-window evolution
The two types of multi-window styles are: floating style and docked style. The multi-window feature usually includes the open/close, resize, and swap functions. The open/close function starts/stops the feature. The resize function allows users to change the window sizes. The swap function exchanges the window positions.
Figure 3. Multi-window window styles
In 2013, many solutions, developed by OEM/ODMs, by ISVs, and by the open-source community, emerged on the market. The following table compares the different multi-window solutions.
| Feature | Cornerstone | Standout | Xposed | Tieto |
| Description | Multi-tasking framework for the Android* OS | An open source library that can be used to create floating apps | A multi-window application that supports docked windows style | The project aims to create desktop-like user experience |
| Open/close , resize, maximize | Supported | Supported | Supported | Supported |
| Windows style | Docked | Floating | Docked | Docked/Floating |
| Code modification | Android framework | Application layer | Android framework | Android Framework |
| Application support | Support all applications, but SurfaceView cannot dynamically be adjusted | Some assistant applications, for example, calculator, etc. | Application compatibility and stability need improvement | Support all applications |
| Android version | Android 4.1 ~Android 4.4 | Android 4.1~
Android 4.4 | Android 4.4 | Android 4.4 |
| Official website | http://www.onskreen.com | http://forum.xda-developers.com/showthread.php?t=1688531 | http://forum.xda-developers.com/xposed | https://github.com/tieto/multiwindow_for_android |
Software Architecture
You can modify the Android framework code to adapt more functions. The Android OS architecture is divided into layers.
For Android 4.2 and Android 4.3, the launcher and other applications are all run on one stack, called "main stack". As we know, multi-window needs more stacks to contain the multiple windows, so we need to modify the class ActivityManagerService of the framework to add stack creation and stack management interface. To modify the class WindowManagerService of the framework to adapt the view, we need to modify the inputManager of the framework to dispatch the touch event to the corresponding windows.
Stack management was changed significantly when Android 4.4 and Android 5.0 were released. The launcher and other applications can run on different stacks. Stacks and stack management functions were added. Below shows the stack differences between Android revisions.
Figure 4. Stack management changes between Android* 4.3 and Android 4.4
Let’s focus on Android 5, codenamed Lollipop. As we know, the Android* OS uses a callback method to trigger the activity interface function. But the main function is realized on the framework, so we will introduce two important classes ActivityManagerService and WindowManagerService.
Figure 5. Lollipop’s software structure
Lollipop Activity Management
Because the multi-window feature depends on the stack, the following shows how to create a stack and how to start an activity on a stack.
In Lollipop, the IactivityManager.java added the following interface functions.
Table 1. Lollipop source code changes
| IactivityManager.java new add interface function | Description |
public void moveTaskToStack(int taskId, int stackId, boolean toTop) | Move task to another stack |
public void resizeStack(int stackBoxId, Rect bounds) | Resize the stack size |
public void setFocusedStack(int stackId) | Set the current focus stack |
Public Boolean isInHomeStack(int taskId) | get the task whether is in the HomeStack |
After the startup, the SystemServer process will launch the activity management services and window management services. We can add the RuntimeException sentences to trace the process.
Figure 6. Progress of stack creation in Lollipop
Now let’s show how to start an activity on a stack.
Figure 7. Start activity on a stack
In Lollipop, adb (android debug bridge) added the following commands.
Table 3. Lollipop’s new adb commands
| ADB COMMANDS | Function | Description |
| Adb shell am stack start | Start a new activity on <DISPLAY_ID> using Intent | Kitkat 4.4, adb commands contain adb shell am stack create.
Lollipop 5.0, adb commands adb shell am stack create deletion |
| Adb shell am stack movetask | Move <TASK_ID> from its current stack to the top or bottom of STACK_ID | Usage: adb shell am stack movetask task_id stackid true/false
Notice: it is ok on KitKat, but it doesn’t work on Lollipop. |
| Adb shell am stack resize | Change <STACK_ID> size and position to <LEFT, TOP, RIGHT, BOTTOM> | Usage: adb shell am stack resize task_id weight |
Lollipop window management
WindowManagerService is the core window manager. Its function contains input event dispatching, screen layout, and surface management.
Figure 8. WindowManagerService role in the graphics architecture2
Some common questions on multi-window
Multi-window has a resize function. We have seen cases where a gaming animation can’t be resized. The root cause of this is that the Android function SurfaceFlinger can’t dynamically adjust the size.
Figure 9. Games using SurfaceFlinger can’t dynamically adjust the windows size
Another issue is that some applications don’t display correctly in the multi-window. An example below shows the calculator not displaying correctly in the multi-window because the application uses a bad compatibility configuration.
Figure 10. Calculator with bad configuration
Will Android’s next version support multi-window?
Will Google release the multi-window feature in its next version of the OS. I found the following log information in the Lollipop source code. We use the following command to search the multi-window log.
git log --grep "multiwindow"
The log content contains a line that reads "defer tap outside stack until multiwindows". So we conclude that multi-window may be on Google’s roadmap.
Figure 11. Lollipop log about multi-window
Case study: Cornerstone
Onskreen developed Cornerstone, the first multi-window framework solution. It is for large screen devices and tablets. You can download the source code from GitHub3. It is only supported on the Android 4.1 and Android 4.2 versions. It not released yet on higher Android versions. But we can analyze the source code of Android 4.2 to get more technical details.
Figure 12. Cornerstone modifications on Jelly Bean
Summary
Many mobile devices now use Intel® processors running the Android OS. How can developers improve the user experience? How can their products best compete? These questions push us to constantly improve our products on Intel® Architecture (IA) devices. Multi-window is a good differentiating feature. It is convenient and allows consumers to do two or more things simultaneously. They can watch a video and IM friends with video feedback. They can play a game and read reviews of it. Several devices now support the multi-window feature: Ramos i12 tablet, Teclast x98 tablet, and Cube i7, which uses the Remix OS.
Figure 13. Multi-window feature on IA devices
Reference
[1] http://www.onskreen.com/cornerstone/
[2] http://himmele.googlecode.com/svn/trunk/Google%20Android/Android%20Graphics%20Architecture.pdf
[3] https://github.com/Onskreen/cornerstone
Resource
http://www.customizeblogging.com/2015/04/android-6-0-muffin-concept-video-shows-multi-windows-quick-reply-feats.html
About the Author
Li Liang earned a Master’s degree in signal and information processing from Changchun University of Technology. He joined Intel in 2013 as an application engineer in the Developer Relations Division CCE (Client Computing Enabling). He focuses on helping developers to differentiate their apps on the Android platform.
