Hybrid Cross-Platform Applications: Approach to Using Cocos2d-x on Top of Native iOS and Android Components

• Download WebViewPoc.zip - 4.1 MB

Introduction

As it is known Cocos2d-x is an open-source game framework written in C++, with a thin platform dependent layer. It is widely used to build games, applications and other cross platform GUI based interactive programs (http://cocos2d-x.org). The engine itself is focused on providing convenient cross-platform means of creating and manipulating graphics elements (sprites, textures, etc.).

In spite of that it provides a set of GUI components called widgets (buttons, text fields, scroll views, etc.) they have quite limited features. Moreover, some of these components are based on native implementations of each operating system. It can be seen in the source code of Cocos2d-x (edit box, web view, video player, etc.). Therefore, it is quite clear how native UI components are used on top of the Cocos2d-x OpenGL layer.

However, there are cases when projects require using native components under the Cocos2d-x OpenGL layer:

• playing movies in background;
• using web content as a part of the application;
• creating other specific components based on native implementations.

Such kind of functionality is not provided in Cocos2d-x right out of the box. Sometimes it makes developers look for solutions, which are not common for this engine.

Let’s consider a general approach to using Cocos2d-x GUI on top of native components and discover some coding specifics in Cocos2d-x, iOS and Android. It will be done by implementing the native web view in the default demo sample of Cocos2d-x framework.

Background

It is required to consider the following ideas in order to achieve the introduced goal:

1. Implement native components for each operating system.
2. Provide a container view for native components. The view must be located under Cocos2d-x main view in the view hierarchy.
3. Make Cocos2d-x main view transparent so that we can see the native components through it.
4. Prepare a mechanism of touches in the cross-platform code of Cocos2d-x to be ready for interaction with native code.
5. Support the mechanism of touches in the native code of each operating system.

Touches require a special approach, as they are not forwarded from Cocos2d-x OpenGL view by default. The chosen solution is based on whether any Cocos2d-x component subscribes to listen to the "touches begin" event and handles it returning true in the corresponding callback.

For example, if there is some Cocos2d-x GUI widget handling the event, the event will not be forwarded to the native component view. Otherwise, the event will get through to the native component view and it will be handled by native controls.

Therefore, if there are some Cocos2d-x components (e.g., image view) ignoring touches by default, touches on such components will be handled by the native component view.

The following picture demonstrates the described ideas.

Using the code

The code is based on the Cocos2d-x version 3.8.1 and its demo sample. It has to be combined with the full Cocos2d-x framework before it is compiled.

The suggested coding solution is one of many possible solutions. It has been chosen for its simplicity and a minimum number of changes in order to see quick results and get some kind of “proof of concept”.

Cocos2d-x code

Prepare the method to return the status of interaction into the native code. It returns true if the event is handled by Cocos2d-x.

// CGLView.cpp

bool GLView::handleTouchesBegin(int num, intptr_t ids[], float xs[], float ys[])
{
    //...
    touchEvent._eventCode = EventTouch::EventCode::BEGAN;
    auto dispatcher = Director::getInstance()->getEventDispatcher();
    dispatcher->dispatchEvent(&touchEvent);

    // EK: cancel the event, otherwise it stays in the cache and can block similar events
    if (!touchEvent.isClaimedByCocos())
    {
        handleTouchesCancel(num, ids, xs, ys);
    }

    // EK: return the status into the native code
    return touchEvent.isClaimedByCocos();
}

Update the status if the event is handled by Cocos2d-x.

// CCEventDispatcher.cpp

void EventDispatcher::dispatchTouchEvent(EventTouch* event)
{
    //...
    auto onTouchEvent = [&](EventListener* l) -> bool {
        EventListenerTouchOneByOne* listener = static_cast<EventListenerTouchOneByOne*>(l);
            if (!listener->_isRegistered)
                return false;
            event->setCurrentTarget(listener->_node);
            bool isClaimed = false;

            std::vector<Touch*>::iterator removedIter;
            EventTouch::EventCode eventCode = event->getEventCode();
            if (eventCode == EventTouch::EventCode::BEGAN)
            {
                if (listener->onTouchBegan)
                {
                    isClaimed = listener->onTouchBegan(*touchesIter, event);
                    if (isClaimed && listener->_isRegistered)
                    {
                        listener->_claimedTouches.push_back(*touchesIter);
                    }
                }
            }
            //...

            // EK: update the status whether cocos takes care of the event
            event->setClaimedByCocos(isClaimed);

Set clear color for Cocos2d-x frame buffer.

// CCDirector.cpp

void Director::setOpenGLView(GLView *openGLView)
{
    //...
    _defaultFBO = experimental::FrameBuffer::getOrCreateDefaultFBO(_openGLView);

    // EK: clear color
    _defaultFBO->setClearColor(cocos2d::Color4F(0.0, 0.0, 0.0, 0.0));
    _defaultFBO->retain();
}

Update JNI implementation to receive the status of the event in Android.

// TouchesJni.cpp

extern "C" {
    JNIEXPORT jboolean JNICALL Java_org_cocos2dx_lib_Cocos2dxRenderer_nativeTouchesBegin(JNIEnv * env, jobject thiz, jint id, jfloat x, jfloat y) {
        intptr_t idlong = id;

        // EK: pass the status to Android
        return (jboolean) (cocos2d::Director::getInstance()->getOpenGLView()->handleTouchesBegin(1, &idlong, &x, &y) != false);
    }

iOS code

Implement the native container view and the web view.

// AppController.mm

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {   
    //...   

    // EK: create container view
    UIView *containerView = [[UIView alloc] initWithFrame:[window bounds]];
 
    // EK: create web view
    UIWebView *webview=[[UIWebView alloc]initWithFrame:[window bounds]];
    NSString *url=@"https://www.google.com";
    NSURL *nsurl=[NSURL URLWithString:url];
    NSURLRequest *nsrequest=[NSURLRequest requestWithURL:nsurl];
    [webview loadRequest:nsrequest];
   
    // EK: add subviews
    [containerView addSubview:webview];
    [containerView addSubview:eaglView];

    _viewController = [[RootViewController alloc] initWithNibName:nil bundle:nil];
    _viewController.wantsFullScreenLayout = YES;

    // EK: set container view instead of eagle view
    _viewController.view = containerView;

    // EK: set eagl view as transparent
    eaglView.opaque = NO;

Add the following method to hit test Cocos2d-x. Returning a valid pointer or nil from hitTest for "touches begin" defines to which view all other related events such as "touches move", "touches end" and "touches cancel" will be forwared. That is why it is needed to check only "touches begin" on Cocos2d-x side. Since this is a simulation of "touches begin", the event is cancelled if it is handled by Cocos2d-x. After returning a valid pointer all the following events including "touches begin" will be handled by the existing methods: touchesBegan, touchesMoved, etc. Otherwise, the events will be forwarded to the native component view.

// CCEAGLView-ios.mm

- (UIView *)hitTest:(CGPoint)point withEvent:(UIEvent *)event
{
    UIView* view = nil;
    UIView* ids[IOS_MAX_TOUCHES_COUNT] = {0};
    float xs[IOS_MAX_TOUCHES_COUNT] = {0.0f};
    float ys[IOS_MAX_TOUCHES_COUNT] = {0.0f};

    ids[0] = self;
    xs[0] = point.x * self.contentScaleFactor;
    ys[0] = point.y * self.contentScaleFactor;
   
    // EK: simulate hit test
    auto glview = cocos2d::Director::getInstance()->getOpenGLView();
    if (glview->handleTouchesBegin(1, (intptr_t*)ids, xs, ys)) {
        view = self;
        glview->handleTouchesCancel(1, (intptr_t*)ids, xs, ys);
    }

    return view;
}

Android code

Implement the web view and use the existing frame layout as the container view.

// Cocos2dxActivity.java

public void init() {
    //...

    // EK: create web view
    ViewGroup.LayoutParams pars = new ViewGroup.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT,
        ViewGroup.LayoutParams.MATCH_PARENT);
    WebView webView = new WebView(this);
    webView.setLayoutParams(pars);
    webView.setWebViewClient(new WebViewClient());
    webView.loadUrl("https://google.com");

    // EK: add view to layout
    mFrameLayout.addView(webView);

    this.mGLSurfaceView = this.onCreateView();

    // EK: set z order to make transparency working
    this.mGLSurfaceView.setZOrderOnTop(true);

    mFrameLayout.addView(this.mGLSurfaceView);

Check the event on Cocos2d-x side and forward all related events to Cocos2d-x or the native component view. Returning bool status from onTouchEvent for "touches begin" defines to which view all other related events such as "touches move", "touches end" and "touches cancel" will be forwared. That is why it is needed to check only "touches begin" on Cocos2d-x side in Android also.

// Cocos2dxGLSurfaceView.java

@Override
public boolean onTouchEvent(final MotionEvent pMotionEvent) {
    //...
    EventRunnable runnable = null;
    switch (pMotionEvent.getAction() & MotionEvent.ACTION_MASK) {
        //...
        case MotionEvent.ACTION_DOWN:
            final int idDown = pMotionEvent.getPointerId(0);
            final float xDown = xs[0];
            final float yDown = ys[0];

            runnable = new EventRunnable() {
                @Override
                public void run() {
                    this.setIsClaimed(Cocos2dxGLSurfaceView.this.mCocos2dxRenderer.handleActionDown(idDown, xDown, yDown));
                    synchronized (this) {
                        this.setIsProcessed(true);
                        notifyAll();
                    }
                }
            };
            break;
    //...

    boolean result = true;
    if (runnable != null) {
        this.queueEvent(runnable);
        synchronized (runnable) {
            if (!runnable.isProcessed()) {
                try {
                    runnable.wait();
                } catch (InterruptedException e) {
                }
            }
            result = runnable.isClaimed();
        }
    }

    return result;
}

The example produces the following result, where the Cocos2d-x close button is clickable as well as the native web view is fully functional.

Points of Interest

The web view has been hardcoded as a native component for time saving and demonstration purposes. Therefore, it is a good area of improvement to implement approaches to adding such components by means of Cocos API.

History

• Added Cocos2dxRenderer.java file to the sources.