Multi touch and phonegap

[troels]

In ST 2.1 on Android 4, I'm seeing an issue when doing multi touch that causes my app to go into an error state, where no touch events are propagated to any components. i.e. the app completely freezes.

On any container, it need not be scrolling or anything fancy, a drag with two fingers will cause this error. Note that the two fingers must hit the screen at the exact same time. After triggering the error, the browser will start throwing TypeErrors like:

Uncaught TypeError: Cannot read property 'point' of undefined at file:///android_asset/www/resources/js/sencha-touch-all-debug.js:75290


The culprit is the Ext.event.recognizer.Pinch class in the onTouchMove function that expects two points from the touch event, but only receives one. This crash seems to stop propagation throughout the rest of the recognizer classes.

When in this faulty mode, I can see the PhoneGap is still sending touch events because the (appearently immortal) line

WARN/webview(29014): Miss a drag as we are waiting for WebCore's response for touch down.

still pops up when touching the screen.

On containers that are scroll enabled, I have successfully undone this faulty state, by spamming multi touch events. At some point, ST realizes that a new multitouch event has begun and (im guessing) stops listening for updates to the one that crashed.

A fix for this would be nice, but I can't launch a app with an error like this, so my question is:
Where should I catch the TypeError and how to I cancel the touch event Ext.event.recognizer.Pinch still thinks is going on?

[mitchellsimoens]

I have not had this issue in any of my apps. How can I reproduce it? Anything that triggers the error?

[troels]

Hello Mitchell and thanks for answering,

The conditions for triggering this error is fairly common, so I didn't find a detailed scenario necessary. In my experience, any scrollable component has this issue.

I can reproduce the error by wrapping the TouchTweets app in PhoneGap 2.2 and running it on the HTC Sensation running Android 4.0.3.

To trigger the error, I do a two finger drag on the scrollable component (both lists will do it). The movement is a drag, not pinch or rotation, but I'm not sure sure that plays a role. What is important is, that the two fingers has to hit the screen at the exact same time, if one hits before the other, it won't fail.

Mind you that doing the exact same thing, also on the HTC Sensation running Android 2.3 will not trigger the error (yes, I have two of those).

[troels]

I have implemented a working fix for this issue. Using the code:

var orgTouchMoveMethod = Ext.event.recognizer.Pinch.onTouchMove;
Ext.define('app.overrides.Pinch', {
    override: 'Ext.event.recognizer.Pinch',


    onTouchMove: function(e){
        if (!this.isTracking) {
            return;
        }


        if(e.touches.length < 2){
            // the framework is tracking multi touch with only one point
            this.end();
        }
        else{
            orgTouchMoveMethod.apply(this, arguments);
        }
    }
});

the error state is never entered.

I still think this fix or similar should be implemented in the framework.

[ingo.hefti]

In ST 2.1 on Android 4, I'm seeing an issue when doing multi touch that causes my app to go into an error state, where no touch events are propagated to any components. i.e. the app completely freezes.

What version of Android 4? A 4.0.n or a newer one like 4.1.n or 4.2.n?

[troels]

I wrote this in my second post. Confirmed on HTC Sensation running Android 4.0.3

[ingo.hefti]

Sorry, you are right...

But same for me. I noticed that on Android 4.0.n only. So far no problem on my Nexus Galaxy with 4.2.1.

[troels]

It appears that this was triggered by ADB on a general level... sometimes (!).

What happens is, it interrupts touch events in some way. If the phone goes in this mode, doing drags in HTC Sense are also interrupted which UI wise looks exactly like the drag was released. However it looks like the native apps are better at catching this error and are responsive on the next touch (after my fingers actually leaves the screen). The problem with ST is, it stays in "drag mode" and expects the next events to take the drag somewhere.

Pulling the USB plug and thereby stopping ADB from mingling instantly fixes the error.

[gunjan.it2k4]

Me also faced exactly same issue. Good thing is that this is not there in sencha 2.0 with any phonegap version , So moving back to Sencha Touch 2.0.

[troels]

The fix for this in Sencha 2.1 was more complicated than the first code i posted. It seems to happen on all Android 4 phones i've tried, whenever using multi touch. What actually happens is, when releasing a multi touch, one of the released points is never reported to the Sencha event processor. I expect this is an error in Android, like many others. The weird behavior is because Sencha thinks you're still holding one of your fingers on the screen.

I solved it by first disabling all event processors i didn't need.

Ext.application({
    name: 'app',
    // disable all the below recognizers
    eventPublishers: {
        touchGesture: {
            recognizers: {
                rotate: null,
                doubleTap: null,
                longPress: null,
                swipe: null
            }
        }
    },

    ...

})

Then overriding the touch event processor so it's able to reset itself

Ext.define('app.overrides.TouchGesture', {
    override: 'Ext.event.publisher.TouchGesture',


    reset: function(e){
        if(Ext.os.is.Android4 && this.currentTouchesCount > 0){
            e.changedTouches = Ext.Object.getValues(this.currentTouches);
            this.onTouchEnd(e);
        }
    }
});

Then detect when the error occurs by overriding the Pinch detector

window.orgPinchEndMethod = Ext.event.recognizer.Pinch.prototype.end;
Ext.define('app.overrides.Pinch', {
    override: 'Ext.event.recognizer.Pinch',


    end: function(e){
        var wasTracking = this.isTracking,
            result = window.orgPinchEndMethod.apply(this, arguments);
        if(wasTracking){
            this._resetDetection(e);
        }
        return result;
    },


    _resetDetection: function(e){
        // At this point, the pinch event is tracking meaning there should be two fingers on the screen.
        // There is only one active touch event tho, meaning that Android has not propagated the touch end event,
        // which would end the tracking. This now has to be done manually through this nasty hack
        var tg = Ext.event.Dispatcher.getInstance().getPublishers().touchGesture;
        setTimeout(function(){
            tg.reset(e);
        }, 0);
    }
});

Mind that this solution uses one touch to fix itself when the error occurs.