Sencha Inc. | HTML5 Apps

Blog

Motorola Xoom: The HTML5 Developer Scorecard

February 24, 2011 | Aditya Bansod

We posted our last HTML5 developer scorecard in December when the Samsung Galaxy Tab was released. At that time it was the first Android tablet device, running Android 2.2. A few months later we have the Motorola Xoom, the first Android 3.0 / Honeycomb device on the market, which is being touted as the first true Android tablet. Like we did with the Samsung Galaxy Tab we’re going to put the Xoom through the wringer, focusing on the browser to see how it performs and behaves for the mobile HTML5 developer. The short answer? The Xoom browser is not ready for prime-time – even for “HTML4” – and it urgently needs a patch update if Motorola wants the product to succeed.

The Xoom browser is not ready for prime-time, even for HTML4.”

Our First Look “Methodology”

As usual we’re going to test the Motorola Xoom’s browser in four areas: standards using Acid3, features using Modernizr, performance using SunSpider, and a set of real world tests.

Acid3 on the Motorola Xoom

Acid3 and Modernizr

The Xoom scores an impressive 100/100 on the Acid3 tests. This is the first Android tablet that has given a 100 score, which means a couple of things have gotten better, most notably the addition of SVG support. But to understand the true results of an Acid3 test, we have to look beyond the number and compare the final result with the Acid3 reference results, and the shine starts to wear off. The Xoom has two noticeable rendering bugs — first, the letters “Acid3” are the wrong color and are missing the drop shadow. Second, in the top right there’s a small red box, which is an obvious rendering bug. The Xoom has a perfect numeric score, but it still fails Acid3.

We move to Modernizr, a great tool to see what a specific browser supports. The Xoom moves Android a long way forward. Many features that were not supported in the Galaxy Tab are now supported. Modernizr detects a fairly complete range of HTML5 features, including SVG, Inline SVG and CSS3 3D transformations. There are still features lacking such as WebGL support and Web Sockets and Web Workers. We’re excited to see Google pushing Android forward and working to modernize the Android browser, but as we’ll see later that just because something is present, doesn’t mean it works.

Performance Testing

The Xoom’s browser is a much newer version than the browser that shipped in earlier Android based tablets. Under the hood, the browser’s User-Agent reports WebKit version 534.13 which is very recent. The Motorola Xoom boasts some serious hardware firepower, shipping the Nvidia Tegra 2, which is a dual-core 1GHz mobile chip. The new WebKit plus serious CPU power led us to expect that on the SunSpider tests the Xoom would perform well and it did. We compared it against the two other devices we’ve tested and the Xoom easily comes out ahead. The Xoom is indexed at 1, so all the results are in comparison to the Xoom’s results.

Motorola Xoom SunSpider results

On the other side of performance testing, we also ran real world tests using our own CSS3 vs. Flash Ads page. As has been well publicized in the press, the Xoom does not ship with Flash so we were unable to see any Flash ads loading on our tests page. To make things doubly worse, the CSS3 animations are almost completely broken. We often found even for the most basic animations the browser skipped frames, incorrectly rendered elements, or didn’t run the animation to completion. If Animations were simply slow, that would be one thing, but the Xoom CSS3 Animation support faces basic correctness issues.

Moving to more complex animations, we next look at the more advanced CSS3 animations created by Sencha Animator. In an improvement from the Galaxy Tab, the animations actually render. On the other hand, again, they render incorrectly. We found that text sometimes doesn’t appear, parts of the artwork are clipped incorrectly, fonts are rendered poorly, and frames of the animation are dropped. For some of the 3D effects, the browser simply drops the 3D or tries and fails to render the effects. For anything but the most basic CSS transitions and animations the Xoom does not make the grade.

It’s been nearly a year since Apple shipped the iPad and we’re still incredibly surprised that Google and Motorola have yet to build a mobile browser that has a correct and high-performance CSS3 implementation. The silicon power is definitely present to support it.

Unlike on the Galaxy Tab, Modernizr says the browser has SVG and in fact we are able to successfully test SVG. It performs reasonably well, allowing you scale the vector artwork quickly. We next look at Canvas by checking out a github network graph. Like on the Galaxy Tab the graph renders perfectly, and reasonably quickly. We did find that interacting with Canvas elements didn’t always work how we thought it would. The browser often starts to make big blue selection rectangles instead of passing a click event to the DOM. We also checked out Canvas Cycle on the device and we’re surprised that it works. The framerate isn’t great but it does actually work. So generally speaking, Canvas support gets a gentleman’s C.

Finally we test embedded HTML5 audio and video. We are able to get HTML5 audio to work, although we find that sometimes the audio plays even after we left the page or even closed the browser. We were unable to get HTML5 video to work at all.

Sencha Touch Kitchen Sink

As you might expect from the preceding tests, most of the Sencha Touch Kitchen Sink work, and the UI is smoother than the Galaxy Tab but it’s not a fantastic experience. The browser still has a long way to go accelerating CSS3 transforms.

The Motorola Xoom: Suggestions for the HTML5 App Developer

Any HTML5 app developer looking at the Xoom: wait for the inevitable patch(es).

We were excited about the first true Android operating system for tablets and had high hopes for a mobile browser that was as powerful as the platform. Sadly, the Xoom and Honeycomb are a real disappointment. We found consistent and reproducible issues in CSS3 Animations and CSS3 Transitions among other things. We had issues where the browser either hung or crashed. Regular scrolling was slow or below full framerate. We had issues where media playback failed or performed incorrectly. At times it felt like we were using a preproduction device, but we bought our test device from a Verizon Wireless store.

And beyond the HTML5 features, there were many more mundane web rendering issues: form element borders disappearing unpredictably at various zoom resolutions, CSS border radii with flattened edges, the accelerometer object being upside-down, the virtual keyboard causing layout bugs etc. Hopefully the patches will come quickly.

Until Google and Motorola ship a patch to update the browser to production quality, don’t expect good results from the Motorola Xoom. We said it in the Galaxy Tab review, and we’ll say it again now: we’re still waiting for the first awesome Android tablet. Next week we’ll do a sneak peak at HTML5 features in the RIM BlackBerry Playbook — but we’ll give you a one word prequel: “awesomeness”.

There are 31 responses. Add yours.

Tof

4 years ago

This is as very good analysis, but very sad news.

Let’s hope it will get better in time.

Chad

4 years ago

Does it support scrollable divs and “contenteditable” (both of which are required by rich text editors like TinyMCE)?

Kai

4 years ago

Aditya, how does fixed positioning with CSS work out?

Michael Mullany Sencha Employee

4 years ago

Hi Chad - scrollable divs are supported (one finger scroll - no scroll bar indicator though) and contenteditable works but the highlighting is a bit buggy.

Aditya Bansod

4 years ago

@Kai—Yes, it does. When scrolling the div doesn’t move until the page stops scrolling, but as soon as it does, the div shows up where it’s supposed to be positioned.

Chad

4 years ago

That’s great to hear Michael! Seems like things are trending in the right direction…

Ethan

4 years ago

Does Sencha feel that the iPad has decent canvas, html/js, and svg performance?

I went to the Apple store and loaded up the tests from:

http://themaninblue.com/writing/perspective/2010/03/22/

And at best i was seeing 5 fps. Maybe it’s okay for simple animation but anything advanced brings mobile safari to it’s knees and beats it sensless. You can’t do much with 5fps.

Then I see the performance of Flash Player 10.2 on Android and it’s impressive to say the least.

If Apple wants to promote “HTML5” They need to have decent performance-not just “official support.” As an animator/multimedia developer I see huge issue in the iPad in this area, unless i go lock myself up with an Obj-C codebase and app store deployment.

Michael Mullany

4 years ago

@Ethan - Canvas on the iPad is ok, but not great, solid B - in our opinion you wouldn’t want to build anything complex with Canvas on iPad v1. Even iPhone 4 can handle a max of about 25 concurrent sprites in Canvas. Static SVG is ok on the iPad, but dynamic/animated SVG can have tearing issues. On the other hand CSS animations are stellar and CSS3 styling effects render correctly. General HTML/JS performance is above the bar on iPad.

Steven Roussey

4 years ago

How do the animations run if done via JavaScript instead of hoping for CSS3 hardware acceleration?

Michael Mullany

4 years ago

Javascript animations won’t save you, they’re also very choppy.

Henry

4 years ago

Wow, based on this feedback I’m going to hold off buying this device.  Thanks for saving me 800+ USD

Ryan

4 years ago

Ok, so I don’t have any fancy tests or charts to prove myself, but I just don’t see why your xoom performed so poorly. I’ve never had a render issue, the scrolling is top notch, and its never lagged. I’m not saying you are lying or anything, I’m just saying you might have a faulty tablet or there is something wrong. Mine is nice and fast, the only issue I have is the lack of flash.

Michael Mullany

4 years ago

Well Ryan, all the above tests are available at public URLs, so I’d encourage you to duplicate them. It’s always possible we have a bad Xoom. It would be nice if that were the case.

Tom Blackmore

4 years ago

Thanks for a nice article. I really hope they patch soon. Did you test the device api ? Would be interesting to know if those new features are buggy too?

Caroline

4 years ago

I am not native english speaking, though I do really well, but what does “awesomeness” mean?

Aditya Bansod

4 years ago

@Tom—we didn’t test the native Android SDK, but from what we can tell it seems to work very well. The native apps all work well. We did test the HTML5 accelerometer API and it works, although in some orientations the gravity is backwards.

@Caroline—great or amazing. grin

Alex

4 years ago

I can confirm that the Xoom’s Android’s browser is nowhere near production ready.  We pre-ordered 10 units at best buy and made the mistake of opening all of them in our excitement.  The browser is by far the worst application on Andriod 3.0. 

As with most organizations, we bought them to distribute to our sales team.  Unfortunately the Xoom has a very unstable browser.  All of our apps are web apps and we don’t know if we should return these tablets, or hold on and hope we get a better browser for Andriod soon.  Anyone from Google reading this?  I have 10 of these dust collectors, and I’m being pressured to return them.  If we do return them, does anyone from Sencha have the inside scoop on whether we should we wait for the iPad2?  Or is the Playbook a true contender?

I’m so aggravated at the browser on Honeycomb.  It’s definitely damaged my reputation with my sales peers who were pushing for iPads.  I hope someone from Google makes a point to address this quickly.

Caroline

4 years ago

Waiting for the ipad2 you can hardly call waiting… Wednesday it’s released and virtually immediately buyable.
Sorry that this is not in favor of google or Android, but for now this is reality.
Personally I think that Android will close the gap, but when….
Until then you may also switch to the awesomeness wink of the Playbook, what a name for a company oriented tablet…

Rob

3 years ago

...and as a result of yesterday’s launch of the iPad 2, another mortal blow to Android Honeycomb tablets, at least for another year.  I have to agree with the general sentiment of serious disappointment that the Google Honeycomb browser guys really messed up, if these findings by Sencha are anything to go by.  So for now, the iPad has this territory to itself.  Thanks, Sencha, for carrying out this research!

Adriana

3 years ago

The browser of the Motorola xoom crashes regularly. It’s annoying and I am thinking about paying the restocking fee to Verizon and get the ipad2 or the playbook. I was wondering if it was my unit that was bad but now I see that the browser has real and serious problems.

Sean Wilson

3 years ago

Well, at least it has Flash.

Oh… right.

dagamer34

3 years ago

So in other words, the Xoom was rushed to market.

Rod Christiansen

3 years ago

What about iPad running iOS 4.3? It has a much more improved JavaScript engine now. Not to mention how iPad 2 will score even better.

Aditya Bansod

3 years ago

@Rod—we’ll be testing the iPad 2 as a whole when it’s available. We do expect the SunSpider marks to go up with the new JIT for for iOS 4.3, too.

Jay

3 years ago

You can always go get Firefox 4 for Android on the Xoom!

Michael Heller

3 years ago

Why does this whole article use the term “Xoom browser”? It’s not the Xoom’s browser, it’s Honeycomb. This will be the same browser on all Honeycomb tablets. Motorola isn’t responsible for issuing a patch to bring it up to snuff, because it’s Google’s product.

Michael Mullany

3 years ago

@michael - the Android team builds the browser, but it’s up to the OEM when they take the snapshot. The version of Honeycomb on show at the Android stands at MWC had a better browser than this, so we believe that Motorola didn’t pull from top of trunk for the Xoom. It’s also up to the carrier and OEM when they upgrade and push out patches. AT&T, for example is still on a fall build of RIM OS6 for the Blackberry Torch - since the patches roll out only after extensive validation.

Jill

3 years ago

Just don’t try to buy the Xoom from Motorola.com.  I tried to on 2-28-11 - was sent emails saying it was shipped - but no tracking numbers.  After several phone calls and days a supervisor called me back saying it would be delivered on 3-3-11 or 3-4-11.  On 3-3-11 I called again for a tracking number.  After navigating thru automated hell and 6 people and an hour and 45 minutes I was finally told that it hadn’t been shipped… I tried to cancel my order.. but told I couldn’t.  I wrote a complaint and 3 days later a customer service person called me and said she would send all the accessories I ordered to me for free as an apology.  Yesterday I got an email that said they were shipping a gel protector (not something I ordered) but nothing else.  So Motorola.com = liars
I went to Verizion bought the Xoom and reec’d a $200 discount, and signed a two year contract.  Walked out with a fully operational xoom.  Buy one.. it’s a wonderful product.. but Motorola has light years to go regarding their customer service!

Jill

Gabriel

3 years ago

@Caroline ..I waited a lot for iPad 2 as well… Finally it came in Romania.. and people buys it like crazy!

Peerless

3 years ago

God help me, I put aside a whole afetrnoon to figure this out.

Suzy

3 years ago

I had no idea how to arppoach this before-now I’m locked and loaded.

Comments are Gravatar enabled. Your email address will not be shown.

Commenting is not available in this channel entry.