HTML5 Scorecard: Amazon Kindle Fire

As part of our continuing series, we ran the new Amazon Kindle Fire through our tests to see how it stacks up as an HTML5 app platform.

When the Kindle Fire was announced, we were excited about getting our hands on it. As the first mass-market tablet at the $200 price point, we knew it had a good shot at selling millions of units. But at the same time, we were apprehensive that Amazon might have skimped on hardware capabilities to reach that price. We were also concerned about the browser platform. The Fire runs a customized version of Android 2.3.4 (Gingerbread) and in the past, we’ve been disappointed with the quality and completeness of that browser. We were hoping that Amazon would improve the stock Gingerbread browser significantly.

“The Kindle Fire is a competent but minimal HTML5 platform that reflects its $200 price and Android 2.x lineage.”

After putting the Kindle Fire through our test wringer, we can say that while it’s a solid browser for normal page browsing, it lags the best of the competition in HTML5 capabilities. Constrained by its Gingerbread foundation, it’s a competent but minimal HTML5 platform that reflects its $200 price point. At this stage in the tablet game, we would expect better.

Our First Look “Methodology”

Our HTML5 scorecard consists of a series of tests aimed to help mobile web developers understand new devices and new form factors as they come to market. We test in a number of areas, namely JavaScript performance, HTML5/CSS3 features, rendering performance, and rendering accuracy. To get there, we use a variety of tests, including Modernizer, Acid3, SunSpider, Sencha Animator demos, and our Sencha Touch Kitchen Sink.

Acid3 and Modernizr

The Fire scores a less than perfect 95/100 on the Acid3 test. Like other Android tablets, it lacks SVG support and incorrectly displays the Acid3 visual test compared to the reference implementation. With both the iPad2 and the BlackBerry PlayBook now scoring 100/100, this puts the Fire behind the pack. In addition. the Fire failed test 46 (media queries) and came in slower than required on three other tests including the self-described “challenging” test 26 for garbage collection speed.

We then turned to Modernizr, one of our favorite tools to look under the hood of a browser. Since the Fire hardware lacks a camera, an accelerometer and a GPS, some of the results (geolocation failure!) are predictable. What Modernizr did find was support for CSS 2D transforms and Canvas as well as other Android 2.x capabilities. Notably absent: 3D transforms, web sockets, web workers and many HTML form input types. Again, not that surprising considering the Android 2.x heritage.

Happily, web fonts seem to be well supported. Both Google Fonts and Typekit dynamic fonts render correctly. But Typekit font loading and page scrolling on the Google Fonts page (with tens of different onscreen web fonts) was noticeably slow.

Performance Testing

When checking the user agent received, web servers report that our Fire was using WebKit version 533.1, which is much older than the WebKit found on the Xoom and PlayBook. On other hand, when the Kindle is put in “desktop browsing mode”, it advertises itself as Safari 5/Mac OS X/Webkit 533.16. Under the hood, according to the iFixit teardown, the Fire contains a dual-core ARM processor, specifically the TI OMAP 4430, which is the same processor as the BlackBerry PlayBook. We, therefore, expected SunSpider scores in the same range as other tablets. As we’ve mentioned before, this current generation of tablets all ship with dual-core processors, and all have roughly the same JavaScript optimizations. So the results are again fairly predictable.

For the most part, the Amazon Kindle Fire has similar SunSpider results compared to Apple iPad 2 and BlackBerry PlayBook.

The SunSpider tests are synthetic tests that push the JavaScript engine to its limit. So, next, we turned our attention to some more real-world tests, looking at CSS3 performance using our own Sencha Animator demos as well as a few other tests.

We threw caution to the wind and started off on our most challenging CSS3 animation test, the Kick Fu game that was developed for the launch of Sencha Animator. While the game did play, the frame rate was poor, and touch responsiveness while animations were running was also substandard. The Fire lacks a separate GPU, but the CPU has a competent GPU core – and even so, it looks like it wasn’t leveraged for hardware acceleration. The Playbook, for example, does a far better job leveraging the same GPU core.

Next, we tried simpler animations. The Fire performed more smoothly but had a notably bad implementation of timing. In the video below, the wave movements are controlled by CSS3 animations and are declared at a constant speed. Instead, the animation speeds of each element diverge and lag noticeably and visibly from each other. We’ve never seen artifacts of this magnitude before. This got us wondering about the Kindle’s general timer quality, so we ran John Resig’s classic JavaScript timer test and found that the quality of the timer is similar to other Androids: with setInterval triggered every 10ms +5/-3s, with some noticeable latency spikes. (This compares to iOS 5’s best-in-class implementation of 10ms +/-1ms.) This isn’t supposed to matter as much once people get used to using requestAnimationFrame for JavaScript animations, but the Fire doesn’t have that either.

We tried a Canvas-based GitHub network graph. Rendering performance was fine and the result was accurate. Pinch/zoom happily also works although Android’s tendency to toggle into low-quality mode when moving content reared its head once again. We also tried out Canvas Cycle and it worked reasonably well. The frame rate was solid and we were able to change the scene without issue. However, like all webkit implementations, all bets were off when we zoomed and panned the page. Animation stuttered, froze and occasionally the accompanying audio cut out. We also looked at other Canvas animations and they worked at reasonable frame-rates.

Finally we tested embedded HTML5 audio and video. HTML5 audio playback seemed to work well, although the default playback controls were tiny, hard to touch and didn’t seem to work very well. HTML5 video is “supported’ but tapping play simply launches the video in the native player. Complete HTML5 video support with embedded playback and effects is not there. There also appears to be no Ogg Theora codec on the device as we couldn’t get any Ogg Theora demo to play.

“The Kindle Fire doesn’t seem designed to run HTML5 apps as a primary goal.”

Sencha Touch Kitchen Sink

Our Kitchen Sink app is built with Sencha Touch and tests all aspects of our mobile web app framework. The Kitchen Sink gives mobile browsers a good workout as it exercises a huge set of browser features including a wide range of CSS3 effects.

The first thing we noticed was the crispy CSS rounded corners. They’re not anti-aliased properly on the Fire. Webkit masks are also not supported. Then we got the first really big surprise of our run-through. We found that the Kindle Fire has problems processing touch events with good responsiveness. Update: and we’re not the only reviewers to notice this. And similar to early HTC phones (since fixed), the OS and browser seem to fight over who gets touch events. Sencha Touch 2 was designed to adapt to this behavior, so the Touch 2 preview kitchen sink works better than the Touch 1 version. And of course, since the Fire is based on Android 2.x, full multi-touch with independently tracked touches is not supported either.

As you’d expect with a device lacking a GPU, graphics effects like transitions were less than smooth, although display artifacts were not as bad as the full-screen flashes and repaints that are a problem on honeycomb-based tablets.

No to low Impact from “Accelerated Browsing”

One of the main selling points of the Kindle browser is supposed to be its cloud-caching and pipelined HTTP connection that uses the SPDY protocol. This does seem to speed up normal page browsing a little, but it’s not very noticeable and we didn’t test this rigorously. But for HTML5 web apps, where code is downloaded and executed, there doesn’t seem to be any performance difference when we tested with acceleration on and off. It doesn’t appear as if client JavaScript is executed on the server-side at all, so the Kindle does not seem to have Opera Mini-style server-side execution. And SunSpider scores were essentially the same when accelerated browsing was turned on or off.

Kindle Fire: Suggestions for the HTML5 App Developer

In summary, the Amazon Kindle Fire doesn’t seem designed to run HTML5 apps as a primary goal. It does a good job of displaying ordinary web pages and its resolution and rendering capabilities meet that need well. But there are too many sharp edges, performance issues, and missing HTML5 features for us to recommend that any developer create web apps primarily for the Kindle Fire. The iPad 2 running iOS 5 continues to be the tablet to beat, with the PlayBook a respectable runner-up in HTML5 capabilities.