Sencha Inc. | HTML5 Apps

Blog

5 Myths About Mobile Web Performance

August 01, 2013 | Michael Mullany

Recently we’ve heard some myths being repeated about mobile HTML performance that are not all that accurate. Like good urban myths, they sound compelling and plausible. But these myths are based on incorrect premises, misconceptions about the relationship between native and web software stacks and a scattershot of skewed data points. We thought it was important to address these myths with data that we’ve collected over the years about performance, and our own experiences doing optimizations of mobile web app performance.

TL;DR

Myth #1: Mobile web performance is mostly driven by JavaScript performance on the CPU

Reality: Most web performance is constrained by the optimization of rendering pipelines, speed of DOM interactions and degree of GPU acceleration. Faster JavaScript is always helpful, but it’s rarely critical.

Myth #2: CPU-Bound JavaScript has only become faster because of hardware improvements (aka Moore’s Law)

Reality: 50% of mobile JavaScript performance gains over the last four years has come from software improvements, not hardware improvements. Even single-threaded JavaScript performance is still improving, without considering that most app devs are using Web Workers to take advantage of multi-threading where possible.

Myth #3: Mobile browsers are already fully optimized, and have not improved much recently

Reality: Every mobile browser has a feature area where it outperforms other browsers by a factor of 10–40x. The Surface outperforms the iPhone on SVG by 30x. The iPhone outperforms the Surface on DOM interaction by 10x. There is significant room to further improve purely from matching current best competitive performance.

Myth #4: Future hardware improvements are unlikely to translate into performance gains for web apps

Reality: Every hardware generation in the last three years has delivered significant JavaScript performance gains. Single thread performance on mobile will continue to improve, and browser makers will advance the software platform to take advantage of improved GPU speeds, faster memory buses & multi-core through offloading and multi-threading. Many browsers already take advantage of parallellism to offload the main UI thread, for example: Firefox offloads compositing; Chrome offloads some HTML parsing; and IE offloads JavaScript JIT compiliation.

Myth #5: JavaScript garbage collection is a performance killer for mobile apps

Reality: This is trueish but a little outdated. Chrome has had an incremental garbage collector since Chrome 17 in 2011. Firefox has had it since last year. This reduced GC pauses from 200ms down to 10ms — or a frame drop vs, a noticeable pause. Avoiding garbage collection events can deliver significant improvements to performance, but it’s usually only a killer if you’re using desktop web dev patterns in the first place or on old browsers. One of the core techniques in Fastbook, our mobile HTML5 Facebook clone, is to re-cycle a pool of DOM nodes to avoid the overhead of creating new ones (and the associated overhead of GC collections on old ones). It’s very possible to write a bad garbage collector (cf. old Internet Explorers) but there’s nothing intrinsically limiting about a garbage collected language like JavaScript (or Java).

Ok: The Essentials

First, let’s remind ourselves of the basics. From 50,000 feet, the browser is a rich and complex abstraction layer that runs on top of an Operating System. A combination of markup, JavaScript, and style-sheets, uses that abstraction layer to create application experiences. This abstraction layer charges a performance tax, and the tax varies depending on what parts of the abstraction layer that you’re using. Some parts of the abstraction layer are faster because they match the underlying OS calls or system libraries closely (aka Canvas2D on MacOS). Some parts of the abstraction layer are slower because they don’t have any close analog in the underlying OS, and they’re inherently complicated (DOM tree manipulation, prototype chain walking).

At Sencha, we know that a good developer can create app experiences that are more than fast enough to meet user expectations using mobile web technologies and a modern framework like Sencha Touch.”

Very few mobile apps are compute intensive: no-one is trying to do DNA sequencing on an iPhone. Most apps have a very reasonable response model. The user does something, then the app responds visually with immediacy at 30 frames per second or more, and completes a task in a few hundred milliseconds. As long as an app meets this user goal, it doesn’t matter how big an abstraction layer it has to go through to get to silicon. This is one of the red herrings we’d like to cook and eat.

At Sencha, we know that a good developer can create app experiences that are more than fast enough to meet user expectations using mobile web technologies and a modern framework like Sencha Touch. And we’ve been encouraged by the performance trends on mobile over the last 3 years. We’d like to share that data with you in the rest of the post.

It’s not our intention to claim that mobile web apps are always as fast as native, or that they are always comparable in performance to desktop web apps. That wouldn’t be true. Mobile hardware is between 5x and 10x slower than desktop hardware: the CPU’s are less powerful, the cache hierarchy is flatter and the network connection is higher latency. And any layer of software abstraction (like the browser) can charge a tax. This is not just a problem for web developers. Native iOS developers can tell you that iOS CoreGraphics was too slow on the first Retina iPad, forcing many of them to write directly to OpenGL.

Digging Deeper Into the Myths

Having worked for three years on optimization in Sencha Touch for data driven applications, we can say with confidence that we’ve rarely been stymied by raw JavaScript performance. The only significant case so far has been the Sencha Touch layout system, which we switched over to Flexbox after finding JavaScript to be too slow on Android 2.x.

Far more often, we’ve run into problems with DOM interactions, the browser rendering engine and event spam. All of these are limitations created by the architects and developers of each browser, and nothing to do with the inherent characteristics of JavaScript or JavaScript engines. As one example, when we’ve worked with browser makers on performance optimization, we’ve seen a 40x improvement in one of the browser operations (color property changes) that bottleneck our scrolling list implementation, as just one example.

JavaScript Performance on iOS & Android

Although we’ve said JavaScript performance isn’t that important on mobile, we want to demolish the myth that it’s not improving. Below is a graph showing a four year history of SunSpider scores on iOS (lower is better) by model and by iOS version. (Luckily, SunSpider is a very widely used test, so there are records all over the web of testing performed on older iOS versions.) In 2009, the original iPhone 3GS running iOS3 had a score of over 15,000 — quite a poor performance, and about a 30x performance gap vs. the 300–600ish score of desktop browsers in 2009.

5 Myths about Mobile Web Performance

However, if you upgraded that iPhone 3GS to iOS4, 5 and 6, you’d have experienced a 4x improvement in JavaScript performance on the same exact hardware. (The big jump in performance between iOS 4 and iOS5 is the Nitro engine.) The continual gains in SunSpider continue in iOS7, but we’re still under NDA about those. Compared with today’s desktop browsers, edge mobile browsers are now about 5x slower — a big relative gain vs. 2009’s 30x gap.

For even more detail on hardware and software advances in iOS hardware, see Anandtech’s review from last October.

There have been similar levels of improvement in the Android platform. From our testing lab, we assembled a collection of Android platforms from the last three years that we believe represent typical high end performance from their time. The four phones we tested were:

  • Samsung Captivate Android 2.2 (Released July 2010)
  • Droid Bionic Android 2.3.4 (Released September 2011)
  • Samsung Galaxy Note 2 Android 4.1.2 (Released September 2012)
  • Samsung Galaxy S4 Android 4.2.2 (Released April 2013)

As you can see below, there has also been a dramatic improvement in SunSpider scores over the last four years. The performance jump from Android 2.x to Android 4.x is a 3x improvement.

5 Myths about Mobile Web Performance

In both cases, the improvement is better than what we’d expect from Moore’s Law alone. Over 3 years, we’d expect a 4x improvement (2x every 18 months), so software is definitely contributing to the improvement in performance.

Testing What Matters

As we previously mentioned, SunSpider has become a less interesting benchmark because it only has a faint connection with the performance of most applications. In contrast, DOM Interaction benchmarks as well as Canvas and SVG benchmarks can tell us a lot more about user experience. (Ideally, we would also look at measurements of CSS property changes as well as the frames per second of css animations, transitions and transforms — because these are frequently used in web apps — but there are still no hooks to measure all of these correctly on mobile.)

First DOM interaction tests: we used Dromaeo Core DOM tests as our benchmark. Below are the results from testing on our 4 Android models. We indexed all Core DOM results (Attributes, Modifications, Query, Traversal) to the Captivate’s performance, and then took the average of the 4 Core DOM indices.

5 Myths about Mobile Web Performance

As can be seen, there was a 3.5x performance improvement from Android 2.x to 4.x, although the S4 was only a minor improvement on the Note 2. We can also look at Dromaeo results on iOS. Sadly, we can’t do a comparison of performance across older versions of iOS, but we can show the steady improvement across multiple generations of iPhone hardware. Interestingly, the performance improvement even within iOS6 across these device generations is better than the increase in CPU speed, which means that improvements in memory bandwidth or caching must be responsible for the better than Moore’s Law performance increase.

5 Myths about Mobile Web Performance

In order to show that there is still a lot of potential for browsers to match each other’s performance, we also benchmarked the Surface RT. The poor performance of DOM interactions in IE has always been a source of performance frustration, but it’s worth pointing once again the large performance gap in DOM interaction performance on the iPhone vs. the Microsoft Surface RT running IE10. One of the myths we’d like to destroy is that the mobile software stack is as good as it gets. This is not true for Windows RT- a 10x performance gap is waiting to be filled. (We’ll show where iOS falls down in a later benchmark.)

Graphics Benchmarks

In addition to significant improvements in JavaScript and DOM interaction, we also wanted to show improvements in Canvas and SVG Performance. In iOS5, we previously found that Canvas2D performance increased by 5–8x on the same hardware vs. iOS4. Some micro-benchmarks ran 80x faster on the iPad 2 when upgraded to iOS5. And because Canvas is closely bound to CoreGraphics on the iPhone, when native graphics gets faster, so does Canvas. In this round of testing, we used the mindcat Canvas2D benchmark to measure performance. Here, we see a substantial increase in Canvas performance on multiple generations of iPhone hardware — all running the same generation of iOS.

5 Myths about Mobile Web Performance

And remember, this is on top of a big performance gain from iOS4 to iOS5. As can be seen, the performance improvement in Canvas2D performance (7x) is better than the CPU improvement over the same time period (4x) and reflects the improved GPU and GPU software stack on the iPhone over time. GPU improvements and the increased use of GPU acceleration needs to be factored in to the increase in mobile web performance improvement.

5 Myths about Mobile Web Performance

When we look at the same tests on Android, we see an interesting data point — a lack of correlation between CPU performance and Canvas. The big change from Android 2 is that Android 2 had no GPU acceleration of Canvas at all. Again, a purely software change — GPU acceleration — is the big driver of performance improvement.

SVG Benchmarks

SVG is another area that shows the broader story about web performance. Although not as popular as Canvas (largely because Canvas has become so fast), SVG has also shown solid performance improvements as hardware has improved. Below is a test from Stephen Bannasch that tests the time taken to draw a 10,000 segment SVG path. Once again, the test shows constant steady improvement across hardware generations as the CPU and GPUs improve (since these are all on iOS6).

5 Myths about Mobile Web Performance

The main gap in performance is in software: the Surface RT is more than 30x faster than the iPhone 5 (or the iPad 4 — we also tested it but haven’t shown it.) In fact, the Surface RT is even 10x faster than Desktop Safari 6 running on my 1 year old Macbook! This is the difference that GPU acceleration makes. Windows 8/IE10 fully offloads SVG to the GPU, and the result is a huge impact. As browser makers continue to offload more graphics operations to the GPU, we can expect similar step-function changes in performance for iOS and Android.

In addition to long path draws, we also ran another SVG test from Cameron Adams that measures the frames per second of 500 bouncing balls. Again, we see constant performance improvement over the last four generations of hardware.

5 Myths about Mobile Web Performance

What’s more interesting than the improvement is the absolute fps. Once animations exceed 30 frames per second, you’ve exceeded the fps of analog cinema (24 fps) and met viewer expectations for performance. At 60 fps, you’re at the buttery smoothness of GPU accelerated quality.

Real World Performance: Garbage Collection, Dynamic Languages and More

We hope that the preceding tour through mobile performance has demonstrated a few things, and demolished a few myths. We hope we’ve shown that:

  • JavaScript performance continues to increase rapidly
  • This performance increase is driven by both hardware and software improvements
  • Although this is a “good thing”, much of app performance has nothing to do with JavaScript CPU performance
  • Luckily, these other parts that impact performance are also improving rapidly, including DOM interaction speed, Canvas and SVG.

And although we could show it with high speed camera tests, all mobile web developers know that performance of animations, transitions and property changes has vastly improved since the days of Android 2.1, and they continue to get better each release.

So having slain a few false dragons, let’s mix metaphors and tilt at some real windmills. The last claim we’ve heard going around is that mobile web apps will always be slow because JavaScript is a dynamic language where garbage collection kills performance. There is definitely some truth in this. One of the benefits of using a framework like Sencha Touch, where DOM content is generated dynamically, is that we manage object creation and destruction as well as events at a level above the browser in the context of a specific UI component. This enables, for example, 60fps performance for data-backed infinite content (grids, lists, carousels) by recycling DOM content, throttling events and prioritizing actions.

Without this level of indirection, it’s very easy to produce poor performing mobile web app experiences — like the first generation mobile web app from Facebook. We believe that applications written on top of UI Frameworks like jQuery Mobile that are too closely tied to the underlying DOM will continue to suffer from performance problems for the foreseeable future.

Bringing It All Together

There is a lot of data and a lot of different topics covered in this piece, so let me try to bring it all together in one place. If you’re a developer, you should take a few things away from this:

  • Mobile platforms are 5x slower than desktop — with slower CPUs, more constrained memory and lower powered GPUs. That won’t change.
  • Mobile JavaScript + mobile DOM access is getting progressively faster, but you should still treat the iPhone 5 as if it’s a Chrome 1.0 browser on a 2008-era desktop (aka 5–10x faster than desktop IE8).
  • Graphics have also been getting much faster thanks to GPU acceleration and general software optimization. 30fps+ is already here for most use cases.
  • Garbage collection and platform rendering constraints can still bite you, and it’s essential to use an abstracted framework like Sencha Touch to get optimal performance.
  • Take advantage of the remote debugging and performance monitoring capabilities that the mobile web platforms: Chrome for Android now has a handly fps counter, and compositing borders can show you when content has been texturized and handed off to the GPU and how many times that texture is then loaded.

We hope that the review of performance data has been a helpful antidote to some of the myths out there. I want to thank everyone at Sencha who contributed to this piece including Ariya Hidayat for review and sourcing lots of links to browser performance optimizations, and Jacky Nguyen for detail on Sencha Touch abstractions and performance optimizations.

There are 11 responses. Add yours.

Jacob G

12 months ago

Thanks for the article. A couple questions:
1) To compare Sencha performance to other javascript libraries, what does the extra DOM that Sencha adds to components to support its layout & theming systems do to overall performance?
2) Apple has neutered native-hosted web views by not running them in Nitro, but it doesn’t look like you considered that. Do you have more to say about native-hosted web views?

Mike

12 months ago

What doesn’t make sense is that people are still working really hard to make software run faster in the desktop browser.  That’s why Google is working on Dart, NaCl, and PNaCl.  In addition, Mozilla is working on asm.js.  So, if desktop browser software isn’t fast enough, wouldn’t it hold that mobile software isn’t fast enough?  Certainly there are some cases where HTML5, mobile and desktop, have perfectly acceptable performance. 

The question that nobody seems to be able to answer with mobile is where is the current limit.

Michael Mullany

12 months ago

@Mike. Frankly we’d a little mystified about why Google is putting so much emphasis on next gen fastness for desktop class machines. It feels like engineers tackling “hard engineering problems” rather than actually working on making real world apps faster. The fact that there are so many people at Google workng on Dart, while there is a bunch of work to be done on the real bottlenecks is very confusing. But that’s the way Google works.

@ Jacob. The reason why we have two frameworks is that there is a much smaller performance envelope to work with on mobile, and you have to design the internals differently. As I mention in the post, we couldn’t afford the performance hit of Ext JS style layouts in Touch, so all you get are what you can do with Flexbox (no docked layout, for example). On Ext JS, there is a *lot* of optimization in 4.1 and 4.2 to accommodate the new flexible layouts and class system that we put into 4.. There is no other JavaScript framework that’s as focused on themable pixel-perfect rendering cross browser (including IE6-8) as we are, and you need a deep DOM to do that.

Bob Thulfram

12 months ago

Fascinating stuff, especially considering your expertise. I’m surprised to hear that many app devs are taking advantage of web workers. My own myth is that someday web workers will make web pages more efficient. That someday is now!

Also really, really fascinating to see the SVG figures. For IE come from behind (with no SVG support in 8) to blazingly fast is really encouraging. SVG seems like it is now supported in every browser and can be great for what it does.

Again, thanks for a really great article to encourage people to see that mobile is valid.

Joe T

12 months ago

The thing that’s *really* missing from excellent articles like this ( and Drew Crawford’s “Why mobile web apps are slow” http://sealedabstract.com/rants/why-mobile-web-apps-are-slow/ ) is __what kind of applications they’re talking about__.

If the most computationally intensive part of your mobile web app is getting your medium-complexity UI to display at 30fps, surely even Drew would agree that it’s possible on mobile today. (Perhaps with a bit of help from a decent framework.) But that leaves out large classes of apps that *don’t* bottleneck on UI but on what it’s doing in the background.

Michael, when *you* say that the mobile web is perfectly fast enough, you’re talking about apps like sudoku, photo galleries, product catalogs, running journals etc.

When people say mobile web apps are slow they’re referring to the classes of apps where rendering performance is the least of your worries: sketchbooks, photo/video editing apps, live games (that are 3D (with AI)), etc. In fact *desktop* web apps have issues with these, and *that’s* why google and mozilla are trying to push performance with things like Dart and asm.js.

All I’m saying is authors need to specify which apps they’re talking about explicitly, *and* acknowledge the classes of apps that are outside the scope of their discussion.

Jim Trainor

12 months ago

You missed a very important subject: Javascript software quality. Is it improving? Are developers adopting the engineering practices necessary to produce code that is of sufficient quality to match native applications which have intrinsic benefits because they are developed using languages that lack the intrinsic problems presented by Javascript?  It is very difficult to judge, and even more difficult to measure. But it is really important. The performance is there to build good web apps. Is the ability to produce quality software quality there when applications get big? I think Sencha’s future depends on this question more than anything else. As the saying goes, attributed to a Einstein - a pretty smart guy: not everything that matters can be measured and not everything that can be measured matters.

Michael Mullany

12 months ago

@Joe T - Good point about specifying a class of apps. The two categories of apps that Drew specifically mentioned as inherently slow in JavaScript were spreadsheets and photo editing. Now, it’s been possible to do a fast spreadsheet since Lotus 1-2-3, so 30 years of Moore’s Law is helping us there. Our new Touch Grid for mobile that we showed off at SenchaCon last month, can handle infinite rows at 60 fps, but we’re still working on infinite columns, and we think we need browser help to do that. But again, it’s nothing to do with JavaScript and everything to do with other constraints. As far as photo editing goes, I wouldn’t try it on anything that’s not well-GPU accelerated. We did an SVG filter based editor (filterfactory.mobi) - it’s decently performant on iPad 4, but would benefit enormously from a GPU acclerated SVG filter implementation on iOS.

@Jim T We see some real horror code out there in the wild done with cargo cult JavaScript. We see the need for much better JavaScript programming & built in testing. We recommend Jasmine or Siesta for now, but we’d like more people using Sencha Architect when they get started to see what best practices code looks like, and start using testing tools early early. Ariya Hidayat has been thinking a lot about this area recently.

Jim Trainor

12 months ago

Well, thanks. I’m glad to hear someone is wrapping their head around it. I’ve developed the point of view that web development, and Javascript app dev in particular, is not cpu, memory, graphics, io, bandwidth, or latency bound. Applications that are bound by any of those things are unique and they simply find their right place based on those hard constraints. If fact, the fundamental bound is going to be a quality bound. I’ve not heard the term “quality bound” before but it captures my mood on the subject. It’s just too hard to achieve in a web app using accepted practices of today what is possible in native apps using decades old, very well established, good practice. Yet web-apps are headed in a direction that requires the same reliability as native apps. Further, and perhaps the deeper problem, there appears to be no strong culture of quality in web development - and humans are hard to change. There is no Moore’s law working your favour! This is no greater issue that will separate web apps from native apps going forward. I don’t mean, or intent, to side track the article, so I won’t continue to comment beyond this. Performance issues just don’t resonate with me these days. I’m beyond it. I know that the device will run the code fast enough if I do my job well, and faster still going forward.

 

Nathan Cerny

12 months ago

I am very impressed with the improvements that Sencha has made to allow for infinite scrolling and other performance increases. However, this article just seems to be a marketing ploy to hide the fact that ExtJS layout system is slow. As an ExtJS user there is nothing more frustrating than the performance issues with the ExtJS docs. I thought maybe I was being impatient so I decided to measure the performance. What I found is that when I bring up a new tab in the docs. After the documentation loads it takes around 2 seconds for the new doc to render and layout. What is Ext doing to fix these performance issues?

Michael Mullany

12 months ago

@Nathan Are you trying to bring them up on an iPad? I mention in the post that Ext JS layout system is going to be slow on mobile, it’s a significant difference between sencha touch and ext js - that we DON"T do layout in JavaScript for mobile, it’s not fast enough. A 2 second load time on an iPad is actually pretty good for Ext JS. It’s not designed for mobile first, Sencha Touch is.

Tom Coulton

11 months ago

Hi, we translated this blog into Japanese here:
http://www.xenophy.com/sencha-blog/7986

Also this is the page for the Japanese Sencha Meetup Group:
http://www.meetup.com/Japan-Sencha-User-Group/about/

Comments are Gravatar enabled. Your email address will not be shown.

Commenting is not available in this channel entry.