Sencha Inc. | HTML5 Apps

Blog

UI Testing a Sencha App

January 10, 2013 | Arthur Kay

A few months ago, I wrote a post titled Automating Unit Tests that covered how developers could write unit tests for their business logic and validate their JavaScript syntax. Understanding these concepts is essential when building an enterprise application: bugs must be caught before changes are pushed into production or catastrophic consequences may follow.

The one area that I did not cover in that post was the idea of “UI Testing” (also known as Integration Testing). After reading many of the comments to that post and hearing the community’s feedback, I wanted to address this topic by adding UI Tests to my demo Ext JS application and discussing strategies for enterprise application testing.

UI Testing: An Overview

As I mentioned in my earlier article, UI Tests are not the same thing as Unit Tests—but more often than not these ideas are confused.

UI Tests attempt to subjectively verify that elements on the screen behave (and often look) as expected, both statically (i.e. the flat render) and also dynamically (i.e. as users perform given actions). Unit Tests attempt to isolate small pieces of code and objectively verify application logic.

The key difference here is the subjective vs. objective nature of the tests.

Taking this idea a step further, we can break UI Tests into their constituent parts: QA Tests and Component Tests.

  • QA Tests simulate real-world interactions with your application as if a user is using the app.
  • Component Tests isolate independent (often reusable) pieces of your application to verify their display and behavior.

In this article, we’ll take a look at both types of UI Tests.

Common Problems with UI Testing

Selecting the Right Tool

Testing the look and the complex interactions of an application is an enormous task—and it’s no surprise that many web developers struggle to implement (and often abandon) UI Tests that adequately solve QA problems.

Perhaps the biggest hurdle that developers must address is selecting the best tool for the job. Some tools rely on XPath or CSS selectors to navigate the application; others require complicated server configurations to allow for test automation. At the end of the day, it is vitally important to select a tool that is flexible; QA Tests may be frequently rewritten as business requirements and as the app changes, so it’s important that the tests are easy to build and maintain.

In my experience, I have only found three tools that can address the needs for writing QA Tests against a Sencha application:

Each of these tools has specific pros and cons; having said that I personally think Siesta is the easiest of them to setup, and Siesta’s API works seamlessly with Sencha’s frameworks.

Disclaimer: Other UI testing tools exist, and I do not claim to have used all of them; nor do I claim that Siesta is necessarily the “best” tool in this space. I simply offer an opinion based on my own experiences.

Mocking Data

One other important (and often overlooked) problem with UI Tests is that tests should typically not be written against live APIs. Regardless of the source, APIs are inherently unreliable: servers go down, networks experience latency, and unexpected bugs occur.

The purpose of UI Tests is to verify display and behavior—not that specific data exist at any given time. UI Tests should mock API data when possible, but this is not always an easy problem to solve. Some implementations use static data in place of AJAX requests; others redirect network calls to a mock API.

More than a handful of JavaScript mocking libraries exist, but I am not going to dive too deeply into the subject in this post. I simply want to raise awareness of the issue because it is a common source of frustration. I chose to use Sinon.js to stub the API calls for my sample application.

Having said all of that, you may have situations where you do want to use the live API—but I would normally only recommend that to verify a release is ready to be pushed live.

Sample Application

In the sample Ext JS application, you’ll find a /ui-tests/ folder containing both QA (/app/) and Component (/ux/) tests written with Siesta.

Siesta Example

QA Testing

Under the /app/ folder, you can view the index.html file in your browser to see the Siesta interface with our QA Tests in place. The goal here is to launch the actual application and test the real-world interactions that users would expect. While the sample application is a relatively simple example, the two QA Tests demonstrate various ways in which we can test the behavior of the app as a whole.

The first test, titled “Test tabs for data in grids” (/app/tests/01_tabs.js) simply loads the application and checks to make sure the required views are correctly in place. Although this particular example is rudimentary, this sort of test could be very useful if your application dynamically created its interface based on user roles, preferences, or some other logic.

The second test, titled “Test double-click functionality” (/app/tests/02_RsvpWindow.js) again loads the entire application. This time, we begin to simulate interactions with our grids and tabs to ensure the desired behaviours execute as expected.

Take note of the fact that I use Sinon.js to stub the JsonP requests for these data stores. Doing this allows me to test the functionality of the application without the assumption that the live API is accessible and working properly. There are a number of ways you could accomplish this; I chose to override the behavior of Ext.data.JsonP.request() to automatically return mock data (see /ui-tests/app/api_stub.js).

Component Tests

Under the /ux/ folder, you can view the index.html file in your browser to see the Siesta interface with our Component Tests in place. Unlike our QA Tests, the goal in this case is to isolate individual components and test their behaviors. By testing the components outside of the greater application, we can isolate known bugs and guarantee future compatibility.

The only test I’ve written for this sample application (/apuxp/tests/01_RsvpWindow.js) examines the display and behavior of our RsvpWindow view. This view extends the Ext.window.Window class, and contains a grid with a custom column renderer. Using Siesta, we create an isolated instance of this view and verify the cell renderer behaves as expected.

 
   var defaultWin = Ext.create('ChicagoMeetup.view.RsvpWindow');
 
   //I know there are only 2 rows in this grid because I mocked the API
   var firstRow = Ext.get(grid.getView().getNode(0)),
       secondRow = Ext.get(grid.getView().getNode(1));
 
   var regExp = /^(<img)/g;
   var innerHtml = firstRow.query('.x-grid-cell-inner')[1].innerHTML;
 
   t.is(regExp.test(innerHtml), true, 'First row should contain an image in the second column');
 
   innerHtml = secondRow.query('.x-grid-cell-inner')[1].innerHTML;
   t.is(regExp.test(innerHtml), false, 'Second row should NOT contain an image in the second column');
 

Using Siesta’s robust testing API, we could simulate a variety of other interactions with this component (click, drag, etc). Although the RsvpWindow component isn’t all that exciting, you can imagine the possibilities for your own custom UX classes.

Conclusion

Building unit tests for a web application can be a difficult task, but when done correctly the payoff for your efforts is invaluable. In closing, I would like to reiterate these important points:

  • Unit Tests and UI Tests are not the same thing. Both are valuable ways to maintain stable code, but they solve different problems.
  • Mind your syntax. Just because your code runs correctly in one browser doesn’t mean it will run correctly in every browser.
  • Test your custom components. It’s alright to assume your framework works as expected—but don’t assume your UX is written correctly.
  • Don’t shoot for 100% code coverage. You may want to test the entire application, but beware the cost of maintaining an elaborate test suite.

This series of posts on unit testing is based on my own personal experience helping Sencha’s customers solve common problems. You can learn more in a webinar I'll be hosting January 31st alongside Mats Bryntse that introduces developers to realistic methods for testing your Ext JS and/or Touch applications. Registration is available here. In addition, I invite you to share your own thoughts and experiences below—we can help each other make web application testing an easier goal to achieve.

UPDATE (1/2013): I wrote a follow-up post titled UI Testing a Sencha App which expands upon the content in this article. I have also updated the sample project on GitHub - so there are some slight differences in the examples posted here compared to the updated repo, although the concepts are exactly the same. If you have questions or comments, please start a thread on the Sencha forums and I'll be sure to respond!

There are 9 responses. Add yours.

Fred

2 years ago

Hi,

good post. Totally agree with you. I started with Selenium in the first place. While it is very mature and has a long history it is not suited for sencha touch. Doing simple things like tapping on a button is frustrating. Since the dom is completely generated it is very hard to write the XPath queries.
Then I discovered Siesta and I think its the least painful way testing a Sencha Touch App.

Mats

2 years ago

Hey Fred,

Any ideas for how we can improve Siesta to make it even less painful, and more ‘enjoyable’? smile
We’re always interested in ideas for improvement!

testme

2 years ago

Hy Mats,
I was looking forward to create a sencha application and then test it under automation testing using selenium or as you suggest Siesta.
Being an automation tester, i have a good knowledge of selenium, but not sencha, Can you provide me by an example (as i am totally new to sencha) , how to go about making the app using sencha.
Also, whether i ll be needing any other interface to test it my machine, like a tablet or mobile emulator ?

Thanks.

Mats

2 years ago

@testme: Selenium is not a good fit for Sencha applications due to the dynamic nature of the generated HTML. This is where Siesta does a better job offering you Component Queries instead which are much easier to use to target things in the DOM. Siesta is completely web based so you can launch it in any browser.

Arthur Kay

2 years ago

@all - As several of you pointed out that my original code sample was not a very useful test, I have updated the GitHub repo (and this post) with a better test (the column renderer). Please check this out if you have a chance!

testme

2 years ago

@Mats: Thanks Mats, i get that i should use siesta and i have convinced my automation team to go for it, But,,,
@all:  My assignment actually involves building a tablet sencha application and then testing it,
and i am looking for a example/demo which can help me using develop an tablet application using eclipse (Java) and i can run it on an emulator (android, or any else u prefer.) Now , being new to sencha development, it would be of great help if anyone can tell stepwise how to go for it, what will i need , and how to programme atleast for a demo.
So guys, can you share any link or tell stepwise how to go about with it ; from what are the requirements, to whether i ll be needing what emulator, and how to go about them, and in the end, how to test it on siesta,
@ Arthur Kay
I have looked at your new post , and will be glad if you can help me out in what i posted above.

Arthur Kay

2 years ago

@testme - The problem of physical device testing is yet another facet to the unit testing puzzle. I have yet to see any good emulators for tackling this issue in an automated fashion, though I have heard some rumors of tools coming in the next few months. I’ll be sure to post more details if/when I hear more, but I’m definitely curious if anyone else has found a solution for that issue.

Mario

2 years ago

A seedable random generator is the missing link. One can have a dynamic app produce consistent IDs, for example, by loading a random number generator in Javascript. Take a look at http://davidbau.com/archives/2010/01/30/random_seeds_coded_hints_and_quintillions.html

I really believe that this is one of the missing links to tie everything together and take UI testing combined with Unit testing to the next level.

Kazuhiro Kotsutsumi

1 year ago

I translated it into Japanese.

http://www.xenophy.com/extjsblog/5459

Provision: Japan Sencha User Group
http://www.meetup.com/Japan-Sencha-User-Group/about/

Comments are Gravatar enabled. Your email address will not be shown.

Commenting is not available in this channel entry.