UI Testing a Sencha App
The one area that I did not cover in that post was the idea of “UI Testing” (also known as Integration Testing). After reading many of the comments to that post and hearing the community’s feedback, I wanted to address this topic by adding UI Tests to my demo Ext JS application and discussing strategies for enterprise application testing.
UI Testing: An Overview
As I mentioned in my earlier article, UI Tests are not the same thing as Unit Tests—but more often than not these ideas are confused.
UI Tests attempt to subjectively verify that elements on the screen behave (and often look) as expected, both statically (i.e. the flat render) and also dynamically (i.e. as users perform given actions). Unit Tests attempt to isolate small pieces of code and objectively verify application logic.
The key difference here is the subjective vs. objective nature of the tests.
Taking this idea a step further, we can break UI Tests into their constituent parts: QA Tests and Component Tests.
- QA Tests simulate real-world interactions with your application as if a user is using the app.
- Component Tests isolate independent (often reusable) pieces of your application to verify their display and behavior.
In this article, we’ll take a look at both types of UI Tests.
Common Problems with UI Testing
Selecting the Right Tool
Testing the look and the complex interactions of an application is an enormous task—and it’s no surprise that many web developers struggle to implement (and often abandon) UI Tests that adequately solve QA problems.
Perhaps the biggest hurdle that developers must address is selecting the best tool for the job. Some tools rely on XPath or CSS selectors to navigate the application; others require complicated server configurations to allow for test automation. At the end of the day, it is vitally important to select a tool that is flexible; QA Tests may be frequently rewritten as business requirements and as the app changes, so it’s important that the tests are easy to build and maintain.
In my experience, I have only found three tools that can address the needs for writing QA Tests against a Sencha application:
Each of these tools has specific pros and cons; having said that I personally think Siesta is the easiest of them to setup, and Siesta’s API works seamlessly with Sencha’s frameworks.
Disclaimer: Other UI testing tools exist, and I do not claim to have used all of them; nor do I claim that Siesta is necessarily the “best” tool in this space. I simply offer an opinion based on my own experiences.
One other important (and often overlooked) problem with UI Tests is that tests should typically not be written against live APIs. Regardless of the source, APIs are inherently unreliable: servers go down, networks experience latency, and unexpected bugs occur.
The purpose of UI Tests is to verify display and behavior—not that specific data exist at any given time. UI Tests should mock API data when possible, but this is not always an easy problem to solve. Some implementations use static data in place of AJAX requests; others redirect network calls to a mock API.
Having said all of that, you may have situations where you do want to use the live API—but I would normally only recommend that to verify a release is ready to be pushed live.
In the sample Ext JS application, you’ll find a /ui-tests/ folder containing both QA (/app/) and Component (/ux/) tests written with Siesta.
Under the /app/ folder, you can view the index.html file in your browser to see the Siesta interface with our QA Tests in place. The goal here is to launch the actual application and test the real-world interactions that users would expect. While the sample application is a relatively simple example, the two QA Tests demonstrate various ways in which we can test the behavior of the app as a whole.
The first test, titled “Test tabs for data in grids” (/app/tests/01_tabs.js) simply loads the application and checks to make sure the required views are correctly in place. Although this particular example is rudimentary, this sort of test could be very useful if your application dynamically created its interface based on user roles, preferences, or some other logic.
The second test, titled “Test double-click functionality” (/app/tests/02_RsvpWindow.js) again loads the entire application. This time, we begin to simulate interactions with our grids and tabs to ensure the desired behaviours execute as expected.
Take note of the fact that I use Sinon.js to stub the JsonP requests for these data stores. Doing this allows me to test the functionality of the application without the assumption that the live API is accessible and working properly. There are a number of ways you could accomplish this; I chose to override the behavior of Ext.data.JsonP.request() to automatically return mock data (see /ui-tests/app/api_stub.js).
Under the /ux/ folder, you can view the index.html file in your browser to see the Siesta interface with our Component Tests in place. Unlike our QA Tests, the goal in this case is to isolate individual components and test their behaviors. By testing the components outside of the greater application, we can isolate known bugs and guarantee future compatibility.
The only test I’ve written for this sample application (/apuxp/tests/01_RsvpWindow.js) examines the display and behavior of our RsvpWindow view. This view extends the Ext.window.Window class, and contains a grid with a custom column renderer. Using Siesta, we create an isolated instance of this view and verify the cell renderer behaves as expected.
var defaultWin = Ext.create('ChicagoMeetup.view.RsvpWindow'); //I know there are only 2 rows in this grid because I mocked the API var firstRow = Ext.get(grid.getView().getNode(0)), secondRow = Ext.get(grid.getView().getNode(1)); var regExp = /^(<img)/g; var innerHtml = firstRow.query('.x-grid-cell-inner').innerHTML; t.is(regExp.test(innerHtml), true, 'First row should contain an image in the second column'); innerHtml = secondRow.query('.x-grid-cell-inner').innerHTML; t.is(regExp.test(innerHtml), false, 'Second row should NOT contain an image in the second column');
Using Siesta’s robust testing API, we could simulate a variety of other interactions with this component (click, drag, etc). Although the RsvpWindow component isn’t all that exciting, you can imagine the possibilities for your own custom UX classes.
Building unit tests for a web application can be a difficult task, but when done correctly the payoff for your efforts is invaluable. In closing, I would like to reiterate these important points:
- Unit Tests and UI Tests are not the same thing. Both are valuable ways to maintain stable code, but they solve different problems.
- Mind your syntax. Just because your code runs correctly in one browser doesn’t mean it will run correctly in every browser.
- Test your custom components. It’s alright to assume your framework works as expected—but don’t assume your UX is written correctly.
- Don’t shoot for 100% code coverage. You may want to test the entire application, but beware the cost of maintaining an elaborate test suite.
This series of posts on unit testing is based on my own personal experience helping Sencha’s customers solve common problems. You can learn more in a webinar I'll be hosting January 31st alongside Mats Bryntse that introduces developers to realistic methods for testing your Ext JS and/or Touch applications. Registration is available here. In addition, I invite you to share your own thoughts and experiences below—we can help each other make web application testing an easier goal to achieve.
UPDATE (1/2013): I wrote a follow-up post titled UI Testing a Sencha App which expands upon the content in this article. I have also updated the sample project on GitHub - so there are some slight differences in the examples posted here compared to the updated repo, although the concepts are exactly the same. If you have questions or comments, please start a thread on the Sencha forums and I'll be sure to respond!