Sencha Inc. | HTML5 Apps

Blog

Automating Unit Tests

August 02, 2012 | Arthur Kay

One of the first questions I always hear when starting with a new client is “How can I build unit tests for my application?”

It’s obvious that many people understand the benefits of unit tests—developers want to minimize the number of bugs in their code and managers want to reduce the amount of time required to test an application before release. Although the concept of unit testing has existed for years, software teams are only now beginning to explore building tests for their Rich Internet Applications (RIAs).

In my experience, it’s obvious that most people don’t fully understand what it means to build unit tests for RIAs. Considering there are few tutorials explaining how to accomplish this, I wanted to share a recent experience with the community in hopes that it will stimulate ideas and discussion.

Keep an open mind as you follow along with this post and think about how these examples might fit into your own processes. This implementation is one of any number of possible solutions, and I don’t attempt to solve every problem. That being said, implementing unit tests into your project will produce cleaner, more stable code and will reinforce coding standards throughout your organization.

I. Unit Testing Web Applications: An Overview

Layers Diagram

In the diagram above, the “Presentation Layer” represents the user interface - which in the case of RIAs is frequently a JavaScript-driven DOM. This is the case for most Ext JS and Sencha Touch applications.

When building unit tests for RIAs, it’s important to remember that the user interface (the “presentation layer”) is very different from the supporting layers of the application.

The UI accepts random input from the user and is expected to look AND behave the same across many environments (which is both subjective AND objective). One language (JavaScript) performs multiple tasks (logic and presentation). Because its code is never compiled, syntax errors in JavaScript are often not discovered until runtime.

Thus developers more familiar with server-side unit tests often fail to realize the complexity involved in writing unit tests for JavaScript-driven applications. As a result, we cannot build unit tests for the presentation layer in the same way we would for the underlying data-driven tiers of an application.

Because the presentation layer has multiple responsibilities, it is important to separate unit tests into the following three areas:

  • Syntax Checks – Syntax checks are not exactly unit tests because there is no logic being tested. Instead we analyze our code for syntactical errors – errors that may or may not be caught before application execution.
  • Unit Tests – True to its contemporary meaning within software development, unit tests are a series of objective logic tests. Unit tests attempt to isolate small pieces of our code: given specific inputs, unit tests expect specific outputs. In this manner unit tests are essentially mathematical proofs (remember those from high school?) that confirm what our business logic is supposed to do.
  • UI Tests (aka Integration Tests) – UI Tests are not the same as unit tests. UI Tests attempt to subjectively verify that elements on the screen behave (and/or look) as expected when a user performs a given action. There is no math or logic involved: the tests render the environment as a whole (containing all runtime dependencies) and wait to verify that the DOM has changed in a desired way. You can think of these tests as a robot manually testing the application.

I do not attempt to solve the problem of UI Tests in this post, but there are helpful links are included at the end if you're interested.

II. A Sample Application

The key to implementing unit tests is to automate the process - if the tests aren’t run every time the code changes (or at some regular interval), the tests become ineffective and useless.

I have created a sample project that uses shell scripts to run each individual part (syntax checks and unit tests). These scripts return status codes upon success and failure.

The specific process to integrate these automated tests may be quite different at every company. The shell scripts might provide automation by:

  • being integrated into a build process (e.g. via Ant)
  • being integrated into source control hooks (e.g. a Git pre-commit hook on your local machine)

III. Pre-Requisites

The sample project (and the concepts explained below) assume a certain degree of knowledge on the following tools:

  • Sencha SDK Tools (v2.0.0b3) - a suite of automation tools assisting with application creation, development and deployment
  • PhantomJS (v1.4.1) - a headless WebKit browser controller via an easy JavaScript API
  • JSLint (v2012-05-09) - a JavaScript code quality tool
  • PhantomLint (v1.2.1) - an add-on to Phantom.js that automatically lints all JavaScript files in a project
  • Jasmine (v1.2.0) - a behavior-driven development framework for testing JavaScript code

The shell scripts in the sample project were targeted towards Mac OSX/Linux users. However, Windows batch (.bat) files or Ant scripts can easily accomplish the same thing and should be easy to create.

Also, if you're new to the world of unit testing I would highly recommend the book Test Driven JavaScript Development by Christian Johansen.

IV. Syntax Checks

While there are a variety of syntax-checking tools available for JavaScript, perhaps the most widely recognized is JSLint. Those of you familiar with the tool should remember that “JSLint will hurt your feelings” - it complains about everything! On the other hand, it is also highly configurable and helps our code be clean, stable and consistent.

Because JSLint is written in JavaScript, it is traditionally run in the browser on Douglass Crockford’s website (http://www.jslint.com). Although this manual process is sometimes convenient, it is difficult to run against multiple files and impossible to automate.

A better solution is to use PhantomJS - a headless WebKit browser that provides the web environment necessary to run JSLint and our own JavaScript code. Additionally, PhantomJS can access the filesystem; this allows us to execute multiple files against JSLint and report back some success or failure status.

To further simplify the process of checking all of our JavaScript files, I have incorporated a pet project of mine called PhantomLint which logs errors to an output file. In this way, a failure status can alert developers to take corrective action.

Taking a look in the sample project under /tests/, you should see a shell script named “run_lint.sh”. This script launches the PhantomJS environment and initializes PhantomLint for us (JSLint-Runner.js). Given its configuration options, PhantomLint then dives into the filesystem to test our code against JSLint. Any errors are then output to a text file.

If you run the shell script, you should notice a single error in our application code:

 
../app/MeetupApiUtil.js
   Line #: 37
   Char #: 21
   Reason: Unexpected 'else' after 'return'.
 

In this case, JSLint tells us that we have a redundant “else” statement.

V. Unit Tests

There are a variety of JavaScript unit test frameworks currently available, and I chose Jasmine for this sample app. Why Jasmine? It is one of the more popular frameworks currently available, it has an active development community, and Sencha uses it internally to test our own code. Jasmine also has a very fluent API - this makes our test suite easy to understand. By nesting test conditions appropriately, we can build very powerful and descriptive test suites.

The tests can be run locally by visiting the “spec_runner.html” file under the /tests/ folder in one’s browser. The individual test files are located under the /tests/app/ directory.

Let’s start by looking at the class ChicagoMeetup.MeetupApiUtil. This class is ideal for unit testing because it has very few dependencies and doesn’t directly interact with the DOM; it is simply a collection of utility methods that perform logical operations.

Taking a look at the tests inside /tests/app/MeetupApiUtil.js you can see how the unit tests objectively analyze the methods for this class. Each test condition provides a specific input value and expects a specific output value.

 
describe('ChicagoMeetup.MeetupApiUtil', function() {
 
    describe('getUsersUrl() method', function() {
 
        it('should be a function', function() {
            expect(typeof ChicagoMeetup.MeetupApiUtil.getUsersUrl).toEqual('function');
        });
 
        it('should return a string', function() {
            expect(typeof ChicagoMeetup.MeetupApiUtil.getUsersUrl()).toEqual('string');
        });
 
    });
 
    //etc...
});
 

The ChicagoMeetup.MeetupApiUtil class and the related unit tests are a terribly simple example - in fact there’s very little logic involved. More often than not, we want to build unit tests for custom components in our Sencha applications. How can we achieve this?

Consider the class ChicagoMeetup.view.Events in our sample application. This is a custom class, extended from Ext.grid.Panel, that contains some specific methods and behavior for our app.

In our test code (/tests/app/view/Events.js) we first create setup and teardown methods that provide fresh instances of our custom component for each test case. We do this to avoid polluting our test environment with abandoned objects and DOM elements.

 
describe('ChicagoMeetup.view.Events', function() {
 
    //reusable scoped variable
    var eventGrid = null;
 
    //setup/teardown
    beforeEach(function() {
        //create a fresh grid for every test to avoid test pollution
        eventGrid = Ext.create('ChicagoMeetup.view.Events', {
            renderTo : 'test' //see spec-runner.html to see where this is defined
        });
    });
 
    afterEach(function() {
        //destroy the grid after every test so we don't pollute the environment
        eventGrid.destroy();
    });
 
    it('should inherit from Ext.grid.Panel', function() {
        expect(eventGrid.isXType('grid')).toEqual(true);
    });
 
    //etc...
});
 

It is important to note that we temporarily render our custom component to the DOM for each test (via the “renderTo” config). We do this in order to test any logic that might depend on the DOM – but the key is to destroy these components in-between tests so the greater test environment is not polluted (negatively affecting subsequent tests).

Although we are rendering the component to the DOM, I have to re-emphasize that we are not using Jasmine to build UI (or integration) tests. Jasmine doesn’t care what the components look like - our unit tests are only here to analyze the business logic.

Now that we understand how to properly unit test our components, the next step is automating this process. Similar to how we automated our syntax checks, we will again use PhantomJS to run our unit tests and output failure messages to a log file.

Taking a look in the sample project under /tests/, you should see a shell script named “run_jasmine.sh”. This script launches PhantomJS and initializes the Jasmine parser for us (Jasmine-Runner.js). After our tests have run, any test failures are output to a text file.

If you run the shell script, you should notice a single test failure:

 
Spec: linkRenderer() method
Description:
   - should return a string (HTML link snippet)
Failure Message:
   - Expected 'string' to equal 'function'.
 

In this case, Jasmine tells us that a test against linkRenderer() failed. We expected that this method would return a function, but our test case encountered a string. This particular example fails because the spec incorrectly expected “function” - which demonstrates that unit tests can contain errors themselves!

 
describe('linkRenderer() method', function() {
 
    it('should return a string (HTML link snippet)', function() {
        var testUrl = 'http://www.sencha.com';
 
        expect(typeof eventGrid.linkRenderer).toEqual('function');
        expect(typeof eventGrid.linkRenderer(testUrl)).toEqual('function'); //THIS SHOULD FAIL! We should have written toEqual('string')
 
        //TODO: more robust regular expression checks to ensure this is *actually* an HTML link tag, correctly formatted
    });
 
});
 

It is unrealistic and nearly impossible to achieve 100% code coverage. Because unit tests are designed to test small pieces of code in isolation, it becomes very difficult to write good tests for pieces with many dependencies or major DOM interactions. Try to focus on problem areas or base classes, and write tests that check for bugs as they pop up (to prevent regressions).

VI. UI Tests

As I mentioned earlier, I am not attempting to solve the problem of UI Tests in this post. Testing the look and the complex interactions of an application is a humongous task, and is a topic better suited for its own dedicated tutorial.

That being said, here are some of my thoughts on accomplishing UI Tests:

  • Building UI tests for dynamic RIA’s is sufficiently difficult because the DOM is always changing in unpredictable ways. Tools like Selenium use XPATH or CSS selectors to verify “success” – but because the DOM is unpredictable, the tests written for these tools become very brittle. Thus maintaining your tests often takes more time than creating new features.
  • Many prominent UI developers prefer to use “specification” rather than “UI tests” for exactly these reasons. A good resource for you is the book “Specification by Example”, in which tools like Cucumber are recommended.
  • Siesta is becoming a more popular tool for automated UI testing and is certainly worth a look.

VII. Conclusion

Unit tests are an important part of the software development process. By clearly defining what we expect from our code, unit tests allow us to develop a high degree of confidence that our applications will function as intended. Automating our unit tests reduces the number of bugs and decreases the amount of time we need to spend manually testing our code.

Take a look at the sample project on GitHub and consider how this solution might integrate with your own processes. This post is meant to foster ideas and discussion within the community - so please chime in with questions, comments and solutions of your own! Feel free to fork the sample project and add or edit the examples!

There are 30 responses. Add yours.

Nickolay Platonov

2 years ago

Nice post. But the tests in the examples looks quite simple. Can you post how to test some real-world scenario, like:
- selecting a row in a grid
- then clicking “Delete” button
- The record from grid’s store should be removed
- there should be no presentation for that record in the DOM (grid row should be removed as well)

Thanks, Nickolay

Dawesi

2 years ago

Nice stuff,  I can feel a ‘BB’ session coming on…

Arthur Kay Sencha Employee

2 years ago

@Nickolay - That would fall under UI testing, and as I mentioned I’m not covering that topic in this post.

Michael Gai

2 years ago

Great article. Let me summarize some of the conclusions:

working against the DOM to do UI testing can be very painful, especially with Ext JS since *it* is doing the DOM generation for us (kind of the whole point)
you have to stand up DOM emulators like Phantom JS to get testing going (and so does every developer on the project)
“unit testing” coverage is going to be sporadic, minimalistic, and uninteresting because of the complexity of triggering interactions. Even if not testing the DOM for *output*, you still need to figure out how to locate and trigger Ext JS button clicks, combo box selects, etc. for *input*. That’s not even mentioning that many interactions are going to fire off an Ajax call that you’ll have to manage or ignore. So you’re stuck with “unit tests” that don’t test much at all.
Ext JS developers should look into specification-based (aka BDD) style frameworks like Cucumber to really address covering their business behavior.

And this is why we built GluJS (http://www.glujs.com). We didn’t think automated testing should be something hard you barely get to, but something simple that you start with on every project to drive it forward with full coverage of your app’s behavior:

DOM-less testing aimed squarely at ExtJS - so no extra tools (e.g. PhantomJS) needed
specification-driven (BDD) and follows a Cucumber-like approach (but in JavaScript)
bridges UI-testing and unit-testing by letting you ‘stand-up’ the entire application each test and run it through its paces - without the complexity of tracking down DOM or Ext JS control inputs.
an Ext JS Ajax request/response simulator so that your app can work normally in end-to-end testing without having to coordinate a live server.
meaning it enables true test-first (TDD) development on what you care about—*your* app and nothing else!

@Nickolay Platonov:

Here’s a snippet of what a GluJS spec looks when testing a removal operation. Note that instead of just testing that the row is gone, we simulate the *actual* interactions with the back-end (this is in CoffeeScript because specs are easier to read that way, but you can do it in JavaScript too of course):

When ‘user selects two assets’, ->
  selectedAsset1 = null
  selectedAsset2 = null
  Meaning ->
    selectedAsset1 = firstAssetSet.assetList.getAt(0)
    selectedAsset2 = firstAssetSet.assetList.getAt(1)
    firstAssetSet.set ‘assetSelections’, [selectedAsset1,selectedAsset2]
  When ‘user executes remove command’,->
    Meaning -> firstAssetSet.removeAssets()
    ShouldHave ‘asked for confirmation’,->
      expect(firstAssetSet.confirm).toHaveBeenCalled()
    ShouldHave ‘NOT called backend to remove asset’,->
      expect(backend.getRequestsFor(‘removeAssets’).length).toBe 0
    When ‘user confirms removal’, ->
      Meaning -> firstAssetSet.confirm.respond(‘yes’)
      ShouldHave_called_service ‘removeAssets’
      When ‘backend remove asset call returns OK’,->
      Meaning -> backend.respondTo(‘removeAssets’)
      ShouldHave_called_service ‘assets’
      When ‘grid request returns’, ->
        Meaning -> backend.respondTo(‘assets’)
        ShouldHave ‘returned fewer assets’, ->
          expect(firstAssetSet.assetList.getTotalCount()).toBe 9


Note that rather than verifying in the DOM (which is counter-intuitive in Ext JS because it is supposed to handle that structure for us) we just put all of the interesting/custom/potentially dangerous state interactions in a view model and then have our view 100% behaviorless. That lets us test *our application code* instead of re-testing other people’s libraries.

Granted it would be nice to at least also test the state of ExtJS controls if not the DOM. That’s simple with our approach as well since you can use the the binding map on our view models to easily locate the controls you need for both input (there’s a helper for that) and for expectations. BUT, since Ext JS controls don’t always work DOM - less (some do, some don’t, and we’ve been working on taming them), that means you have to fire up PhantomJS or the equivalent. In practice we don’t find the extra work all that helpful - the bindings aren’t nearly as interesting as the behavior in our application view models.

Michael Gai

2 years ago

Whoops, sorry the bullets didn’t come through on the cut-and-paste - but hopefully still readable grin

Alok

2 years ago

Thank you for the article and the tools listed by you. You are right that UI Test automation is picking up and that is where Unit Test and UI Testing has become the explicit focus area of the overall project plan.

Joeri

2 years ago

@Nickolay:

Checking the DOM is the wrong way to go about it, since ExtJS is supposed to handle the DOM manipulation for us, and we should rely on that being the case. In the example you give I would programmatically create a selection in the grid, activate the delete method, and check that the row is removed from the underlying store. If you want to test the actual click handler, you could fire a click event on the grid’s view, but in my opinion that’s not a unit test (that’s a functional test and you should be using something like selenium to run those).

Chris

2 years ago

I wholeheartedly agree with @Michael Gai :
There is no point to unit test that your component is inheriting from XXX or that method returns a XXX !
What you must test is that your component has doing something usefull when the user (or an XHR call or ...) has done something (and eventually that all your components are playing well together)

I never heard of GluJs before but I’ll take a look because I’m very seduced by the concept (albeit you must use a complicate structure to isolate ui logic from ui presentation and I’m still doubtful about this)

If you are interested about our approach, here it is : we do not have unit tests, we only have integration tests done with Selenium 2 and jBehave.

A test looks likes :
Given the grid ‘Contacts’ has 0 row
When the user fills the form ‘Add contact’ with
|field|value
|lastname|Smith
|firstname|John
When the user clicks on button ‘submit’ of the form ‘Add contact’
When the user waits
Then the grid ‘Contacts’ has 1 row
The the cell 1,1 of the grid ‘Contacts’ contains ‘Smith’
The the cell 1,2 of the grid ‘Contacts’ contains ‘John’

In java we then have
@When(“the user clicks on button ‘$btn’ of the form ‘$form’”)
public void clickBtn(String btn, String form) {
  ExtJsForm f = this.extjs.findForm(form);
  f.clickOnButton(btn);
}

The logic behind findForm and clickBtn is quite complex because we refer to screen labels not extjs ids or css BUT this logic is correct for all the form of the application (except some very custom components).
Ex :
public class ExtJsForm {
      public void clickOnButton(String buttonText) {
  this.clickOnButton(buttonText, 1);
}

public void clickOnButton(String buttonText, int nth) {
  String xpath = getButtonXPath(buttonText, nth);
  selenium.ajaxIsElementPresent(xpath);
  selenium.click(xpath);
}

  private String getButtonXPath(String buttonText, int nth) {
      String xpath = findFormCterXpath();
xpath += “/descendant::button[text()=\”” + buttonText + “\”]”; //find the button in the cter
xpath += String.format(”[%s]”, nth);
      return xpath;
  }

  public abstract String findFormCterXpath()
}
public class ContactForm extends ExtJsForm {
  public String findFormCterXpath() {
  return String.format(”//span[text()=\”%s\”]/../..”, this.title); 
  }

Nickolay Platonov

2 years ago

@Joeri

I don’t not agree here. I’m not a fan of dividing testing in several different sub-types and only doing one particular type of testing and not other. Testing is making sure your application works. If, in some version of Ext, there will be a bug in grid code and your tests won’t catch that - this will mean you’ll ship broken code.

Testing should be done on different layers of course. Testing model (aka business logic) is much simpler naturally, since there’s no DOM involved. Testing UI components is slightly harder, but also very doable. Testing application as whole is the hardest thing. But you can avoid that, because the application consists from individual components and testing one component is not hard.

Arthur Kay Sencha Employee

2 years ago

@Michael Gai - You summation of my post is incorrect.

I completely disagree with your statements that “unit testing coverage is going to be sporadic, minimalistic, and uninteresting” and “you’re stuck with unit tests that don’t test much at all”. My examples are simplistic, but enterprise JavaScript applications will require that business logic is properly tested - and this logic has absolutely nothing to do with DOM interactions. Any proprietary application might have more lines of unit tests than actual code - and to imply that they don’t test anything (or anything important) is just false.

I have not heard of GluJS - so thank you for bringing that into the conversation. It looks interesting.

@Chris - I also disagree with your statement that “There is no point to unit test that your component is inheriting from XXX or that method returns a XXX”. Redundancy is integral to building solid tests - you would be surprised how often tiny (or obvious) mistakes creep into an application. Large and distributed development teams constantly have that problem - and true “unit tests” are the only way to solve that.

@Nickolay - You said that “I’m not a fan of dividing testing in several different sub-types and only doing one particular type of testing and not other. Testing is making sure your application works”.

I 100% agree with you. Although I didn’t attempt to solve the problem of UI Testing in this post, I would definitely encourage all developers to have a plan for that area - whether it be screen automation (through Selenium and similar tools) or via specification or something else.

Rodrigo Rosenfeld Rosas

2 years ago

I’d really suggest you to take a look at oojspec testing framework:

https://github.com/rosenfeld/oojspec

It is specially designed for better handling integration tests (or UI tests like you call them). For example, it supports beforeAll/afterAll while Jasmine don’t. It also allows you to write specs using an OO approach which can be quite useful for complex integration tests.

Ryan Smith

2 years ago

I’ve actually been using GluJS for a little while now, and its totally transformed how we do development and unit testing of our business logic.  It is definitely a switch in thinking about how you develop your applications, but once you are used to it you’ll never turn back.

@Chris - You mentioned that you are doubtful about using a complicated structure to isolate ui logic from ui presentation, but that is exactly what ExtJS MVC is attempting to do.  You write your views, and you have controllers that manipulate the views.  The problem is that the link between the two requires you to write even more code (as references and selectors) to get a reference to the view controls.  The benefit that I’ve seen with GluJS is that it does all this binding automatically for you since it is based on an MVVM framework (like Knockout or AngularJS for ExtJS controls).  This allows us to test the business logic, and let GluJS hook it up with our ExtJS controls (views).

@Arthur Kay - I think Michael Gai has a good point. It has been my experience that any kind of unit testing that I’ve coded in the past doesn’t do a great job because you’re using tools like selenium to write presentation layer unit tests that actually test the business logic (which end up being extremely fragile and “uninteresting” because of the complexity involved that breaks every time you touch the code in some slight way).  Your article is the tip of the iceberg for solving the problem of business logic testing and keeping it totally separate from presentation layer testing (which is fantastic!).

A business requirement like “When the user selects a row in the grid, the details for that record should be displayed in the form next to it” doesn’t translate very easily to presentation layer testing.  If you are writing that test in selenium, then you are testing the wrong layer with the wrong tool, and that’s where problems arise.  Michael’s point is that GluJS encourages you to do business logic testing.  And if you are using GluJS to test the business layer, you also get the benefit of having it auto-bind to the presentation layer (ExtJS controls).  This leaves you free to use selenium (or others) to test the presentation layer and make sure that your application is rendered properly by the ExtJS framework.  However, once we started testing our business layer properly, we’ve found that the ExtJS controls do a fantastic job of manipulating the DOM and displaying things properly across browsers.  It is what they were made to do, so we’ve found that there isn’t a lot of added value in testing the presentation layer anymore (with the exception of custom controls).

Keep up the great work!

Michael Gai

2 years ago

@Arthur:

First, let me say you wrote an excellent blog post on a subject that sorely needs addressing. Thank you!

Second, I couldn’t agree with your response to me more! Obviously I wasn’t clear the first time (I blame the mis-formatting and not anything sloppy in my writing wink

We agree: the *business logic* of the application is absolutely the interesting thing you want to test, and you *should* have full coverage of it. In fact, you should *build your application* around full behavioral testing as you go via a specification approach. Let Ext JS manage visual rendering (the library does that amazingly well) so you can concentrate on defining and building your app.

My point is that when building UIs *if* you arbitrarily separate out full interaction/behavioral testing as something essentially separate from unit testing, then what’s remaining for those unit tests isn’t much. That’s because unlike back-end code, what you need to test about UIs are *precisely* domain-specific interactions and behavior and little else!

My customers are all enterprises or enterprise ISVs that use Ext JS to provide a unified development environment and a unified look and feel for web apps. Some of them have *100s* of business apps that need conversion to Ext JS. They have limited resources to spend on testing; they have bigger issues with just getting the apps minimally working quickly. Our approach is a “sweet spot” that gets them full behavioral test coverage in a way that *also* actually helps them build their apps faster.

The technical solution (GluJS): put all of the interesting state into the controller so that it is a testable state machine (which by definition makes it a view model as in “MVVM”). Then have an infrastructure piece that two-way binds it to a behavior-less view so that the views are simply pushing the pixels that correspond the the state machine. Then test-first the heck out of the state machine using specifications that *cover all behavior* as you go. And (because the binder is handling boilerplate wiring code), write much less code to test in the first place!

So when you say:

  “whether it be screen automation (through Selenium and similar tools) or via specification”

we at CoNarrative are just saying “yes. and if you have to choose one, start with specification.” Because that gets you a high degree of coverage that isn’t an additional cost, but instead tangible savings.

@Ryan: I think I’m just repeating what you said grin

Arthur Kay Sencha Employee

2 years ago

@Michael - Now that you’ve clarified your points, I think we are in agreement. I think UI Testing is an important thing to tackle in the enterprise RIA world… but as everyone has noticed, it’s tremendously difficult to solve.

I look forward to trying your company’s tools, as almost everything else I’ve seen (*cough* Selenium) makes it impossible to add features or redesign the UI without rewriting the entire test suite. CasperJS is a promising tool and Siesta also looks cool - but I haven’t played with them enough to fully endorse them. Perhaps I will like GluJS grin

Having said that, I still disagree with this sentiment: “if you arbitrarily separate out full interaction/behavioral testing as something essentially separate from unit testing, then what’s remaining for those unit tests isn’t much. That’s because unlike back-end code, what you need to test about UIs are *precisely* domain-specific interactions and behavior and little else!”

That statement assumes your code only serves up a UI. Many of the enterprise clients I see have very large proprietary frameworks that sit on top of ExtJS (or Touch), and these frameworks are constantly used to build new “applications”. In situations like this, each “application” would certainly have relevant UI Tests… but wouldn’t you want to know that the underlying codebase would always behave the same, regardless of where it was used?

Having built a lot of custom ExtJS widgets for enterprise clients, I can assure you that there are MANY things that need to be tested in isolation - particularly if it’s a widget being used in multiple places. One would want to build UI Tests for the widget… but the Unit Tests in these cases are sometimes more important as there are no UI pieces to test (e.g. private functions, getters/setters, etc).

I realize I’m sort-of ranting about the idea, but all-too-often I feel that developers get hung up on UI Tests, and they don’t understand the inherent value in Unit Tests.

Perhaps I’ll have to write a follow-up post on the subject of UI Testing and compare the available options grin

I’m glad I’ve stimulated some conversation with this post… let’s keep it coming!

Michael Gai

2 years ago

@Arthur: Thanks! I hope you do like GluJS - the guide at http://conarrative.github.com/glujs-guide/ is a great place to start. If you have any questions feel free to write me at mike <at> conarrative <dot> com.

I couldn’t agree more with your further point: There absolutely is a vital place for isolated unit testing of new “black-box” components that are developed for use across projects within an enterprise.

In my experience usually there’s a core team of very few (sometimes just one) developers who are the Ext JS “gurus” developing those controls (I’ve done more than I care to count myself). Those controls need isolated unit tests. Then there are many, many, more “domain application” developers who want to use the custom control as “just another component”. They have lots and lots of application-specific behavior to build. It’s the needs of the latter that Glu JS is aimed at and that I was addressing.

Even when it’s the same people doing both, you’re still going to want to put isolated unit tests around your new components and other fundamental building blocks (the point of unit tests). Then you can include them in your application without having to test every aspect of their behavior every time they are used (as you point out).

Thanks for your great post and insightful replies - I’m looking forward to more.

Chris

2 years ago

@Arthur @Michael @Ryan Thanks for the interesting debate !

First, I still wholeheartedly agree with Michael wink ... but I also agree a little more with Arthur
IMHO, you have to unit tests 2 things :
1. your reusable components
2. your business logic *if* you have any

In my 5 years experience with extjs (2 companies building SaaS platforms for enterprises with a GUI in ExtJs), I found that there are very few specific components and that I do all business logic server-side or ouside extjs components (i.e in specific javascript classes that are unit tested)

So, in my POV, the main challenge is to test the “glue” logic between components (i.e user interactions, events, callbacks ...) so let us wait the post about UI testing to see what Arthur thinks about it !

Finally, I suppose that the MVVM architecture of GluJs is a good choice here but I still had no time to test it (I started reading the user guide which seems very good) ...

George Jempty

2 years ago

If one of the pre-requisites is Sencha SDK Tools, then this would be for an ExtJS 4 application, correct?  Than why speak of unit testing logic in components, since Sencha’s “MVC” wants logic to reside in controllers?

Arthur Kay Sencha Employee

2 years ago

@George - I think I may have left the SDK Tools prereq in by accident… the approach taken in this post could apply to either a Touch or an ExtJS project (though the example is written in ExtJS).

I specifically chose to write a unit test for a component because many of our customers write re-usable UX components, and the logic in this case would live on the component.

However, you could very likely have unit tests applied to your controllers. The key would be to test the individual parts in isolation (as opposed to having to load the entire application from some state). But because the controllers are often tightly coupled with the DOM, I would encourage any true logic operations be extracted into utility classes. That way you could test the utility classes in isolation, which is far easier (and faster) than launching a full MVC app.

Chris

2 years ago

Can sencha apply a minimum spam filtering on the comments BEFORE notifications are sent please.
I keep receiving notifications about new comment from people with names that are really not real !

Michael Mullany Sencha Employee

2 years ago

@chris - believe it or not, we catch about 95% of spam comments with our spam filter, so the ones that you see have cleverly disguised themselves as real comments as far as the filter is concerned. We try to clear out spam comments every few days - but there’s no way to catch them before they post without going to a first time post moderation system, which brings it’s own problems.

Marcelsnews

2 years ago

Nice post
I’ am using Siesta in my case and it works perfectly good to test my Models, Stores, Views and i am learning how to build test cases for my Controllers (in JS). 
@NOTA: I am using ExtJs4 and SenchaTouch2.

Testing Views are not easy, especially when it is the moment to simulate user’s interaction in chain ... fiouuuuu .
If someone has TEST’S STRATEGIES i will be glad to hear about. I will be glad to share mine.

hudoor games

2 years ago

Unit Test completely implemented by coder. Only coder done but so other don’t interfere in program code.

TitisanAriel

2 years ago

hmmm…. i dont understund yet

724canl?

2 years ago

yeas =) Can sencha apply a minimum spam filtering on the comments BEFORE notifications are sent please.
I keep receiving notifications about new comment from people with names that are really not real !

Brian Kotek

2 years ago

Because Sencha doesn’t offer any platform-agnostic build system or Maven integration, you wouldn’t believe the incredible pain involved in trying to integrate Sencha apps into a normal enterprise build process. That includes automated testing as well, since obviously the build has to work before a CI server and run any tests.

People are crying for this on the forums but no one at Sencha seems to be taking an interest (http://www.sencha.com/forum/showthread.php?192915-Ext-JS-in-a-maven-project and http://www.sencha.com/forum/showthread.php?207567-Sencha-SDK-and-Maven).

Arthur and Michael, it’s really strange to hear talk about building “enterprise RIAs” with Sencha’s platform when you offer no support for the build system used by practically every enterprise IT department on the planet. Can we get someone to post to one of the forum threads on this subject and explain what, if anything, Sencha is doing on this front?

Thanks,

Brian

Constantine Smirnov

2 years ago

Very useful article. Thanks! You inspired me for this project (has some improvements for your scripts): https://github.com/kostysh/UnitTests-bundle-for-Sencha-Touch

Chris Westbrook

2 years ago

@Brian Kotek
I agree with you it has been a major pain point for us in the enterprise automation.

You would think that Sencha has had to solve some of these problems internally with a framework this large. I know from looking at the SVN and the forums that they use ANT to build internally for nighties and Jasmine for Ext Core testing. Yet, we have little visibility into how they do this automated testing, UI Testing. I am pretty confident that they are not testing this by hand, it’s too big a framework and we don’t see a lot of regression defects.

We had a Sencha trainer on site who said that have full automation in place… but he didn’t know any details.

So, @Sencha, can you please explain how you have test automation in place for Ext JS internally?

QA Thought Leader

2 years ago

“Regression testing is a type of testing that verifies the functionality of the application after developers make changes or additions. There’s no doubt that, regression testing consumes a lot of time, resources and money. Most of the times software testers treat regression testing as waste of money. If testers are really committed to product’s quality then quality of on-going testing should be as important as product’s development. Testing scripts can be generated through automation that detect the breakage in the code after changes have been made. Below are some factors tester should consider before automating regression test cases:

- Criticality of the project
- Re-Evaluate the cost while keeping your needs in mind
- Advice for avoiding testing pain
- A sensible action plan


With automation, testers can develop testing scripts to efficiently detect breakage after changes have been put in place. The goal is to fully test the application at every release without increasing the amount of testing done per release. You want the amount of effort to stay the same.

DanM

2 years ago

interesting approach, and although i agree that automation has its benefits, i trust that there is no universal answer, whether to use manual vs. automated testing. One can choose to use one or the other, depending on the software that you are building, also depending on the situation and the stage of the development cycle. When the complexity of the software requires human skills and judgement, then you have no choice but using manual testing - however if you need to repeat a simple testing cycle many times, then automated testing will save you time.

Because software products are dynamic and they tend to change constantly, the testing strategy and the test suites will need to be updated as well, to keep up with the updated functionality - therefore you may even end up going back and forth between manual and automated testing, for the same product, as time goes by.

some time ago, i’ve written an article about manual vs automated testing, if you’re interested, you can find it here: http://www.sbp-romania.com/Articles/manual-vs-automated-testing.aspx

Hi

2 years ago

Thank you for article.

As for me I have mixed opinion about Unit Testing in javascript.
From one part - I constaly come across Sencha team topics and messages that Unit Testing is very cool.
For example Mats Bryntse in his presentation write phrase
“Without Unit tests, your are not refactoring. You are just changing shit”.
It is here http://www.slideshare.net/mankz/testing-ext-js-and-sencha-touch page 66.
It is written that prase belong to Hamlet D’Arcy

I found this expression in Hamlet blogs
http://hamletdarcy.blogspot.com/2009/06/forgotten-refactorings.html

But I still think it is very rude and does not suit for real bussness presentation.
I think it is bad to use filthy language to discribe his new project Siesta.


On the other hand I really do not need any Unit Testing.
I just write good code.
And I do not think that I change shit when I do refactoring of my code.

I many times tryed to implement Unit Testing in work but I do not need it at all.

Unit testing change the way of developing.
Instead of thinking how to write good extention I think on what test to write.
There are not so much sititation when my code can be broken.

Maybe I am wrong but I do not need Unit Testing at all.
It gives me nothing but inlarge time of developing and distarct form real work.

Comments are Gravatar enabled. Your email address will not be shown.

Commenting is not available in this channel entry.