Sencha Inc. | HTML5 Apps

Blog

Behind the Sencha Command Utility and the Build Process

May 02, 2012 | Jacky Nguyen

Behind Sencha Command and the Build Process The Sencha command utility is a cross-platform command line tool that helps make it easier than ever to develop applications with Sencha Touch 2. The tool consists of many useful automated tasks around the full lifecycle of your applications, from generating a fresh new project to deploying an app for production.

This article will help you understand the Sencha command utility as it’s used for your Sencha Touch 2 application’s build process.

This article is written with the assumption that you are already familiar with Sencha Touch 2 SDK, SDK Tools and basic operations using the Sencha command utility. If you aren’t familiar with these topics yet, head to the Getting Started Guide and learn how to quickly setup your development environment and generate a working Sencha Touch 2 application.

How It Works

The Sencha command utility is a pure JavaScript application running within the Node.js runtime. The source code is located inside the command directory of the Sencha Touch 2 SDK. This application requires that Node.js and other tools exist on your system, such as PhantomJS, Open-vcdiff, Closure Compiler, YUICompressor, etc. These binaries are included in the SDK Tools installer, so you don’t have to worry about downloading them yourself.

After the SDK Tools is installed, from a command line or Terminal, sencha command simply delegates to the following:

/path/to/sdk_tools/bin/node /path/to/sdk/command/sencha.js [arguments…]

with /path/to/sdk_tools being where the SDK Tools were installed, and /path/to/sdk/command being the path to a Sencha command utility directory. It is that simple.

Application Structure

The Sencha command utility helps you create a fresh new application with a one-line command:

sencha app create MyApp /path/to/myapp

Two of the most important parts from the output are the sdk directory and the app.json file.

The ‘sdk’ directory

This directory is a copy of the SDK from which the application was generated. This way all your application source code is self-contained which makes it easy to archive, share and version-control. Also, to upgrade or downgrade an existing application, you simply replace the sdk directory.

However, if for some reason you need to change the name of this directory or store it in a different location outside of the application, simply edit its path in the .senchasdk file. This special file gives the Sencha command utility a hint on where to look for the SDK within the current working directory.

The ‘app.json’ file

Getting a Sencha Touch 2 application up and running can be as simple as creating an HTML document and including a bunch of JavaScript / CSS files in the right order. However, when it comes to deployment, things can get complex quickly, and there is a lot of work involved in optimizing your application in a production environment. Usually pre-production optimization boils down to:

  • Minimizing network payload size
  • Minimizing the number of HTTP requests
  • Providing instant UI feedback when the application is loaded for the first time
  • Caching non-frequently changed resources on the client, when possible
  • Minimizing network transfer when updating between different versions

In other words, that means:

  • Figuring out a minimal list of dependencies your application needs instead of including the whole library
  • Concatenating separate JavaScript/CSS assets into single bundles and minifying them
  • Using asynchronous loading mechanism
  • Storing all JavaScript/CSS assets locally on first load and evaluating them on subsequent access without any network connection required
  • Generating deltas between all versions of your application and automatically instructing the client to download and then patch their local copies

In order to automate all of that for you in just one single command (sencha app build), the Sencha command utility needs to know everything about your application’s structure, for example:

  • Where the required JavaScript/CSS files are, and in what order to include them
  • What update mechanism to use for each of these assets
  • What files and directories to copy along when doing the build
  • Where to generate the builds

All of this information is contained inside a JSON-formatted configuration file named app.json. Two of the most important configuration items to be highlighted here are “js” and “css”.

With the new application structure, JavaScript and CSS assets should never be physically linked inside the HTML document (for example, manually added as <script>, <style> or <link> tags). Instead, they should be specified and maintained inside app.json as array of JavaScript and CSS items in the right execution order for the “js” and “css” keys, respectively. Such items may look like this:

"js": [
    {
        "path": "sdk/sencha-touch.js"
    },
    {
        "path": "app.js"
    },
    {
        "path": "vendor/foo-library.js"
    },
    {
        "path": "3rd-party/analytics.js"
    }
]

This enables the use of the micro-loader, as described below:

The Micro-loader

The micro-loader was first introduced in Sencha Touch 2.0.0-GA. It is a small script that is responsible for loading up all JavaScript/CSS assets asynchronously, caching them locally, and updating their local copies when a new version of the application is deployed.

In fact, it is, and should be, the only script that is physically included inside your HTML document. During development, every time the application loads, the micro-loader simply requests and parses your app.json file, then dynamically loads all of your JavaScript/CSS assets in the specified order.

In a production environment, however, this process only happens once. Upon completion of the first load, the content of each asset is stored locally inside the client browser’s Local Storage. As a result, subsequent access to the application requires absolutely no network connection. The asset’s content is retrieved locally and evaluated immediately, still in the right execution order, and as fast as the device’s CPU can handle. Your application not only is offline-capable, it also loads in a “blink” on mobile devices.

But, you may wonder, doesn’t HTML5 application cache already provide all of those features? That question brings us to the next section.

The Delta-update Mechanism

HTML 5 Application Cache seemed like a dream come true to web developers when it was first introduced. If you are new to this feature, we highly recommend that you read this beginner-friendly article to have a better understanding of the subject. As a quick summary, your web application can finally run without the device always having to be online. All specified assets inside the manifest file will be retrieved for the first time and cached locally. Sounds perfect, but what happens when an asset’s content changes on the server-side?

When using HTML5 Application Cache, a GET request is made in the background to retrieve the content of your manifest file every time your application loads. This newly retrieved content is then compared to the old one that was stored locally from the last time your application was used. Any difference in the comparison indicates that something has changed, and an update is required. The browser then makes a new GET request for each and every cached item listed in the manifest file to retrieve its latest content and refresh the cache.

In other words:

  • The browser knows something has changed but doesn’t know exactly what. That could mean all of your images and JavaScript files have to be pulled down again, even if you only tweak some styling in one CSS file. With the current state of this feature and the current format of the manifest file, there is no way to specify only the items that were modified and reduce all the unnecessary requests.
  • The update mechanism is still a full update. The whole file content is downloaded again even if you might only have changed a few lines of code. In reality, for rich web applications such as those written with Sencha Touch, that means hundreds of KB wasted for a few KB of code change. This becomes a serious drawback especially for mobile devices when cellular data connections are often slow, unreliable, and expensive with limited bandwidth. For businesses, rolling out updates as frequently as possible with small and incremental changes has always been a rule-of-thumb. Yet, this limitation makes that practice painful to watch.

That is where Sencha Touch 2 micro-loader comes into play with its built in delta-update mechanism. This simplified diagram illustrates the process:

Diagram illustrates how users are served only the small delta update file corresponding to their current version. This saves them from having to re-download the entire app.js.
The above diagram illustrates how users are served only the small delta update file corresponding to their current version. This saves them from having to re-download the entire app.js.

Delta update files Each time you make code changes and build the application for production, the Sencha command utility stores a copy of each asset that has been changed. This is called archiving. By keeping all previous versions of the assets in the archive directory, it can then generate a delta between each version. Client devices will then download the right delta content and patch their own Local Storage copies to become identical with the latest ones.

Similarly to the manifest file mechanism in HTML5 Application Cache, once your application finishes loading, the micro-loader issues a GET request in the background to check if there’s any update (only if the device is currently online). This information is presented in JSON format, with each asset having its latest checksum values. By comparing these checksum values with the previously cached ones, it knows exactly what asset has changed and proceeds with the patching process.

This solves both problems we mentioned earlier in the most optimal way. Even better, it is not meant to replace HTML5 Application Cache. In fact, both mechanisms are used together to perfect the solution. Local Storage caching and delta-update are only suitable for text-based assets such as JavaScript and CSS. Binary resources like images and videos should still be handled using the manifest file.

Summary

The Sencha command utility was born to automate many of your repetitive tasks during a project development cycle, while ensuring the outputs are optimized. I hope this article helps you gain some essential understanding of the work that the Sencha command utility does behind the scenes.

Written by Jacky Nguyen
Jacky Nguyen is one of the core engineers behind Sencha frameworks. With his well-rounded knowledge and experiences on both server-side and client-side development as well as industry best practices, he is responsible for designing and improving Sencha frameworks’ overall architectures. He was the author of MooTouch, which has become a part of Sencha Labs. Jacky is passionate about bringing innovation to software development and developers.

Share this post:
Leave a reply

There are 25 responses. Add yours.

Nigel Dahl

3 years ago

Jacky
This is all great stuff, and really well explained.  As someone who’s only recently started working with SenchaTouch2.0, but with a few years ExtJS experience behind me, I’m really excited by the step-change in experience, also to be seen with ExtJS v4.1 smile, and the integrated environment being provided.
What would be really helpful next is another article that explains the details of what happens with native app packaging to a similar level of detail.  I’ve successfully tested creating an Android native app but not yet got my head round how it will need to be updated as part of ongoing version control.  Does it always need a full rebuild if there is a small change in one of the component files, and what is the recommended mechanism for making the client aware that a new version of the app exists and should be downloaded?

slemmon

3 years ago

Is app building in ExtJS 4.x using “sencha app build” on the roadmap as well?

dwgw

3 years ago

“The Micro-loader In fact, it is, and should be, the only script that is physically included inside your HTML document.”

So where do you put google maps or other hosted scripts. I tried a putting url in app.json but did not know where to put it and I can’t seem to find any documentation on app.json. I tried pulling the script and saving it to in app folder then adding it to js section in app.json but build failed.

Juris

3 years ago

@dwdg : If you need to reference other scripts, then you just include it in index.html file like any other .js file like this:
[removed][removed]
[removed][removed]

In case You have some other non standard folders that should be copied during build process, they should be references inside app.json file in section “extras”

Amarjit

3 years ago

Great! 
I developed an application with sencha touch 2, but i have one problem when i try to packaging my application for android.
When i launch “sencha app build native”, the application build successful but after, i have this error :

Error: Bad component name: com.mobile.TestPack1.STActivity

““C:/devTools/android/android-sdk-windows\tools\android” create project—target android-7—name TestPack1—activity STActivity
—path C:\dev\sencha_ws\test1\build\native\—package com.mobile.TestPack1”

[ERROR] 1855 KB/s (809419 bytes in 0.426s)

My Packager.json is:


{
/**
  * @cfg {String} applicationName
  * @required
  * This is the name of your application, which is displayed on the device when the app is installed. On IOS, this should match
  * the name of your application in the Apple Provisioning Portal.
  */
“applicationName”:“TestPack1”,

/**
  * @cfg {String} applicationId
  * This is the name namespace for your application. On IOS, this should match the name of your application in the Apple Provisioning Portal.
  */
“applicationId”:“com.mobile.TestPack1”,

/**
  * @cfg {String} versionString
  * @required
  * This is the version of your application.
  */
“versionString”:“1.0”,

/**
  * @cfg {String} iconName
  * This is file name of your icon. This should be in the same directory of this configuration file.
  *
  * For iOS, please refer to their documentation about icon sizes:
  * https://developer.apple.com/library/ios/#documentation/userexperience/conceptual/mobilehig/IconsImages/IconsImages.html
  *
  * For Android, please refer to the Google Launcher icons guide:
  * http://developer.android.com/guide/practices/ui_guidelines/icon_design_launcher.html
  */
“iconName”:“resources/icons/Icon.png”,

/**
  * @cfg {String} inputPath
  * @required
  * This is location of your Sencha Touch 2 application, relative to this configuration file.
  */
“inputPath”:“build/native”,

/**
  * @cfg {String} outputPath
  * @required
  * This is where the built application file with be saved.
  */
“outputPath”:“build/”,

/**
  * @cfg {String} configuration
  * @required
  * This is configuration for your application. `Debug` should always be used unless you are submitting your app to an online
  * store - in which case `Release` should be specified.
  */
“configuration”:“Debug”,

/**
  * @cfg {String} platform
  * @required
  * This is the platform where you will be running your application. Available options are:
  *  - iOSSimulator
  *  - iOS
  *  - Android
  *  - AndroidEmulator
  */
“platform”:“AndroidEmulator”,

/**
  * @cfg {String} deviceType
  * @required
  * This is device type that your application will be running on.
  *
  * If you are developing for Android, this is not necessary.
  *
  * Available options are:
  *  - iPhone
  *  - iPad
  *  - Universal
  */
/* “deviceType”:“Universal”,*/
/**
  * @cfg {String} certificatePath
  * This is the location of your certificate.
  * This is required when you are developing for Android or you are developing on Windows.
  */
“certificatePath”:“C:/dev/certificate-codesigningkeys-flashMobile/Android/nttdata.p12”,

/**
  * @cfg {String} certificateAlias
  * This is the name of your certificate.
  *
  * IF you do not specify this on OSX, we will try and automatically find the certificate for you using the applicationId.
  *
  * This can be just a simple matcher. For example, if your certificate name is “iPhone Developer: Robert Dougan (ABCDEFGHIJ)”, you
  * can just put “iPhone Developer”.
  *
  * When using a certificatePath on Windows, you do not need to specify this.
  */
“certificateAlias”:”“,

/**
  * @cfg {String} sdkPath
  * This is the path to the Android SDK, if you are developing an Android application.
  */
“sdkPath”:“C:/devTools/android/android-sdk-windows”,

/**
  * @cfg androidAPILevel
  * This is android API level, the version of Android SDK to use, you can read more about it here: http://developer.android.com/guide/appendix/api-levels.html.
  * Be sure to install corresponding platform API in android SDK manager (android_sdk/tools/android)
  */
“androidAPILevel”:“7”,

/**
  * @cfg orientations
  * @required
  * This is orientations that this application can run.
  */
“orientations”: [
  “portrait”,
  “landscapeLeft”,
  “landscapeRight”,
  “portraitUpsideDown”
]
}

 

John

3 years ago

We want this for ExtJS!!!!

Giuseppe

3 years ago

@Amarjit Very interesting, you should post your entire project here to help us address the issue. Don’t forget sencha-touch-all-debug.js so that we can see which version you’re running

amarjit

3 years ago

I am using version 2.0.0. Currently for testing puposes, i have crtaed project using command line tool and paste the code for “Welcome to Sencha Touch 2” exxample only. which is working fine in the broswer, only issue is with native packaging.

Nigel mentioned he was able to package the application sucessfully. he may help us with that.

dwgw

3 years ago

@Amarjit I believe stbuild is not correct (on windows). I had the same problem but fixed it by replacing the new version of stbuild with the one from original Beta tools.

Shuchi Agarwal

3 years ago

@Jacky How do I load resources remotely? I have an application where all my js/css/images need to be loaded from a cloudfront.

AwesomeBobX64

3 years ago

Just a thought but shouldn’t the delta update mechanism perform a HEAD request (vs a GET) to compare the version on the server with the one stored locally?

Ropes Course Builders

3 years ago

Very nice. Information is really that helpful. ant wait to use that software.

Ropes Course Design

3 years ago

Very nice. Information is really that helpful. ant wait to use that software.

Milan

3 years ago

What about “building” ExtJS web apps?

AwesomeBobX64

3 years ago

@Milan That functionality exists too. Check out the tutorials for 4.0 and 4.1.

Milan

3 years ago

@AwesomeBobX64 It supposedly exists, but as we can see in forums it does not work reliably or without putting huge effort into making it work. I was hoping to see more details and a truly working version.

Ajay

3 years ago

This command line tool is helpful. Are there any plans for the tool to generate Coffeescript .coffee files in addition to the Javascript .js files? I find Coffeescript takes the pain out of managing the mass of [, {, }, ], commas, etc. that your Sencha files end up with.

slemmon

3 years ago

No dev dialog in the blog comments?

binaya pradhan

3 years ago

good to see ur tutorial…..nice one

PriscillaBARLOW33

3 years ago

When you are in the corner and have no money to move out from that point, you will need to receive the business loans. Just because it should help you for sure. I take student loan every year and feel great just because of that.

Derek

3 years ago

Are the sencha sdk tools supposed to work on Windows 7 64bit?  Because I am getting always this error [ERROR] The system cannot find the path specified.
[ERROR] Failed loading your application from: ‘file:///c:/xampp/htdocs/myapp/index.html’. Try setting the absolute URL to your application for the ‘url’ item inside ‘app.json’

I have tried setting the absolute URL to my application for the the url item inside app.json to no avail.  And the path exists also.  So it’s some weird thing that only manifests on windows.

Also, by the way, you can’t install this on Linux CentOS maybe only on Ubuntu.

Tools work well on my mac, but it’s not fair to say it’s cross platform when it clearly hasn’t been tested.  Fails right out of the box on 2 out of 3 platforms I need it to work on right now.  Can’t get my work done please help.

Derek

3 years ago

I was able to resolve this on Windows 7 64 bit.  The problem turned out to be my environment variable, used by the Application.js module to resolve dependencies was not correct.

Check SENCHA_SDK_TOOLS_2_0_0_BETA3 environment variable.  Mine was C:\Program Files (x86).  I read in another thread not to install the SDK tools there even though I"m sure 99% of people do because that is the default setting on Windows.

Anway, I had previously moved the SDK Tools to C:\dev\Workspace and updated my path manually to avoid having the space in Program Files and the binaries were somewhere else now, but I didn’t realize this environment variable even existed.

After setting the %SENCHA_SDK_TOOLS_2_0_0_BETA3% environment variable (and updating Path% with it),  I came down to my last error of a different sort

[ERROR] Failed loading your application from: ‘file:///c:/xampp/htdocs/myapp’. 

setting url : “http://localhost/myapp/index.html”  resolved that.

Greg

3 years ago

The below code works in the pre-built scenario standard product but when it is build with “sencha app generate production”  it no longer works.  the utils.js simply has a function utiltest(){alert(‘worked’);}  which is the last thing called in in the launch: function().

What is interesting is that I see the utils.js in the app.json?  and yes i have clear the case and cookies etc in the browser to force a refresh.

also if I manually add a script tag containing utils.js to the index.html it works….

What do you think?



“js”: [

      {
        “path”: “sdk/sencha-touch.js”
      },
      {
        “path”: “app.js”,
        “bundle”: true,  /* Indicates that all class dependencies are concatenated into this file when build */
        “update”: “delta”
      },
      {
        “path”: “utils.js”
      }
  ],

masterpae

3 years ago

Excellent post. I want to thank you for this informative read.

Pipo

3 years ago

Just wanted to say that I like Sencha Touch a lot, but it is very confusing to have the “Sencha Touch SDK” and the “Sencha Touch SDK Tools” which are two different things, but use the same command line keyword “sencha”. It isn’t just confusing in the beginning, but even if you understand the difference it is very hard to find specific problems for one of them with Google.

Comments are Gravatar enabled. Your email address will not be shown.

Commenting is not available in this channel entry.