Guest Blog Post
In this article, I will introduce you to the Leap Motion Controller and describe how we built a Sencha Touch application to work with it.
At DecidoKompetensor, which is based in Malmö Sweden and specializes in ERP systems, we have seen that mobility is an increasingly big factor for success among our customers. We’ve enabled users of our ERP systems to perform tasks from their smartphones and tablets, and we’ve also built applications controlled by interfaces such as the new Leap Motion Controller.
The Leap Motion Controller is a USB device that can track the movements of your hands and fingers as input without the need for physical contact. It was released in the spring of 2013 and comes with its own Airspace app store from which a number of games, explorational and utility apps can be downloaded.
Scope of our app
Our first app using the Leap Motion Controller is a visualizer that makes data available and browsable to workers in a warehouse. The app presents data graphically in charts and illustrations that are displayed on large TV monitors placed near the ceiling in the warehouse. The app is controlled by gestures, enabling users to interact with it from a distance.
Our app was built using the Sencha Touch framework which has a powerful data layer and great components for visualizing data.
Talking with the Leap Motion Controller
Once the Leap Motion software is installed on a computer, a web application can easily get data from the Leap Motion device via a web socket connection. The Leap.js library facilitates this in two ways: either listening passively receiving 60 information frames per second, or actively polling for information when it is needed. Each information frame consists of positional data of hands, fingers/tools and gestures such as swipe or circular gestures.
The Leap Cursor Library
The Leap Cursor Library (written by Ross Gerbasi, a senior developer on the Sencha Touch team) controls the virtual cursor of the Leap Motion device. It supports a large range of interaction types, and even though it is quite extensive, we decided to write our own because our application only needed one main interaction type: a virtual cursor that can fire taps/clicks.
The Virtual Cursor
We decided to build a UI where the user controls a cursor on the screen with the movements of his or her fingers. When the cursor is held over an interactive element for a short period of time, a tap/selection is made. This type of interaction was inspired by many of the Xbox Kinect games and the New York Times’ Leap Motion demo.
We used the Ext.carousel.Carousel component and set the carousel property to make the neighboring slides visible.
Building our Sencha Touch app
We used the Leap.js library to give the Sencha Touch application access to positional data (X and Y coordinates) of the user’s fingers. The application used those coordinates to draw a cursor on the screen giving the user the ability to interact. We used the following Sencha Touch features to create a rich, interactive experience for the user.
Ext.Carousel – the main UI
The main view consists of an
Ext.carousel.Carousel component. The screenshot above is a dashboard with simple graphs and data linking to slides presenting the data in more detail. The default carousel is a full screen component, and we needed to hint to users that there was more information outside the visible area.
We made the neighboring slides visible on the screen by setting the carousel’s property:
itemLength: browserWidth * 0.8.
On each side, layered over the carousel, we placed two transparent buttons. These are used for navigating to the next or previous slide, by firing:
Carousel.previous() and Carousel.next().
We used Sencha Touch Charts, making it easy to populate all of the charts with data and animate them. Each chart has its own store, and to switch data in the charts when different scopes are selected in the UI, we call:
This redraws the chart, now with new data, and animates the bars.
We used Sencha Touch Charts, making it easy to populate all of the charts with data and anim