Understanding Memory consumption

Can someone please recommend a resource to help understand memory consumption in web browsers?

I have inherited a sencha touch 2.0.1 application and chrome's task manager shows the memory consumption increasing constantly while the application poll a server for data. I had thought that memory leaks would not be an issue with Javascript but having done some research it appears that this is definitely not the case.

I also noticed that if I open some tabs in the Sencha Doc's application, and then cycle through the tabs repeatedly, the memory consumption of that page will also increase and does not appear to be garbage collected. This I don't understand because no addition data should be required for a tab that has already been viewed so what is the increase in memory being used for? Doing this with eight tabs can increase the memory consumption to over 1GB quite easily and the garbage collector is not reducing this significantly. Is that a problem with chrome or the web page? Can chrome's task manager be trusted?

In my application's case it is polling a server and updating a DataView every ten seconds. The memory consumption increases steadily - overnight it increased from an in initial 40MB to almost 4GB. All that the application does is read JSON formatted data from a server and populate a DataView with it every ten seconds.

It seems to me that there are issues with memory leaks in the sencha touch/Ext framework that make it unsuitable for applications requiring constant updates of their UI. Hopefully this isn't the case!

Like in any garbage-collected environment you get memory "leaks" when unused data is kept referenced in memory "somewhere". ExtJS and ST apps are sensitive to this because they have many layers of indirection where it's possible to leak memory in this way without knowing.

The most common causes of leaks:

• Having detached DOM nodes which have a circular reference with javascript variables (e.g. event listeners points to object, object property points to dom node). This is why it's very important to correctly clean up all DOM structures after using them (Ext.destroy in ExtJS, not sure what the corresponding method is in ST).
• Creating closures that are kept around as event handlers, with the closure closing over variables which you no longer need in its scope (call "myvar = null" for each variable in the closure's scope which you don't need after creating the closure).

This article provides more insight into general javascript leaks.

To pinpoint the cause of the leaks, you can use the chrome heap profiler, which allows you to see what's on javascript's heap. It can show delta's between snapshots so you can see where the growth in memory footprint is being spent.

I regularly use the heap profiler to go through my ExtJS app (single-page app with almost 100 K lines of code), and the ExtJS framework itself does not contain many leaks (there are some, but not many). Most of the leaks will be in application code written with assumptions about the magic of garbage collection which don't hold up in practice. I cannot say whether Sencha Touch is equally well-designed leak-wise as I have not yet profiled my ST apps.

---

Additionally I can provide you with these notes I made about memory leaks in ExtJS:

Typical ways in which a memory leak may occur in ExtJS:

• Event listeners linking removed components to components still part of the DOM
• Not properly removing components after they are not needed (e.g. not removing / destroying Ext.Window instances after they are hidden)
• Closures that capture references to no-longer-needed components (e.g. a component creates a closure on another component, and accidentally has a references to "this" in the closed-over variables.)
• Circular references across DOM / javascript (detached DOM element points to JS object, JS object points to detached DOM element)
• Javascript objects on removed DOM elements (not a circular reference, but will still leak in some browsers, IE7 is the worst)

Rules of thumb to avoid leaks in ExtJS (v3, we have not upgraded to v4 yet):

• When components are no longer needed, call Ext.destroy on them. Safe to call multiple times, better safe than sorry. Called automatically for nested components in the layout hierarchy. Removes all DOM references + purges event listeners.
• Avoid the DOM API's (use the ExtJS layout system). When adding custom DOM elements to which you attach JS objects or event listeners, destroy them when not needed. (e.g. do clean up in "destroy" method of component that created them; override and call superclass destroy)
• Use Ext.destroy() to do the clean-up on Ext.Element or Ext.Component. When adding components outside the layout hierarchy, destroy them when no longer needed. (e.g. Ext.Window with closeAction 'hide' is not destroyed when hidden; default is closeAction: "close" which calls Ext.destroy on hide)
• When using "autoDestroy:false" on an Ext.Container (e.g. toolbar) be mindful of manually destroying removed elements. Components in the hierarchy are still destroyed when the container is destroyed, but not when they are removed from the container.
• Careful with deferred Tasks using Ext.util.TaskRunner. Call taskrunner.stop(task) in the destroy method. Also remove all properties from inside the task object, because the "stop()" method keeps a reference to the task object around. (this is one of those framework-level leaks which you must work around)

I'm afraid there is no magic bullet, but I can tell you that this is a solvable problem which doesn't require huge engineering, just good comprehension of javascript's memory model.

Thanks for the reply, It's going to take me a little while to digest everything though.

I took a heap snapshot of my application last night and another this morning. 'Private' memory usage has increased to ~1.5GB, Javascript memory usage has increased to ~250MB.

I've saved the snapshots and reloaded them and now it doesn't seem to allow a comparison between them which is annoying. Also the heap snapshot only shows Javascript memory and not the memory used by DOM objects?

The breakdown of the memory is

6928 'Object' Retained size 195,518,600
634,890 'Class' Retained size 98 092 872
56,048 '(array)' Retained size 76,762,200
1,041,689 '(string)' Retained size 46,762,200
634,365 'HTMLDivElement' Retained size 45,675,960
...

I am seeing a lot of HTMLDivElement's which are being retained by 'Detached DOM tree' and 'dom in Class @xxxxx' where the id of said class is 'ext-element-xxxxx'.

Does this mean that I have a bunch of Ext.Element's which are not being released and are preventing the div elements from being released? Not sure what it would be doing here to cause that though as all it's doing is loading an array of JSON objects from a servlet and populating the DataView using the following where EventListContainer is a DataView:

Code:

---

var store = controller.getEventListContainer().getStore();
store.removeAll(false);
store.add(data);

---

Also, I'm not sure what 'Class' refers to in the heap viewer... does this refer to a Javascript class or a CSS class or..?

I created a test program to try to narrow down what is happening with my app and I have been running it for most of the day.

It's not behaving exactly as I expected though... The private memory has increased to ~250MB but the javascript memory has not increased noticably.

Why would this be leaking memory though? This has to be caused by sencha touch?

Code:

---

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8">
    <link rel="stylesheet" type="text/css" href="st/resources/css/sencha-touch.css">
    <script type="text/javascript" src="st/sencha-touch-all-debug.js"></script>
</head>
<body id="thebody">


<script type="text/javascript">


_eventListTemplate = new Ext.XTemplate(
    '<div >',
        '<div <tpl if="Color"> style="color:{Color}"</tpl>>{Name}</div>',
        '<div >{[this.calculateElapsed(values.LastUpdateTime)]}</div>',
    '</div>',
    {
        calculateElapsed: function(value) {
            var now = new Date().getTime(); // 1358836310298;
            if (value > now)
                return 'now';
            var difference = Math.floor((now - value) / 1000);
            if (difference < 60)
                return difference + ' seconds ago';
            difference = Math.floor((now - value) / 60000);
            if (difference < 60)
                return difference + ' minutes ago';
            difference = Math.floor((now - value) / 3600000);
            if (difference < 24)
                return difference + ' hours ago';
            difference = Math.floor((now - value) / 86400000);
            return difference + 'days ago';
        }
    }
);


Ext.define('ADMobile.view.EventList', {
    extend: 'Ext.DataView',
    alias: 'widget.eventlist',
    config: {
        itemTpl: _eventListTemplate,
        store: {
                fields: ['Name','Color','LastUpdateTime','IsProcessed','EventUuid'],
                data: []
        }
    }
});


var names = ['Pete','Mike','Sam','Paul','Dave','Rob','Jim','Frank'];


function rrand(range, llimit) {
return Math.floor((Math.random()*range)+1)+llimit;
}


function generateDataSet() {
      var temp = [];
      var thedate = new Date().getTime();
      var count = rrand(30,50);
      for(var i=0;i<count;i++)
      {
          var processed = rrand(10,0) > 3;
          var nameindex = rrand(8,0) - 1;
          var name = names[nameindex];
          thedate -= rrand(36000,7600);
          var colour = nameindex % 3 == 0 ? 'red' : 'black';
          temp[i] = {
              Name: name,
              Color: colour,
              LastUpdateTime: thedate,
              IsProcessed: processed,
              EventUuid: 'ID' + i
          };
      }
      return temp;
};


var dataset = [];
for( var i=0;i<10;i++)
{
    dataset[i] = generateDataSet();
}


var update = true;


var _GLOBAL = {
      poll: function() {
          var eventList = Ext.ComponentQuery.query('eventlist');
          var datasettouse = rrand(10,0) - 1;               
          if(update) {
              var store = eventList[0].getStore();
                    store.removeAll(false);
              store.add(dataset[datasettouse]);
              //update=false;
              store = null;
          }
          eventList = null;
          datasettouse = null;
          Ext.defer(_GLOBAL.poll, 500);
      }
};


    Ext.application({
    launch: function() {
        var thePanel = Ext.create('ADMobile.view.EventList', {
            fullscreen: true,
            scrollable:'vertical'
        });


        Ext.defer(_GLOBAL.poll, 1);
          }
    });


    </script>
</body>
</html>

---

If I uncomment the line 'update = false' in the poll function then the memory consumption does not increase. What is going on here?

Detached DOM elements being kept in memory are a typical symptom of leaking memory by having a circular reference from the DOM element to a javascript object and back to the DOM again.

The way to solve this is by making sure that the DOM element is properly "destroyed" (all references to it are removed from inside the javascript environment, using something like Ext.destroy to remove it from any caches and also setting all properties that point to it to null).

When I go troubleshooting memory leakage issues with detached DOM elements I'll usually just pick one of the detached elements that I can recognize as belonging to a specific feature, and then step my way through that feature's code where the DOM element gets removed from the DOM tree to figure out why it's not getting completely cleaned up.

Looking at that example code it does look like Sencha Touch itself is leaking memory. I would need to step through the code to make sure of this, but I'm afraid I lack the time to do that right now.

I've spent some time with the heap manager and reading the sencha touch source code and I think I know where my memory is going.

It seems that whenever I add a new item to the DataView it creates a new Ext.Element in the Ext.dataview.element.Container addListItem() function.

Each new element has a new div with a new id, I'm seeing ext-element-1 ... ext-element-nnnn It keeps growing.

For some reason these elements are not being garbage collected. I'm still able to reference very old elements from the console like this: Ext.get('ext-element-100') no matter how long I wait.

I'm not sure if this is a problem with Sencha or with google chrome though, Has anyone else observed this kind of behaviour?

Also, i noted similar behaviour while fixing another memory leak. The app was using an XTemplate to render an array of JSON objects. One property of these JSON objects was to be visualised as a chart so it would create a placeholder div with the format:

'chart' + i

where i = an integer that was incremented for each new placeholder.

I noticed that when regenerating the html from the template I would leak memory unless I reset the value of i to zero.

Could this be a problem specific to chrome where it is not releasing div's with id's? Is there a fix for this problem? I will look at installing a copy of safari and see if there is the same problem, hopefully it's related just to my version of chrome. Otherwise, if anyone else can shed some more light on this it would be great.

Ok, it seems as though I've been driving myself crazy for no reason here. I have determined that the problem is with version 2.0.1 of sencha touch and it is fixed with 2.1 - as per the release notes.

I guess I wasn't clearing the cache correctly between tests and this has lead me astray. Thanks for the help though.

I didn't realise that the scripts included in my html file, e.g, sench-touch-...html/css weren't being reloaded because the changes I made to my controllers/views etc.. were.

Memory consumption is still growing steadily but it's slow enough now not to be a problem. It is slow enough that it may just be normal activity so I'll keep an eye on it and see where it goes.

Is there some way that I can mark this post as resolved?

It will be resolved once you mark an answer as "mark as best answer"

top right of each post.

Thanks