Success! Looks like we've fixed this one. According to our records the fix was applied for EXTJS-9793 in 4.2.2.1144.
  1. #1
    Touch Premium Member
    Join Date
    Nov 2009
    Location
    London
    Posts
    49
    Vote Rating
    6
    mattgoldspink is on a distinguished road

      1  

    Default Ext 4.2 - performance issues when loading associations

    Ext 4.2 - performance issues when loading associations


    Hi,

    I've been profiling our app which uses associations and we've noticed that large dataset loads were pretty slow when associations were in use. I've fixed a few bugs we had on our side but I've found some code that I think could be better optimised in the Sencha stack too to give extra performance boost.

    Assume the following model & association:

    Code:
        Ext.define('User', {
            extend: 'Ext.data.Model',
            fields: ['id'],
            hasMany: {model: 'Order', name: 'orders'}
        });
    
    
        Ext.define('Order', {
            extend: 'Ext.data.Model',
            fields: ['id']
        });
    I've set up a test page (http://jsbin.com/ebagok/4) which uses chrome profiler hooks to kick off and stop the profiler on clicking the button and will then load 50 times the same 200 records from a JSON string using the loadRawData api.

    In the below screenshot of the profiler you can see that the top item (~30%) is spent creating and invoking a new Function for each associated instance.

    bad-code-ext-reader.jpg

    The code for this function is in Ext.data.reader.Reader.js and looks like:

    Code:
        buildRecordDataExtractor: function() {
            var me = this,
                modelProto = me.model.prototype,
                templateData = {
                    clientIdProp: modelProto.clientIdProperty,
                    fields: modelProto.fields.items
                };
    
    
            me.recordDataExtractorTemplate.createFieldAccessExpression = me.accessExpressionFn;
            // Here we are creating a new Function and invoking it immediately in the scope of this Reader
            // It declares several vars capturing the configured context of this Reader, and returns a function
            // which, when passed a record data object, a raw data row in the format this Reader is configured to read,
            // and the record which is being created, will populate the record's data object from the raw row data.
            return Ext.functionFactory(me.recordDataExtractorTemplate.apply(templateData)).call(me);
        },
    From what I can tell if you're bulk loading in a dataset then this function created from the function factory can be re-used and doesn't need to be recreated each time. Therefore I created a patch which caches the function against a key (the string of the function) and then uses the cached function each time. To make sure we don't leak memory we also clear the cached value and it's key out on the next setTimeout (since we'll be in a tight loop creating instances this is fine, we can take the hit of creating the function on each new load). After applying the patch the performance increases massively and drops to 1ms

    bad-code-ext-reader-fixed.jpg

    which equates to 0.04% of the total time. The patch looks like:

    Code:
        Ext.define('PatchedReader', {
          override: 'Ext.data.reader.Reader',
          statics: {
            templateCache: {}
          },
    
    
          buildRecordDataExtractor: function() {
            var me = this,
                modelProto = me.model.prototype,
                templateData = {
                    clientIdProp: modelProto.clientIdProperty,
                    fields: modelProto.fields.items
                };
    
    
            me.recordDataExtractorTemplate.createFieldAccessExpression = me.accessExpressionFn;
    
    
            var fnString = me.recordDataExtractorTemplate.apply(templateData),
                cachedFn = this.getCachedFunction(fnString);
            // Use caching to avoid the performance hit of a new function
            if (cachedFn) {
                return cachedFn.call(me);
            } else {
                // Here we are creating a new Function and invoking it immediately in the scope of this Reader
                // It declares several vars capturing the configured context of this Reader, and returns a function
                // which, when passed a record data object, a raw data row in the format this Reader is configured to read,
                // and the record which is being created, will populate the record's data object from the raw row data.
                var fn = Ext.functionFactory(fnString);
                this.addEntryToCacheAndStartTimer(fnString, fn);
                return fn.call(me);
            }
          },
    
    
          getCachedFunction: function(fnString) {
            var cache = Ext.data.reader.Reader.templateCache;
            return cache[fnString];
          },
      
          addEntryToCacheAndStartTimer: function(key, value) {
              Ext.data.reader.Reader.templateCache[key] = value;
              new Ext.util.DelayedTask().delay(1, this.removeEntryFromCache, this, [key]);
          },
      
          removeEntryFromCache: function(key) {
              delete Ext.data.reader.Reader.templateCache[key];
          }
      });
    And I've set up a similar test page using it here: http://jsbin.com/ebagok/5

    A
    fter fixing this performance issue you're left with some performance problems in addListener which I've not looked at, but this has certainly improved our performance on the larger data sets.

    Matt

  2. #2
    Sencha - Support Team slemmon's Avatar
    Join Date
    Mar 2009
    Location
    Boise, ID
    Posts
    5,380
    Vote Rating
    201
    slemmon has much to be proud of slemmon has much to be proud of slemmon has much to be proud of slemmon has much to be proud of slemmon has much to be proud of slemmon has much to be proud of slemmon has much to be proud of slemmon has much to be proud of

      0  

    Default


    Thanks for the report! I have opened an entry in our tracker.

  3. #3
    Sencha User
    Join Date
    Apr 2012
    Posts
    1
    Vote Rating
    0
    arisk is on a distinguished road

      0  

    Default Patched Reader for Ext Js v. 4.2.1

    Patched Reader for Ext Js v. 4.2.1


    Code:
    Ext.define('PatchedReader', {
        override: 'Ext.data.reader.Reader',
        statics: {
            templateCache: {}
        },
    
    
        buildRecordDataExtractor: function() {
            var me = this,
                modelProto = me.model.prototype,
                templateData = {
                    clientIdProp: modelProto.clientIdProperty,
                    fields: modelProto.fields.items
                };
    
    
            var fnString = me.recordDataExtractorTemplate.apply(templateData),
                cachedFn = this.getCachedFunction(fnString);
            // Use caching to avoid the performance hit of a new function
            if (cachedFn) {
                return cachedFn.call(me);
            } else {
                // Here we are creating a new Function and invoking it immediately in the scope of this Reader
                // It declares several vars capturing the configured context of this Reader, and returns a function
                // which, when passed a record data object, a raw data row in the format this Reader is configured to read,
                // and the record which is being created, will populate the record's data object from the raw row data.
                var fn = Ext.functionFactory(fnString);
                this.addEntryToCacheAndStartTimer(fnString, fn);
                return fn.call(me);
            }
        },
    
    
        getCachedFunction: function(fnString) {
            var cache = Ext.data.reader.Reader.templateCache;
            return cache[fnString];
        },
    
    
        addEntryToCacheAndStartTimer: function(key, value) {
            Ext.data.reader.Reader.templateCache[key] = value;
            new Ext.util.DelayedTask().delay(1, this.removeEntryFromCache, this, [key]);
        },
    
    
        removeEntryFromCache: function(key) {
            delete Ext.data.reader.Reader.templateCache[key];
        }
    });

  4. #4
    Sencha Premium Member jordandev's Avatar
    Join Date
    Feb 2011
    Location
    BC, Canada
    Posts
    42
    Vote Rating
    4
    jordandev is on a distinguished road

      0  

    Default


    Nice fix.

    We are having a large performance hit caused by associations but for some reason most of it is not with buildRecordDataExtractor but this does save a few seconds so thanks for the solution.