Buffered Scrolling Grid example and sort bug

[ericwaldheim]

in Ext 4.1 beta 2, the Buffered Scrolling grid example.

If I click on the Name column header to sort the names:

instead of getting "1 Tommy White" in the first row, I get "37 Tommy Spencer".
"1 Tommy White" appears about 15 rows down.
If I scroll down and back up, it is corrected, "1 Tommy White" appears in the first row.

[mitchellsimoens]

Buffered grid sorting should be disabled.

[ericwaldheim]

Okay, thanks.

Any chance that this feature will be available at some point for grids/stores with local (prefetched, cached) data like this one? (Or hints as to what might need to be modified to make this possible?) Thanks.

[ericwaldheim]

Here's my hack/solution. No idea if this solution works in general, but it works for what I'm trying to do. (This is a drop in replacement for examples/grid/buffer-grid.js. RowNumberer has been removed because I do not use it.)

Ext.Loader.setConfig({enabled: true});

Ext.Loader.setPath('Ext.ux', '../ux/');
Ext.require([
    'Ext.grid.*',
    'Ext.data.*',
    'Ext.util.*',
    'Ext.grid.PagingScroller'
]);

Ext.define('Employee', {
    extend: 'Ext.data.Model',
    fields: [
       {name: 'eid', type:'int'},
       {name: 'rating', type: 'int'},
       {name: 'salary', type: 'float'},
       {name: 'name'}
    ]
});

Ext.onReady(function(){
    /**
     * Returns an array of fake data
     * @param {Number} count The number of fake rows to create data for
     * @return {Array} The fake record data, suitable for usage with an ArrayReader
     */
    function createFakeData(count) {
        var firstNames   = ['Ed', 'Tommy', 'Aaron', 'Abe', 'Jamie', 'Adam', 'Dave', 'David', 'Jay', 'Nicolas', 'Nige'],
            lastNames    = ['Spencer', 'Maintz', 'Conran', 'Elias', 'Avins', 'Mishcon', 'Kaneda', 'Davis', 'Robinson', 'Ferrero', 'White'],
            ratings      = [1, 2, 3, 4, 5],
            salaries     = [100, 400, 900, 1500, 1000000];

        var data = [];
        for (var i = 0; i < (count || 25); i++) {
            var ratingId    = Math.floor(Math.random() * ratings.length),
                salaryId    = Math.floor(Math.random() * salaries.length),
                firstNameId = Math.floor(Math.random() * firstNames.length),
                lastNameId  = Math.floor(Math.random() * lastNames.length),

                rating      = ratings[ratingId],
                salary      = salaries[salaryId],
                name        = Ext.String.format("{0} {1}", firstNames[firstNameId], lastNames[lastNameId]);

            data.push({
					 eid: data.length + 0,
                rating: rating,
                salary: salary,
                name: name
            });
        }
        return data;
    }
    // create the Data Store
    var store = Ext.create('Ext.data.Store', {
        id: 'store',
        remoteSort:true,  ////// HACK: stop buffered/remoteSort:false from marking columns unsortable
        pageSize: 50,
        // allow the grid to interact with the paging scroller by buffering
        buffered: true,
        // never purge any data, we prefetch all up front
        purgePageCount: 0,
        model: 'Employee',
        proxy: {
            type: 'memory'
        }
    });

    var grid = Ext.create('Ext.grid.Panel', {
        width: 700,
        height: 500,
        title: 'Bufffered Grid of 5,000 random records',
        store: store,
        verticalScroller: {
            xtype: 'paginggridscroller',
            activePrefetch: false
        },
        loadMask: true,
        disableSelection: true,
        invalidateScrollerOnRefresh: false,
        viewConfig: {
            trackOver: false
        },
        // grid columns
        columns:[{
            text: 'ID',
            width: 60,
            sortable: true,
            dataIndex:'eid'
        },{
            text: 'Name',
            flex:1 ,
            sortable: true,
            dataIndex: 'name'
        },{
            text: 'Rating',
            width: 125,
            sortable: true,
            dataIndex: 'rating'
        },{
            text: 'Salary',
            width: 125,
            sortable: true,
            dataIndex: 'salary',
            align: 'right',
            renderer: Ext.util.Format.usMoney
        }],
        renderTo: Ext.getBody()
    });

    store.remoteSort = false; // HACK: now that columns are sortable, set remoteSort to false

    var data = createFakeData(5000),
        ln = data.length,
        records = [],
        i = 0;
    for (; i < ln; i++) {
        records.push(Ext.create('Employee', data[i]));
    }
    store.cacheRecords(records);

    store.guaranteeRange(0, 49);

});

Ext.define('override.Ext.data.Store',
{
	override:'Ext.data.Store',
    sort: function() {
        var me = this,
            prefetchData = me.prefetchData,
            sorters,
            start,
            end,
            range;

		 var parentSort = Ext.data.Store.prototype.superclass.sort;

        if (me.buffered) {
            if (me.remoteSort) {
                prefetchData.clear();
//                 me.callParent(arguments); // HACK
                parentSort.call(me, arguments); // HACK
            } else {
                if (me.totalCount == prefetchData.getCount()) // HACK
                { // HACK
                    var newSorters = me.decodeSorters([arguments[0]]); // HACK
                    me.sorters.clear(); // HACK
                    me.sorters.addAll(newSorters); // HACK
                } // HACK
				
                sorters = me.getSorters();
                start = me.guaranteedStart;
                end = me.guaranteedEnd;

                if (sorters.length) {
                    prefetchData.sort(sorters);
                    range = prefetchData.getRange();
                    prefetchData.clear();
                    me.cacheRecords(range);
                    delete me.guaranteedStart;
                    delete me.guaranteedEnd;
                    me.guaranteeRange(start, end);
                }
//                 me.callParent(arguments); // HACK
                   parentSort.call(me, arguments); // HACK
            }
        } else {
//             me.callParent(arguments); // HACK
                parentSort.call(me, arguments); // HACK

        }
    }
});

[ericwaldheim]

Here's a working example of sorting/filtering buffered local data in a grid.
This works on extjs 4.1 b3

Ext.Loader.setConfig({enabled: true});

Ext.Loader.setPath('Ext.ux', '../ux/');
Ext.require([
    'Ext.grid',
    'Ext.data.*',
    'Ext.util.*',
    'Ext.grid.PagingScroller'
]);

Ext.define('Employee', {
    extend: 'Ext.data.Model',
    fields: [
       {name: 'id', type:'int'},
       {name: 'rating', type: 'int'},
       {name: 'salary', type: 'float'},
       {name: 'name'}
    ]
});


Ext.onReady(
	function()
	{
// 		Returns an array of fake data
// 		@param {Number} count The number of fake rows to create data for
// 		@return {Array} The fake record data, suitable for usage with an ArrayReader
		function createFakeData(count) {
			var firstNames   = ['Ed', 'Tommy', 'Aaron', 'Abe', 'Jamie', 'Adam', 'Dave', 'David', 'Jay', 'Nicolas', 'Nige'],
         lastNames    = ['Spencer', 'Maintz', 'Conran', 'Elias', 'Avins', 'Mishcon', 'Kaneda', 'Davis', 'Robinson', 'Ferrero', 'White'],
         ratings      = [1, 2, 3, 4, 5],
         salaries     = [100, 400, 900, 1500, 1000000];
			
			var data = [];
			for (var i = 0; i < (count || 25); i++) {
            var ratingId    = Math.floor(Math.random() * ratings.length),
            salaryId    = Math.floor(Math.random() * salaries.length),
            firstNameId = Math.floor(Math.random() * firstNames.length),
            lastNameId  = Math.floor(Math.random() * lastNames.length),
				
            rating      = ratings[ratingId],
            salary      = salaries[salaryId],
            name        = Ext.String.format("{0} {1}", firstNames[firstNameId], lastNames[lastNameId]);
				
            data.push({
								 id: data.length + 1,
								 rating: rating,
								 salary: salary,
								 name: name
							 });
			}
			return data;
		}
		// create the Data Store
		var store = Ext.create('my.BufferedLocalStore', {
										  id: 'store',
										  model: 'Employee',
										  proxy: {
											  type: 'memory'
										  }
									  });
		
		var grid = Ext.create(
			'Ext.grid.Panel',
			{
				id:'myGrid',
				width: 700,
				height: 500,
				title: 'Buffered Grid of 5,000 random records',
				store: store,
				verticalScroller: {
					xtype: 'paginggridscroller',
					activePrefetch: false
				},
				disableSelection: true,
				invalidateScrollerOnRefresh: false,
				viewConfig: {
					trackOver: false
				},
				// grid columns
				columns:[
					{text: 'ID',
					 width: 60,
					 sortable: true,
					 dataIndex:'id'
					},
					{text: 'Name',
					 flex:1 ,
					 sortable: true,
					 dataIndex: 'name'
					},
					{text: 'Rating',
					 width: 125,
					 sortable: true,
					 dataIndex: 'rating'
					},
					{text: 'Salary',
					 width: 125,
					 sortable: true,
					 dataIndex: 'salary',
					 align: 'right',
					 renderer: Ext.util.Format.usMoney
					}],
				renderTo: Ext.getBody(),
				dockedItems:
				{xtype:'toolbar',
				 dock:'bottom',
				 items:[
					 {id:'f1', text:'Filter for AA',
					  enableToggle:true, toggleHandler:filterGrid},
					 {id:'f2', text:'Filter for High Rating',
					  enableToggle:true, toggleHandler:filterGrid},
					 {id:'f3', text:'Filter for Rich',
					  enableToggle:true, toggleHandler:filterGrid},
					 {text:'Clear Filters',
					  handler: function() {
						  Ext.getCmp('f1').toggle(false);
						  Ext.getCmp('f2').toggle(false);
						  Ext.getCmp('f3').toggle(false);
						  filterGrid();
					  }
					 },
					 "->",
					 {xtype:'tbtext', id:'dcount'}
				 ]
				}
			});

		
		var data = createFakeData(5000),
      ln = data.length,
      records = [],
      i = 0;
		for (; i < ln; i++) {
			records.push(Ext.create('Employee', data[i]));
		}
		store.cacheRecords(records);
		
		store.guaranteeRange(0, 49);
		
		store.remoteSort = false;
	});

function filterGrid()
{
	var h = {
		f1:function(r) { return r.get('name').match(/^Abe Avins/); },
		f2:function(r) { return r.get('rating') == 5; },
		f3:function(r) { return r.get('salary') > 200000; }
	};
	
	var keys = ['f1', 'f2', 'f3'];
	var fs = keys.filter(function(k) { return Ext.getCmp(k).pressed; })
	.map(function(k) { return h[k]; });
	
	var store = Ext.getCmp('myGrid').getStore();
	if (fs.length)
	{
		var f = function(r)
		{
			var pass = true;
			for (var i = 0; pass && i < fs.length; ++i)
			{
				pass = fs[i](r);
			}
			return pass;
		};
		store.filterBy(f);
	}
	else
	{
		store.clearFilter();
	}
	
	var grid = Ext.getCmp('myGrid');
	var count = grid.getStore().prefetchData.getCount();
	Ext.getCmp('dcount').setText("row count: " + count);
}

Ext.define('my.BufferedLocalStore',
{
   extend:'Ext.data.Store',
	
   buffered:true,
   pageSize:50,
   purgePageCount:0,
   remoteSort:false,

   sort: function() {
        var me = this,
            prefetchData = me.prefetchData,
            sorters,
            start,
            end,
            range;

      if (me.totalCount == prefetchData.getCount())
      {
         var newSorters = me.decodeSorters([arguments[0]]);
         me.sorters.clear();
         me.sorters.addAll(newSorters);
      }
		
      sorters = me.getSorters();
      start = me.guaranteedStart;
      end = me.guaranteedEnd;
		
      if (sorters.length) {
         prefetchData.sort(sorters);
         range = prefetchData.getRange();
         prefetchData.clear();
         me.cacheRecords(range);
         delete me.guaranteedStart;
         delete me.guaranteedEnd;
         me.guaranteeRange(start, end);
      }
		var parentSort = Ext.data.Store.prototype.superclass.sort;
      parentSort.call(me, arguments);
   },

   filterBy: function(fn) {
		var me = this;
		var filter = new Ext.util.Filter({filterFn:fn});
		me.prefetchSnapshot = me.prefetchSnapshot || me.prefetchData.clone();
		me.setPrefetchData(me.prefetchSnapshot.filter([filter]));
   },

   clearFilter: function(suppressEvent) {
		var me = this;

		if (!!me.prefetchSnapshot)
		{
			var pfd = me.prefetchSnapshot;
         delete me.prefetchSnapshot;
			me.setPrefetchData(pfd, suppressEvent);
		}
	},
	
	setPrefetchData:function(data, suppressEvent)
	{
		var me = this;
      var range = data.getRange();
      me.prefetchData.clear();
		delete me.totalCount;
      delete me.guaranteedStart;
      delete me.guaranteedEnd;
      me.cacheRecords(range);
		me.guaranteeRange(0, me.pageSize-1);
      if (suppressEvent !== true) {
         me.fireEvent('datachanged', me);
			me.fireEvent('refresh', me);
      }
	}
});

// OVERRIDE extjs 4.1 b3 functions to get buffered local data grid to work

Ext.define(
	'my.Override.Table',
{
	override:'Ext.panel.Table',
   initComponent: function() {
      var me = this;

		var fixSort = me.store.buffered && !me.store.remoteSort;
		var save = [];
      if (fixSort) {
         save = me.columns.map(function(c) { return c.sortable; });
		}

			// PARENT FUNCTION TURNS OFF SORTABLE FOR COLUMNS
      var rv = me.callOverridden(arguments);

      if (fixSort) {
			Ext.Array.each(me.columns, function(c, i) { c.sortable = save[i]; });
      }

		return rv;
	}
});

Ext.define('my.Override.grid.PagingScroller',
{
	override:'Ext.grid.PagingScroller',
    onViewRefresh: function() {
        var me = this,
            newScrollHeight,
            view = me.view,
            viewEl = view.el.dom,
            store = me.store,
            rows,
            newScrollOffset,
            scrollDelta,
            table,
            tableTop;

        if (!store.getCount()) {
            return;
        }

		 // COMMENT OUT SO buffered local data WORKS
//         if (store.getCount() === store.getTotalCount()) {
//             return me.disabled = true;
//         } else {
//             me.disabled = false;
//         }

        me.stretcher.setHeight(newScrollHeight = me.getScrollHeight());

        
        
        if (me.scrollProportion !== undefined) {
            table = me.view.el.child('table', true);
            me.scrollProportion = view.el.dom.scrollTop / (newScrollHeight - table.offsetHeight);
            table = me.view.el.child('table', true);
            table.style.position = 'absolute';
            table.style.top = (me.scrollProportion ? (newScrollHeight * me.scrollProportion) - (table.offsetHeight * me.scrollProportion) : 0) + 'px';
        }
        else {
            table = me.view.el.child('table', true);
            table.style.position = 'absolute';
            table.style.top = (tableTop = (store.guaranteedStart||0) * me.rowHeight) + 'px';

            
            if (me.scrollOffset) {
                rows = view.getNodes();
                newScrollOffset = -view.el.getOffsetsTo(rows[me.commonRecordIndex])[1];
                scrollDelta = newScrollOffset - me.scrollOffset;
                me.position = (view.el.dom.scrollTop += scrollDelta);
            }

            
            
            
            
            
            
            else if ((tableTop > viewEl.scrollTop) || ((tableTop + table.offsetHeight) < viewEl.scrollTop + viewEl.clientHeight)) {
                me.position = viewEl.scrollTop = tableTop;
            }
        }
    }
});

[Animal]

Filtering and Sorting are supported in 4.1.0RC1

But only in remote mode. Filtering or sorting the local 20, 30 or so rows of a 100,000 row dataset is not valid.

The infinite scroll example illustrates sorting. We'd need to rewrite the web service which provides the data to implement remote filtering in that example, but I think it would be a good idea to do that.

Executing a remote filter or sort reverts to page 1 of the dataset.

[ericwaldheim]

Thanks for the response and I appreciate the work you're doing on buffered store / infinite grid.

Just to be clear, my example has it working in *local* mode. It sorts and filters all the data locally.
(Obviously when the data set gets too large this is a bad idea, but for the 1000 to 5000 rows I need, I find this simple and effective.)
(The code uses a BufferedLocalStore derived class and a couple of extjs tweaks where things are hard-wired to make this not work. All of this is included in the example.)

[Animal]

Local sorting and filtering will not be supported in 4.1.0RC1

The pefetch buffer is not just a blob of records. It is a map of pages keyed by page number.

[ericwaldheim]

Ah, now I understand, thanks.

Ouch. I'm disappointed that this use-case will not be supported. Back to the drawing board.

[dongryphon]

Ah, now I understand, thanks.

Ouch. I'm disappointed that this use-case will not be supported. Back to the drawing board.

We talk a lot about storing all the data client side, but in this case, it may be best to let the cache of the store manage this and let the server do the work of sorting/filtering. If the sort or filter changes, the store will drop the cache and refetch from the server using the new criteria.

If you dial up the cache sizes, you can get the best of both worlds: lots of data locally (fetched on demand) and the tricky stuff handled by the server.

Maybe I misunderstand your use case though.

This may help as well:

http://www.sencha.com/forum/showthre...rolling-in-4.1