Results 1 to 8 of 8

Thread: [4.1 RC1] Remote sort from a buffered store fails on small data sets.

    Success! Looks like we've fixed this one. According to our records the fix was applied for EXTJS-5823 in a recent build.
  1. #1
    Sencha User
    Join Date
    May 2011
    Posts
    8

    Default [4.1 RC1] Remote sort from a buffered store fails on small data sets.

    Ext version tested:
    • Ext 4.1.0 RC1
    Browser versions tested against:
    • FireFox 11.0
    Description:
    • When a grid is displaying data from an Ext.data.Store with buffered == true and remoteSort == true, sorting a column has no effect when the number of records displayed is small, probably any time the number of records is less than pagesize. (Although the buffered store with remote store is meant for large datasets, this case may be typical once a filter has been applied that narrows the data set down to sub-pagesize size.)
    Steps to reproduce the problem:
    • Run the example in examples/grid/infinite-scroll-grid-tuner.html.
    • Note that after loading the data, sorting a column works.
    • Change the code in examples/griddev/infinite-scroll-grid-tuner.js so that numRecords is 50 instead of 50*1000.
    • Reload the page and notice that sorting a column has no effect.
    Debugging already done:
    • For the remoteSort buffered case, Store.doSort() clears the page cache and calls prefetchPage(), assuming that the sort will be applied remotely as part of the fetch.
    • Unfortunately prefetchPage() returns early if (me.getCount() === total). Presumably the assumption was that if the data was already there, there is no need to bother the server. However, since the prefetch was only called to have the server do the sort, the amount of data already present doesn't matter. (Also, since doSort cleared the page cache, not doing the prefetch leaves the store in a slightly odd state.)
    ?

  2. #2
    Sencha User Animal's Avatar
    Join Date
    Mar 2007
    Location
    Bédoin/Nottingham
    Posts
    30,892

    Default

    This is not a bug.

    The simulated Ajax object does not perform sorting, so if you are already on page 1, then you will see no change in the data.

    If you are on page 413, then you will see a change because it will request a sort, and flip back to page 1. But the data will not be sorted.

    Of course in the real world, when it's talking to your server, and requesting that the data be sorted, then it will return sorted data.

  3. #3
    Sencha User
    Join Date
    May 2011
    Posts
    8

    Default Ajax never sees the request.

    Animal, thanks for the prompt response, but I still think this is a bug.

    The point is that if the store has already loaded the total available records, prefetchPage() is preventing an Ajax call, thereby preventing any remote sorting, whether it's via my real world server or a simulated one.

    I think this case is important once filtering is incorporated. Even with a large initial data set, applying filters could result just these 50 remaining records. My understanding of the "total" returned by the proxy in the ResultSet is the total available via paging, not the total number of records before filtering. If that's true, then the 50 filtered records would all be loaded in the buffered store, and the buffered store's totalCount would also be 50, so prefetchPage() would prevent any Ajax call from doing any sorting.

    Thanks.

  4. #4
    Sencha User Animal's Avatar
    Join Date
    Mar 2007
    Location
    Bédoin/Nottingham
    Posts
    30,892

    Default

    prefetchPage is and Ajax call.

    The total returned in the return packet is the total size of the current query whatever that query is. So if there are filters which mean that there are only 50 records returned (even from a 1,000,000 row table), then the total dataset size is 50.

    And a sort on that will send both the sort data, and filter data, and the query will return the same 50 records, but in sorted order.

    It all works. You just have to code your server do return exactly what is asked of it.

  5. #5
    Sencha User
    Join Date
    May 2011
    Posts
    8

    Default

    Thanks for the clarification on the total returned from the query. I've confirmed that is how my server is behaving.

    The problem I'm trying to describe is that there are cases where Ext.data.Store.prefetchPage() does not contact the server, so there is no chance for the server to do any sorting, filtering or anything else.

    Below is the code for Store.prefetchPage(). Note the line I commented with "ISSUE:...". In the case where a filter has been applied resulting in 50 records, the store has all the records and the grid happily displays them. However, when a column header is clicked to initiate a *new* sort, that line of code prevents the server from being queried to perform the sort, because store.getCount() and total are both 50.

    Note that in the infinite-scroll-grid-tuner example, column sorting *does* work. But if you make numRecords 50, it does not work due that line of code preventing the server from being queried.

    Code:
        /**
         * Prefetches a page of data.
         * @param {Number} page The page to prefetch
         * @param {Object} options (Optional) config object, passed into the Ext.data.Operation object before loading.
         * See {@link #method-load}
         */
        prefetchPage: function(page, options) {
            var me = this,
                pageSize = me.pageSize || me.defaultPageSize,
                start = (page - 1) * me.pageSize,
                end = start + pageSize,
                total = me.getTotalCount();
                
            if (total) {
                end = Math.min(end, total);
            }
    
    
            // No more data to prefetch.
            if (me.getCount() === total) {
                return;    // ISSUE:  this line prevents querying the server.
            }
    
    
            // Copy options into a new object so as not to mutate passed in objects
            me.prefetch(Ext.apply({
                page     : page,
                start    : start,
                limit    : pageSize
            }, options));
        },
    Thanks! I do appreciate the time you've taken to look at this.

  6. #6
    Ext JS Premium Member veenvliet.morion's Avatar
    Join Date
    Oct 2009
    Location
    Netherlands
    Posts
    24

    Default

    I've exactly the same issue.
    Using Ext.Direct.
    I think it is a bug. 'Cause the store will not prefetch, because it thinks it has all the data. Which is true. But we want new data.

    Is this really rightfully closed?

  7. #7
    Sencha User Animal's Avatar
    Join Date
    Mar 2007
    Location
    Bédoin/Nottingham
    Posts
    30,892

    Default

    I see. Upon remote filter request, it should clear it's old, probably stale totalCount.

    Does this override work for you?

    Code:
    Ext.define('BufferStoreFilterFix', {
        override: 'Ext.data.Store',
    
        filter: function(filters, value) {
            if (Ext.isString(filters)) {
                filters = {
                    property: filters,
                    value: value
                };
            }
    
            var me = this,
                decoded = me.decodeFilters(filters),
                i = 0, count,
                doLocalSort = me.sorters.length && me.sortOnFilter && !me.remoteSort,
                length = decoded.length;
    
            for (; i < length; i++) {
                me.filters.replace(decoded[i]);
            }
    
            if (me.remoteFilter) {
                // For a buffered Store, we have to clear the prefetch cache because the dataset will change upon filtering.
                // Then we must prefetch the new page 1, and when that arrives, reload the visible part of the Store
                // via the guaranteedrange event
                if (me.buffered) {
                    count = me.getCount();
                    me.pageMap.clear();
                    me.totalCount = 0;
                    me.prefetchPage(1, {
                        callback: function(records, operation, success) {
                            if (success) {
                                me.onGuaranteedRange({
                                    prefetchStart: 0,
                                    prefetchEnd: Math.min(count, records.length)
                                });
                            }
                        }
                    });
                } else {
                    // Reset to the first page, the filter is likely to produce a smaller data set
                    me.currentPage = 1;
                    //the load function will pick up the new filters and request the filtered data from the proxy
                    me.load();
                }
            } else {
                /**
                 * @property {Ext.util.MixedCollection} snapshot
                 * A pristine (unfiltered) collection of the records in this store. This is used to reinstate
                 * records when a filter is removed or changed
                 */
                if (me.filters.getCount()) {
                    me.snapshot = me.snapshot || me.data.clone();
                    me.data = me.data.filter(me.filters.items);
    
                    if (doLocalSort) {
                        me.sort();
                    } else {
                        // fire datachanged event if it hasn't already been fired by doSort
                        me.fireEvent('datachanged', me);
                        me.fireEvent('refresh', me);
                    }
                }
            }
        }
    });

  8. #8
    Ext JS Premium Member veenvliet.morion's Avatar
    Join Date
    Oct 2009
    Location
    Netherlands
    Posts
    24

    Default

    Yep, that seems to work!

    I added totalCount = 0 to the sort function aswell. 'Cause I have the same problem with sorting.
    PHP Code:
    // because prefetchData is stored by index
    // this invalidates all of the prefetchedData
    sort: function () {   
     var 
    me this,        
    prefetchData me.pageMap;
        if (
    me.buffered) {
            if (
    me.remoteSort) {
                
    prefetchData.clear();
                
    me.totalCount 0// Added to do a reload to the server on sort            me.callParent(arguments);
            
    } else {
                
    me.callParent(arguments);
            }
        } else {
            
    me.callParent(arguments);
        }


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •