store filter performance issue

[motih]

Hi,

So i am building an iphone like contacts application.

I have a store of ~2600 contacts all displayed in a list, and a search box to filter the list.
When typing in the search box, i apply filter for every keyup event using filterBy method - works great.

However, when clearing the search box, or clicking on backspace in the search box, i am calling clearFilter on the store, which takes a very long time (~6 seconds) due to its size.

Is there an optimized way of clearing the filters or removing only the last applied filter?
Or do we have a performance issue here in the framework level?

Thanks,
Moti.

[DJFliX]

Hi,

So i am building an iphone like contacts application.

I have a store of ~2600 contacts all displayed in a list, and a search box to filter the list.
When typing in the search box, i apply filter for every keyup event using filterBy method - works great.

However, when clearing the search box, or clicking on backspace in the search box, i am calling clearFilter on the store, which takes a very long time (~6 seconds) due to its size.

Is there an optimized way of clearing the filters or removing only the last applied filter?
Or do we have a performance issue here in the framework level?

Thanks,
Moti.

Are you adding a filter with every key press? I'm not sure if removing multiple filters would result in a performance decrease when removing them...

[motih]

Hi,

I am catching the keyup event of the search box, and i apply a filter function using the filterBy method - in order to filter the sore based on the current value of the search box (which changes with every keypress).

While adding filters, the performance is great.

However, when the user clicks on backspace, i need to make sure that that last filter is removed.
Since i couldn;t figure out a way to make it work, i just clear all the filters and then add a filter based on the current value of the search box.

When the search box clear button is presses - i have to clear all filters.

The clearFilter method which i am using is very slow since i have a ~2600 records store.
The question is if there is a better way to remove the filters or last applied filter - or maybe the implementation of clearFilter is not optimized for large stores, and Sencha team can improve this.

Thanks,
Moti.

[DJFliX]

Okay, but if you apply one filter and then remove it, is the performance still bad? Otherwise you could clear the filter on every key press. It makes no sense to filter for both a, ap, app, appl and apple, maybe if you apply one filter the performance increases? It's worth trying.

[motih]

I have tried to apply only one filter and clear it - performance is still bad (~3-4 seconds to clear).
So i guess it is related to the store size (~2600 records in my case) rather than the number of filters being cleared.

I wonder if this can somehow be optimized by Sencha team.

Thanks,
Moti.

[Bunchofstring]

Here's my idea: You seem to suggest that applying filters is fast, but clearing them is slow. By that logic, if the list only responds to applying filters to the bound store, performance would be fine. So... what if you had two stores with the same collection of records, and use one like a buffer?

The following approach assumes that a list gets updated quickly when it is bound to a new store. On keyup, try this (untested/pseudo-code):

var boundStore = list.getStore();
//Set up on both stores so they can reference each other
var bufferStore = boundStore.getTwinStore();
//TRUE if backspace was pressed or the user inserted a character into the middle of the search string
var requiresClear = (newValue.indexOf(oldValue) == 0);

if(!requiresClear){
     //If filters are already in place for "a", it's probably more efficient to just add ones for "ap", "app", etc.
     boundStore.filter('name',newValue);
}else{
     list.setStore(bufferStore);
     bufferStore.filter('name',newValue);
     //Might help to do the clear it's own "thread" (i.e. setTimeout)
     boundStore.clearFilter(true);
}

That may be a crazy/foolish idea but it's worth a shot. One thing to watch out for is if the user pushes backspace a bunch of times (after the first backspace, the swapping and clearing would probably hurt performance).

[Nilanchala]

Here's my idea: You seem to suggest that applying filters is fast, but clearing them is slow. By that logic, if the list only responds to <em>applying </em>filters to the bound store, performance would be fine. So... what if you had two stores with the same collection of records, and use one like a buffer?<br>
<br>
The following approach assumes that a list gets updated quickly when it is bound to a new store. On keyup, try this (<strong>untested/pseudo-code</strong>):<br>

var boundStore = list.getStore();<br>
//Set up on both stores so they can reference each other<br>
var bufferStore = boundStore.getTwinStore();<br>
//TRUE if backspace was pressed or the user inserted a character into the middle of the search string<br>
var requiresClear = (newValue.indexOf(oldValue) == 0);<br>
<br>
if(!requiresClear){<br>
&nbsp;&nbsp; &nbsp; //If filters are already in place for "a", it's probably more efficient to just add ones for "ap", "app", etc.<br>
&nbsp;&nbsp; &nbsp; boundStore.filter('name',newValue);<br>
}else{<br>
&nbsp;&nbsp; &nbsp; list.setStore(bufferStore);<br>
&nbsp;&nbsp; &nbsp; bufferStore.filter('name',newValue);<br>
&nbsp;&nbsp; &nbsp; //Might help to do the clear it's own "thread" (i.e. setTimeout)<br>
&nbsp;&nbsp; &nbsp; boundStore.clearFilter(true);<br>
}

<br>
<br>
That may be a crazy/foolish idea but it's worth a shot. One thing to watch out for is if the user pushes backspace a bunch of times (after the first backspace, the swapping and clearing would probably <em>hurt</em> performance).

Hi, I have the similar problem. I feel your&nbsp;technique&nbsp;might help me to improve&nbsp;performance&nbsp;a bit. Being honest, I am newbie to Sencha, worked in couple of sencha touch projects but this one sucks. It has 1500 records, displayed on a list need to be filtered.

But, the filtering of data is quite slow. I wanted to use your logic to my code. But I really felt bit difficult to understand. If you could help me to fix my problem, I will really be very glad. Thanks in advance.

Here is my filter logic, while user presses key in text field.

 onSearchKeyUp: function(field) {
    	
        //get the store and the value of the field
        var value = field.getValue(),
        store = Ext.getStore("storeid");


        //first clear any current filters on thes tore
        store.clearFilter();


        //check if a value is set first, as if it isnt we dont have to do anything
        if (value) {
            //the user could have entered spaces, so we must split them so we can loop through them all
            var searches = value.split(' '),
                regexps = [],
                i;


            //loop them all
            for (i = 0; i < searches.length; i++) {
                //if it is nothing, continue
                if (!searches[i]) continue;


                //if found, create a new regular expression which is case insenstive
                //regexps.push(new RegExp(searches[i], 'i'));
                
                // to make starts with search
                regexps.push(new RegExp("^"+searches[i], 'i'));
            }


            //now filter the store by passing a method
            //the passed method will be called for each record in the store
            store.filter(function(record) {
                var matched = [];


                //loop through each of the regular expressions
                for (i = 0; i < regexps.length; i++) {
                    var search = regexps[i],
                   
                    didMatch = record.get('term').match(search);


                    //if it matched the first or last name, push it into the matches array
                    matched.push(didMatch);
                }


                //if nothing was found, return false (don't so in the store)
                if (regexps.length > 1 && matched.indexOf(false) != -1) {
                    return false;
                } else {
                    //else true true (show in the store)
                    return matched[0];
                }
            });
        }        
    }

[Nilanchala]

I have tried your logic but no luck. I am sure, by minimizing usages of clearFilter() has not improved anything on my performance.

[Bunchofstring]

Sad to hear you are struggling with this. My advice is to use the framework instead of inventing your own filtering system. It would be helpful if you describe your filter/matching rules using words instead of code, but if I understand your goal correctly, you should be able to get what you want like this (untested):

if(value){
    var match = '^(' + value.replace(/ /g,'|') + ')';
    store.filter('term',new RegExp(match,'gi'));
}

This filter will keep any record with a term that starts with one of the search terms (separated by spaces). It is not case sensitive.

[Nilanchala]

Sad to hear you are struggling with this. My advice is to use the framework instead of inventing your own filtering system. It would be helpful if you describe your filter/matching rules using words instead of code, but if I understand your goal correctly, you should be able to get what you want like this (untested):

if(value){
    var match = '^(' + value.replace(/ /g,'|') + ')';
    store.filter('term',new RegExp(match,'gi'));
}

This filter will keep any record with a term that starts with one of the search terms (separated by spaces). It is not case sensitive.

@Bunchofstring, Thanks for your reply. But If you look inside the logic of mine, I am using the filter of the default sencha store. There is no much difference between your logic from mine. However, I had put a blind try, but as expected it didn't make any difference to the search time.