Store id entries duplicated

[arael78]

I have a problem with the stores. Whenever I update an entry it the id are duplicated.

Here is my model.

Ext.define('MoodleMobApp.model.Course', {
        extend: 'Ext.data.Model',
        
        config: {
                fields: [
                        {name: 'id', type: 'auto'},
                        {name: 'name', type: 'string'},
                        {name: 'timemodified', type: 'int'},
                        {name: 'token', type: 'string'},
                        {name: 'modules', type: 'int', defaultValue: 0},
                        {name: 'isnew', type: 'boolean'},
                        {name: 'newmodules', type: 'int', defaultValue: 0},
                        {name: 'modulestatus', type: 'string', defaultValue: 'counting modules...'},
                ]
        }
});

Here is the store:

Ext.define('MoodleMobApp.store.Courses', {
        extend: 'Ext.data.Store',


        requires: [
                'MoodleMobApp.model.Course',
                'Ext.data.proxy.LocalStorage'
                ],


        config: {
                storeId: 'courses',
                model: 'MoodleMobApp.model.Course',
                autoLoad: true,
                proxy : {
                        id: 'course',
                        type: 'localstorage',
                }
        }
});

Here is the code that adds/updates new entries:

// update local courses store
this.each(
         function(entry) {
              if(courses_store.find('id', entry.getData().id) == -1) {
                              entry.getData().isnew = true;
                              courses_store.add( entry );
                      } else {
                              entry.getData().isnew = false;
                              // update the entry
                              courses_store.remove( entry );
                              courses_store.add( entry );
                      }
              }
        );




      // prepare to write
      courses_store.each(
              function() { 
                      this.setDirty();
              }
      );




      // store data
      courses_store.sync();

When I execute this code the number of the records is the same but the ids are duplicated.

So when I have three records courses_store.getCount() === 3 but if I check in the localstorage the course key contains this: 3,5,8,3,5,8.

If I execute this code one more time I get: 3,5,8,3,5,8,3,5,8.

Everything works but the course value keeps growing.

How can I fix this issue?

Thank you very much for any suggestions.

[mitchellsimoens]

When you update, why are you removing the record and then adding it? Why not use the set method on the record?

[arael78]

When you update, why are you removing the record and then adding it? Why not use the set method on the record?

I didn't use set() because I'm still learning this framework and I overlooked it. Thank you for your suggestion.

I found out what was the problem. I invoked setDirty() on all of my records. Invoking the sync() then duplicated the entries. Now I invoke setDirty() only on the records I pass to add() or set() and there are no duplicates anymore.

Why is it so setDirty() + sync() on the existing entries generates duplicates?
The same does not happens when I use set() or add().

Example: The "module" key contains 3,7,8 in the localstorage

This duplicates the entry 3:

modules_store.getById(3).setDirty();
modules_store.sync();

The value of "module" key is 3,7,8,3.

This does not duplicate the entry:

// here the updated_entry is a record with the id 3 and new values
updated_entry.setDirty();
modules_store.set(updated_entry);
modules_store.sync();

Here the value of "module" key is 3,7,8

[arael78]

Well. I was wrong. Every setDirty() and set() function after the sync duplicates values of the 'course' key and I don't know how to solve this problem.

What is the proper way to update the localestorage without having duplicates?

Every time I use sync() my storage ends up with duplicated values. My id keys are defined as integers and the values returned by getData() are integer but if I check in the web inspector the raw.id is a string. Is this normal?

[arael78]

Anyone has suggestions on how to solve this problem? =(

[alile]

I am having an identical issue (chrome, touch 2.1). I have a 'filter' field in my model that I am updating (so the user can flag items as 'saved'). When I sync (to local storage), the id fields in the chrome inspector are duplicated, but at no point does the getCount() call return other than the correct number.

A possibly important point - If I reload the page, without making any adjustments, and then use as directed I do not get the duplicating behavior. Possibly I have some sort of object persistence as I am loading in data from the server, duplicating the records to a local store, then unregistering the remote store. As I sync on unload, if I have not made any changes to the records, then there is no issue. But if I make changes, perhaps there is some lingering reference and that is what is getting stored. I tried to do a sync() and then a load() on the local store to replicate the reload behavior, to no avail. Merge/reload code below.

Note - I tried removing the setDirty() call, and the sync() saved nothing. So I am loading in the data from local storage the second time, but no way of knowing what is going on in memory.

- added removeAll() in an attempt to clean the data further. It does not fix anything, but it does prove that I am pulling from local storage only (updating the display without the 2nd load shows nothing and getCount() == 0).

Ext.getStore("ServerNotificationStore").each(function(record)    
{
        if(record.get("publishDate") != undefined)        
        {            this.getStore().insert(0,record);            
                      record.setDirty(true);
        }    
}, this);

Ext.StoreManager.unregister("ServerNotificationStore");
this.getStore().sync();
this.getStore().removeAll();   
this.getStore().on("load", this.handleStoreReloaded, this);    
this.getStore().load(); 

[munder]

Does anyone have a fix for this? I see pretty much the same behaviour. I check what's on the server and see there are ten records. I download those to a store and copy the records to an offline store, which is a sqllite table. I then take a look at the contents of the table using Google Chrome's developer tools and see multiple duplicates of each record. When I check how many are in the offline store using getCount(), it returns ten, the number I downloaded, despite the fact that there are clearly many more than ten. When I come to upload the records back to the server, despite the fact that getCount() assures me there are only ten records, it transpires that in fact all records in the table are uploaded to the server. Like Alile, I've tried removeAll() before copying records to the offline store, but it makes no difference. I've even tried dropping the table or the entire database just after restarting the app and before copying records to the offline store, but I still get duplicates.
As a result I find that my original strategy of finding a record by matching id and updating that record after the user had made changes had to be replaced by updating the data on a whole bunch of matching records.
Eventually I have such a slew of records that it takes forever to upload them to the server.

[bluehipy]

Do you have to use setDirty directly? I think the store + model handle this behind the scene. Problem might be on sync with LocalStorage.
Also pay attention on the primary key, if you have one, maybe specify it so the proxy knows what it should use as unique key. By default an internal key is generated and used, with a different ~ not integer auto increment type but rather a guid hash to assure unicity.

[munder]

Thanks for your suggestion, bluehipy. I have specified the idProperty on all other models in my app, which seems to work quite well, as Sencha appears to do the equivalent of INSERT IGNORE, i.e. it just silently disregards any attempt to duplicate a unique key. I do download data for other stores and copy them to local stores and don't have a problem with duplicate entries, so I think that could well be the solution. However, on the particular table which is causing me problems, uniqueness or a row would be provided by two fields and I can't figure out how to specify a composite key. The docs for idProperty refer to one field only. Any thoughts on how to provide a key composed of two fields?
Or, here's another thought. My users initially create data within the app, which are saved locally to the table 'answers'. When they eventually submit those data to the server, each row is given a unique autoincrement id by MySQL on the server. Since it's the server's job to provide the unique id, the app doesn't have an id field in the 'answers' model. When users want to modify their answers, the app downloads their data - and that's where I get all the duplicates. Would it be possible to have no id when the users are initially creating the data and use the server's id when data are downloaded for modification? Would I need to use two different models for that - one with an id field and one without? I think a composite key would be the best solution if only I knew how to specify one. If I do come up with a solution to this, I'll let you all know.

[munder]

Still struggling with this. As I seemed not to be getting duplicates with the stores for the models where I had specified an idProperty, I also decided to specify one for my offline store of the 'answers' table, filling that column with the values drawn from the online table if a user was editing answers previously uploaded to the server. Unfortunately, I still get at least ten duplicates of each record, with the result that the upload takes quite a while. The combination of MySQL and PHP I have on the server ensures that only the required dozen or so rows are inserted into the table, rather than the 120-odd which are uploaded, so there are no problems with data integrity; it's just a pain that it all takes so long to upload and process.