Saving objects that are linked hasMany relation with a single Store

[emilianm]

We are using a main object (A) that has a hasMany relation with a list of B objects.
For the A object we have a store with specific url api for reading and for updating data.

I have to mention that we have this URL configured only in the Store object A.
When we use the .load() method on the A store we receive a json stream that will contain also a list of B objects and this objects are very well fetched in B list form A object.


After we modify A object and some of the related B objects, and after we call .sync() method on AStore object we discover that the related B objects are not present in the generated json stream, so are not sent to the server together with A object.


Do you have any idea how can I synchronize with a single call the main A objects(along with the list of B objects) with the server?

Thank you very much for any idea regarding this.

[emilianm]

We verified in the documentation and in the source code and it seems that the reader is able to pre-load with data the objects linked with hasMany form the parent store, but the writer is not taking into account the hasMany relations, is putting in the final json stream only the fields directly attached to the model.

In conclusion, to achieve the synchronization of the main object and also of the associated objects we need to provide a custom implementation of the default Ext.data.writer.Writer

It seems that this method needs to take into account the has Many associations
getRecordData: function(record)

If you already encountered this issue please do not hesitate to share your information.
Thanks.

[chrisface]

This is something I had to do as well myself and I took the same approach of overriding the getRecordData function to take into account the associations. I also added some code to take care of deleted records as well. I'm adding a forDeletion property only for deleted records so that I can keep it out of the fields on the models.

Ext.data.writer.Json.override({
    /*
     * This function overrides the default implementation of json writer. Any hasMany relationships will be submitted
     * as nested objects. When preparing the data, only children which have been newly created, modified or marked for
     * deletion will be added. To do this, a depth first bottom -> up recursive technique was used.
     */
    getRecordData: function(record) {
        //Setup variables
        var me = this, i, association, childStore, data = record.data;

        //Iterate over all the hasMany associations
        for (i = 0; i < record.associations.length; i++) {
            association = record.associations.get(i);
            data[association.name] = null;
            childStore = record[association.storeName];

            //Iterate over all the children in the current association
            childStore.each(function(childRecord) {

                if (!data[association.name]){
                    data[association.name] = [];
                }

                //Recursively get the record data for children (depth first)
                var childData = this.getRecordData.call(this, childRecord);

                /*
                 * If the child was marked dirty or phantom it must be added. If there was data returned that was neither
                 * dirty or phantom, this means that the depth first recursion has detected that it has a child which is
                 * either dirty or phantom. For this child to be put into the prepared data, it's parents must be in place whether
                 * they were modified or not.
                 */
                if (childRecord.dirty | childRecord.phantom | (childData != null)){
                    data[association.name].push(childData);
                    record.setDirty();
                }
            }, me);

            /*
             * Iterate over all the removed records and add them to the preparedData. Set a flag on them to show that
             * they are to be deleted
             */
            Ext.each(childStore.removed, function(removedChildRecord) {
                //Set a flag here to identify removed records
                removedChildRecord.set('forDeletion', true);
                var removedChildData = this.getRecordData.call(this, removedChildRecord);
                data[association.name].push(removedChildData);
                record.setDirty();
            }, me);
        }

        //Only return data if it was dirty, new or marked for deletion.
        if (record.dirty | record.phantom | record.get('forDeletion')){
            return data;
        }
    }
});

I'm not sure if this code will work under every condition but it's been fine for me so far. If you have any suggestions for improvements or a better way of doing it I'd like to hear your thoughts.

[emilianm]

Thank you very much chrisface, we already put in place a custom Writer that is taking into accounts also the fields of hasMany relations. I will take a look at your solution and after we sill finalise ours we will publish here the code.

I think this should be integrated in the next ExtJs release.

[cstrong@arielpartners.com]

Many thanks to chrisface for the code posting.
As I understand it, the objective here is to read and write an object graph using only a single round trip each.
I think I am still missing one piece...

Let's pretend I am dealing with User and Post. User hasMany Post
I am showing the User information in one place, and I am displaying Post records in their own separate grid.
Now I add a new Post to the Post grid, and then call sync() on the User store.
Nothing happens. Why? Because the User record was never marked dirty even though a new Post object was added like so:
user.posts().add( {} ); // <-- this should mark the user instance dirty, but doesn't
I can fake it by adding a 'datachanged' listener or by manually:
user.setDirty();
I am thinking this linkage should be an option to set up in the hasManyAssociations createStore()
method, to enable this kind of behavior, thus reducing the number of round-trips to the server.
What am I missing?

[chrisface]

You're right cstrong, this wont send back your Posts. I do something a little bit different with my relationships which I should have mentioned for my override to work properly.

Instead of having a hasMany of "Post" models on the "User" I have a "UserPosts" model which I use instead. This model maps to the linker table that I use for saving which Posts belong to what User.

It looks a bit like this:
Id , UserId, PostId

What I then do is that I make my UserPosts model extend my Post model and I still have all the information I need. I also override the idProperty value to use the id for the linker table.

This means that when I'm assigning Posts to a User, what I'm actually doing is creating new UserPost models. They have a Post Id inside of them but no UserPost Id so they get picked up as phantom records.

I use my UserPosts model for handling the relationships and I use the Post model when doing CRUD on Posts. It does mean that there are some cases where they can't be used interchangeably but I either do a load using the Ids I have to get the correct kind of model or do a quick transformation between the objects which I have static methods for in the models.

It's probably not how Sencha intended the persistence of models to work but it was the only way I could get nested saving to work for me.

I ended up making more changes to the code as well because it caused some unforeseen bugs but there may be more that I've not seen yet

Ext.data.writer.Json.override({
    /*
     * This function overrides the default implementation of json writer. Any hasMany relationships will be submitted
     * as nested objects. When preparing the data, only children which have been newly created, modified or marked for
     * deletion will be added. To do this, a depth first bottom -> up recursive technique was used.
     */
    getRecordData: function(record) {
        //Setup variables
        var me = this, i, association, childStore, data = {};
        if(record.proxy.writer.writeAllFields){
            data = record.data;
        }
        else {
            var changes, name,  field, fields = record.fields, nameProperty = this.nameProperty, key;
            changes = record.getChanges();
            for (key in changes) {
                if (changes.hasOwnProperty(key)) {
                    field = fields.get(key);
                    name = field[nameProperty] || field.name;
                    data[name] = changes[key];
                }
            }
            if (!record.phantom) {
                // always include the id for non phantoms
                data[record.idProperty] = record.getId();
            }
        }


        //Iterate over all the hasMany associations
        for (i = 0; i < record.associations.length; i++) {
            association = record.associations.get(i);
            data[association.name] = [];
            childStore = record[association.storeName];


            //Iterate over all the children in the current association
            childStore.each(function(childRecord) {


                //Recursively get the record data for children (depth first)
                var childData = this.getRecordData.call(this, childRecord);


                /*
                 * If the child was marked dirty or phantom it must be added. If there was data returned that was neither
                 * dirty or phantom, this means that the depth first recursion has detected that it has a child which is
                 * either dirty or phantom. For this child to be put into the prepared data, it's parents must be in place whether
                 * they were modified or not.
                 */
                if (childRecord.dirty | childRecord.phantom | (childData != null)){
                    data[association.name].push(childData);
                    record.setDirty();
                }
            }, me);


            /*
             * Iterate over all the removed records and add them to the preparedData. Set a flag on them to show that
             * they are to be deleted
             */
            Ext.each(childStore.removed, function(removedChildRecord) {
                //Set a flag here to identify removed records
                removedChildRecord.set('forDeletion', true);
                var removedChildData = this.getRecordData.call(this, removedChildRecord);
                data[association.name].push(removedChildData);
                record.setDirty();
            }, me);
        }


        //Only return data if it was dirty, new or marked for deletion.
        if (record.dirty | record.phantom | record.get('forDeletion')){
            return data;
        }
    }
});


Ext.data.Store.override({
    remove: function(records, /* private */ isMove) {
        if (!Ext.isArray(records)) {
            records = [records];
        }


        /*
         * Pass the isMove parameter if we know we're going to be re-inserting this record
         */
        isMove = isMove === true;
        var me = this,
                sync = false,
                i = 0,
                length = records.length,
                isPhantom,
                index,
                record;


        for (; i < length; i++) {
            record = records[i];
            index = me.data.indexOf(record);


            if (me.snapshot) {
                me.snapshot.remove(record);
            }


            if (index > -1) {
                isPhantom = record.phantom === true;
                if (!isMove && !isPhantom) {
                    // don't push phantom records onto removed
                    record.set('forDeletion', true);
                    me.removed.push(record);
                }


                record.unjoin(me);
                me.data.remove(record);
                sync = sync || !isPhantom;


                me.fireEvent('remove', me, record, index);
            }
        }


        me.fireEvent('datachanged', me);
        if (!isMove && me.autoSync && sync) {
            me.sync();
        }
    }
});
Ext.data.TreeStore.override({
    onNodeRemove: function(parent, node) {
        var removed = this.removed;


        if (!node.isReplace && Ext.Array.indexOf(removed, node) == -1) {
            node.set('forDeletion', true);
            removed.push(node);
        }
    }
});

[cstrong@arielpartners.com]

chrisface thanks again for the updated posting.
Only issue I see is that belongsTo associations trigger null reference. I added a small fix to work around that issue.
I added an if condition directly underneath your for loop where you loop through the associations to check the type:

        //Iterate over all the hasMany associations
        for (i = 0; i < record.associations.length; i++) {
            association = record.associations.get(i);
            if (association.type == 'hasMany')  {   // <-- this avoids null reference issue
            .....// rest of method goes here
            }

I didn't grok your situation thoroughly, but I suspect you might be using a many-to-many approach (a join class), which might be why you didn't see the issue with belongsTo.
We have a one-to-many relationship, and the belongsTo association is quite helpful because, among other things, it automatically adds a reference to the parent model object inside the child.

We are namespacing our models, so we had to override several configuration parameters in order for the ext association code to generate reasonable getters and setters (otherwise you get nonsense like "Foo.models.MyModelBelongsToInstance" and "getFoo.model.Parent()" )

FWIW, here is the code I wrote to automatically mark the parent dirty when a child is modified, added, or deleted

    associations : [
        {
            type        : 'hasMany',
            model       : 'Foo.model.ChildOb',
            name        : 'childobs', // generates childobs() method
            foreignKey  : 'parent_id',
            storeConfig : {
                listeners: {
                    // Set parent object dirty if we added a new child.  
                    add: function(store, records, index, opts) {
                        var obj = records[0]; // NOTE not fully general but works for our case!
                        if (obj.get('id') === 0) {
                            this.setParentDirty(obj.get('parent_id'));
                        }
                    },
                    // Set parent object dirty if we removed an existing child
                    remove: function(store, obj, index, opts) {
                        if (obj.get('id') != 0) {
                            obj.parent.setDirty();
                        }
                    },
                    // Set parent object dirty if we updated an existing child
                    update: function(store, obj, index, opts) {
                        if (obj.get('id') != 0) {
                            obj.parent.setDirty();
                        }
                    }
                },
                setParentDirty: function(parent_id) {
                    var store = Ext.data.StoreManager.lookup('Parents');
                    var pool = store.getById(parent_id);
                    if (parent) {
                        parent.setDirty();
                    }
                }
            }
        },

Here is my child belongsTo association

    associations: [
        {
            type        : 'belongsTo',
            model       : 'Foo.model.Parent',
            foreignKey  : 'parent_id',   // if not overridden you get 'Foo.model.Parent_id'
            getterName  : 'getParent',   // if not overridden you get 'getFoo.model.Parent'
            setterName  : 'setParent',   // if not overridden you get 'setFoo.model.Parent'
            instanceName: 'parent' // if not overridden you get 'Foo.model.ParentBelongsToInstance'
        }
    ]

[ssamayoa]

I'm using this code (with slightly changes) but wonder if this could be included in the ExtJS's official release.

@chrisface: are you willing to give copyright to Sencha?

@Sencha: are you willing to make this code (as is or enhanced) official?

Regards.

[ssamayoa]

BUMP.

[jinggirl]

would anybody know if this has been fixed yet? i am facing a similar problem. i couldn't write the associated data.