1. #1
    Ext JS Premium Member Munter's Avatar
    Join Date
    Dec 2009
    Location
    Copenhagen, Denmark
    Posts
    55
    Vote Rating
    0
    Munter is on a distinguished road

      0  

    Default Sencha Platform denormalized Data patch

    Sencha Platform denormalized Data patch


    This is a patch proposal for the data package of Sencha Platform.

    The patch extends the Data package to be able to output a Model and its associations as denormalized data blobs, purely based on configuration of the Model.

    I will try to present the general use case, and at some points exemplify with an implementation of the Portable Contacts (PoCo) draft specification found at: http://portablecontacts.net/draft-spec.html

    Premise:
    Denormalized databases, or document databases, have gained a lot of momentum in the last years. There are a lot of use cases where denormalized data has been shown to be a more effective way of storing and scaling large datasets. Thus it is an interesting use case for the Sencha Platform's Data package to implement some way of working with denormalized datasets.

    The current Data package currently handles eager loading of related data through a nested data structure quite well. The model still assumes complete normalization on the backend though. Here are some of the things that would be nice to have when working with denormalized data:

    - A way to define when an association should be treated as denormalized
    I will refer to these relations as 'inner'.

    - Optional properties
    Most document databases allow for schemaless datasets, which don't require all properties to be set. For example the PoCo specification includes a lot of optional properties. The Sencha Model should be able to represent this by not serializing empty properties when saving data.

    - Optional validation
    If a property can be optional, then the model itself should be able to validate even when the optional property is not set.

    - Deep validation
    A Model that has inner relations should do a deep validation of all its inner relations, since upon serialization these are conceptually a part of the Model itself.

    - Deep serialization
    When a Model is serialized, all its inner relations should be serialized as well for remote storage.

    And specific for ExtJS and Sencha Touch:
    - A Record that is marked as dirty should mark its parent relation as dirty as well if it is an inner relation.

    - Saving a modified record should delegate to the records parent record if it is an inner relation.

    Architecture:
    I have had to make some choices along the way to be able to differentiate these different types of properties and relations. I have checked in Sencha Touch 1.0.1a on Github and applied my changes on top of that, available here: https://github.com/One-com/Sencha-To...s/denormalized

    I will run through each of the changes and try to explain my reasoning, and give implementation examples:

    https://github.com/One-com/Sencha-Touch/commit/c64d
    "Added innerOf property to inner associated models on instantiation"

    This commit adds a tracking of whether a model is an inner association of another model by adding a 'innerOf' property on the instantiated Record.
    A configuration property has been added, so you can set this up by adding an 'inner'-property on the configuration for the hasMany association. Example:
    Code:
    { inner: true, model: 'Email', name: 'emails' }

    https://github.com/One-com/Sencha-Touch/commit/41a7
    "Added deep validation of inner associations"

    This commit makes a Record accumulate errors in validation of inner associations.
    It uses the 'inner'-property of the association configuration to determine whether this is relevant.

    https://github.com/One-com/Sencha-Touch/commit/6714
    "Added optional properties"

    Sometimes you will have optional properties in schemaless databases, and you don't really want the ExtJS autoconversion to run with its default settings, converting absent properties to empty string and similar. This commit adds conversion functions for each of ExtJS datatypes, which allow a property to be optional.
    I have chosen to use null as the internal value for absent properties, which will be used as markers to remove when serializing.
    This is an example of an optional property configuration:
    Code:
    { name: 'nickname', type: 'string', optional: true, defaultValue: null }
    Currently I have to add a defaultValue of null for this to work. This shouldn't be needed for optional properties in the long run.


    https://github.com/One-com/Sencha-Touch/commit/8667
    "Added optional validations that accept null as a valid value"

    If the premise of an optional property is accepted, this commit is obvious.
    An optional property with the value of null needs to be valid, hence the validations need to be changed so they return true for optional properties with the value null.


    https://github.com/One-com/Sencha-Touch/commit/87b5
    "Added deep serialization of inner associations and removal of optional properties set to null."

    Since the concept of denormalized data is that the parent node is the only entry in the database, we need to do a deep serialization when saving to the database.
    Instead of just returning Record.data, Writer.getRecordData() now does this:
    - Iterate through all properties and add them to the return object unless they are optional and the value is null.
    - Iterate through all associated records that are marked as inner associations, serialize them and strip all foreign key properties.


    https://github.com/One-com/Sencha-Touch/commit/1a45
    "Added state update for inner records, so the parent record will be marked as dirty if an inner record is marked dirty. Now uses built-in setDirty method."

    This is more of a convenience change for ExtJS. It makes sense that any Record that is marked as dirty should mark its parent as dirty, so that the outer store, which synchronizes with the backend, can see when any data has been updated.
    Implementing this in application logic wouldn't make sense because you would have to do it over and over again.


    https://github.com/One-com/Sencha-Touch/commit/07b3
    "Made all inner relations delegate saving of records to their containing records"

    Another convenience change for ExtJS. When calling the save method on a Record that is an inner association, it is assumed that the Model does not have a proxy configured, and hence will not be able to store the changes on the backend by itself.
    Instead the changed inner Record should call the save method on the Record it is inside, which will eventually lead to a serialization of the complete tree that needs to be synced with the server.


    I expect that I might have made some design choices that Sencha will not agree on. I am prepared to put in the needed work to fix these if the end result will be Sencha accepting the patch.
    I haven't put in any work to port the patch to ExtJS4, since Ed Spencer mentioned in his talk about the data package in Ext4 that it will run on Sencha Platform as well. As I don't see this being the case yet, I will wait for a release that does, and put in the work then.

    I hope that Sencha will take this patch into serious consideration.

  2. #2
    Ext JS Premium Member Munter's Avatar
    Join Date
    Dec 2009
    Location
    Copenhagen, Denmark
    Posts
    55
    Vote Rating
    0
    Munter is on a distinguished road

      0  

    Default Updated for Sencha Touch 1.1.0

    Updated for Sencha Touch 1.1.0


    I have updated the Github repository to Sencha Touch 1.1.0 and applied my patch to it on a new branch:
    https://github.com/One-com/Sencha-To...rmalized_1.1.0

    I would really appreciate some feedback from the Sencha team on this

  3. #3
    Sencha User
    Join Date
    Feb 2011
    Posts
    74
    Vote Rating
    3
    shaneavery is on a distinguished road

      0  

    Thumbs up I applaud your efforts with this

    I applaud your efforts with this


    Munter,

    I applaud your efforts with this. I add my voice to your request for the Sencha Dev team to review and hopefully adopt your code, or at least its intent. I am trying to find solutions for cleanly integrating ExtJS 4 and Sencha Touch with CouchDB, and it appears your approach is a seamless solution.

    While we wait, I was hoping to test your patch with ExtJS 4. Am I to understand from your GitHub that these are the only files that were modified?

    src/platform/src/data/Model.js
    src/platform/src/util/Stateful.js
    src/platform/src/data/writers/Writer.js
    src/platform/src/data/Types.js
    src/platform/src/data/Validations.js
    src/platform/src/data/readers/Reader.js

    If you have any instructions for me, please advise. Thanks for providing this.

  4. #4
    Ext JS Premium Member Munter's Avatar
    Join Date
    Dec 2009
    Location
    Copenhagen, Denmark
    Posts
    55
    Vote Rating
    0
    Munter is on a distinguished road

      0  

    Default


    Quote Originally Posted by shaneavery View Post
    While we wait, I was hoping to test your patch with ExtJS 4. Am I to understand from your GitHub that these are the only files that were modified?

    src/platform/src/data/Model.js
    src/platform/src/util/Stateful.js
    src/platform/src/data/writers/Writer.js
    src/platform/src/data/Types.js
    src/platform/src/data/Validations.js
    src/platform/src/data/readers/Reader.js

    If you have any instructions for me, please advise. Thanks for providing this.
    My colleagues are actually working on some ExtJS 4 stuff which needs this patch as well, so I will put in some time today or tomorrow to port the patch to ExtJS 4 and will make it available as a github repository which you can clone, or you can just cherrypick my commits from it.

    I'll report back with a link to the github repository when I'm done.

    Thanks for your interest. I hope that Sencha will see the value of supporting denormalized database schemas.

  5. #5
    Sencha User
    Join Date
    Feb 2011
    Posts
    74
    Vote Rating
    3
    shaneavery is on a distinguished road

      0  

    Red face This sounds good to me...

    This sounds good to me...


    Peter,

    Thank you for your generosity. I look forward to what you come up with. I have read several posts from users on these forums who use NoSQL style DBs, so I assume there might be quite a crowd who are interested in your work. I will spread the word, and try to rally support.

    I am available to do testing and eval, is there any other way I can contribute?

    Thanks.

  6. #6
    Sencha - Community Support Team edspencer's Avatar
    Join Date
    Jan 2009
    Location
    Palo Alto, California
    Posts
    1,939
    Vote Rating
    7
    edspencer is a jewel in the rough edspencer is a jewel in the rough edspencer is a jewel in the rough

      0  

    Default


    These are some very interesting ideas, and as the author of the 4.x data package I certainly welcome them. My aim with the new package was to maintain continuity with the 3.x package while refactoring some of the less well-rounded parts of the code base and adding a few new features. Adding support for denormalized databases was never an aim of the exercise, but I'm intrigued to see where this could go.

    I will say that any such update will not make it into 4.0, which is very much in lockdown now, but could certainly form part of a 4.x update. My attention is solely on 4.0 for the moment but I will monitor this thread to see where you take it. Hopefully the architecture is solid enough that you can achieve your objectives.
    Ext JS Senior Software Architect
    Personal Blog: http://edspencer.net
    Twitter: http://twitter.com/edspencer
    Github: http://github.com/edspencer

  7. #7
    Sencha User
    Join Date
    Feb 2011
    Posts
    74
    Vote Rating
    3
    shaneavery is on a distinguished road

      0  

    Red face Thanks for your response

    Thanks for your response


    Ed,

    Thank you for responding to this thread personally. I can only imagine how busy you and the rest of the dev team are these days, so I appreciate you taking the time. What you stated sounds reasonable and agreeable to me.

    I am eager to try out Peter Müller's ExtJS-4 patch when it becomes available. If it is alright with him, I would love to post a working example (perhaps as a Couch App) and a tutorial after I test it and get it working with CouchDB.

  8. #8
    Ext JS Premium Member Munter's Avatar
    Join Date
    Dec 2009
    Location
    Copenhagen, Denmark
    Posts
    55
    Vote Rating
    0
    Munter is on a distinguished road

      0  

    Default


    Quote Originally Posted by shaneavery View Post
    I am eager to try out Peter Müller's ExtJS-4 patch when it becomes available. If it is alright with him, I would love to post a working example (perhaps as a Couch App) and a tutorial after I test it and get it working with CouchDB.
    Sadly I haven't been able to find the time to put in some real work on porting the patch yet, since it became clear to me that Ext JS 4 is quite different in its data architecture than Sencha Touch. Or at least different enough that I need to go through all the motions of stepping through every possible branch and do a lot of testing before I can get anywhere.

    As a small note it woul dreally be helpful if the .jsb3 file used to build the ext packages was available, since I really need to set up a development environment where I can include all indidivual source files. This information doesn't seem to be available anywhere as it stands right now.

  9. #9
    Sencha - Community Support Team edspencer's Avatar
    Join Date
    Jan 2009
    Location
    Palo Alto, California
    Posts
    1,939
    Vote Rating
    7
    edspencer is a jewel in the rough edspencer is a jewel in the rough edspencer is a jewel in the rough

      0  

    Default


    Quote Originally Posted by Munter View Post
    it became clear to me that Ext JS 4 is quite different in its data architecture than Sencha Touch. Or at least different enough that I need to go through all the motions of stepping through every possible branch and do a lot of testing before I can get anywhere.
    It's 99%+ identical - very few changes between ST 1.x and Ext JS 4.x. Biggest difference really is the use of the new class system in 4 - all of the same functionality is present in both.
    Ext JS Senior Software Architect
    Personal Blog: http://edspencer.net
    Twitter: http://twitter.com/edspencer
    Github: http://github.com/edspencer

  10. #10
    Sencha User
    Join Date
    Feb 2011
    Posts
    74
    Vote Rating
    3
    shaneavery is on a distinguished road

      0  

    Lightbulb Have you tried Spket?

    Have you tried Spket?


    Peter,

    I have set up Spket as my IDE for JavaScript, since it historically provides good code completion and profile support for ExtJS. It runs on Linux and Windows as a standalone app. It can also be installed as a plugin and run as a perspective in Eclipse, for cross platform compatibility (Mac). See here:

    http://www.spket.com/download.html

    Recently, a member of Spket's forum provided a .jsb2 so that an updated profile can be used for ExtJS-4 PR5. You can download it here, along with his updated plugin file:

    http://forums.spket.com/viewtopic.ph...t=extjs4#p3141

    I have made a small modification to the .jsb2 so that it works with ExtJS-4 Beta1 (have not tried Beta 2 yet). Just edit line 6 and change CODE: "file": "pkgs/all.js", to read CODE: "file": "pkgs/classes.js",.

    Not sure if this is any use to you, but I hope it helps.

Similar Threads

  1. Sencha Touch Platform Compatibility?
    By Alexandr3 in forum Sencha Touch 1.x: Discussion
    Replies: 9
    Last Post: 30 Jul 2012, 1:58 AM
  2. Replies: 2
    Last Post: 9 Sep 2010, 10:24 AM
  3. [patch] Subclass-aware Ext.data.Record.create
    By GArrow in forum Ext 1.x: User Extensions and Plugins
    Replies: 0
    Last Post: 26 Sep 2007, 9:07 AM

Thread Participants: 17

Tags for this Thread

Turkiyenin en sevilen filmlerinin yer aldigi xnxx internet sitemiz olan ve porn sex tarzi bir site olan mobil porno izle sitemiz gercekten dillere destan bir durumda herkesin sevdigi bir site olarak tarihe gececege benziyor. Sitenin en belirgin ozelliklerinden birisi de Turkiyede gercekten kaliteli ve muntazam, duzenli porno izle siteleri olmamasidir. Bu yuzden iste. Ayrica en net goruntu kalitesine sahip adresinde yayinlanmaktadir. Mesela diğer sitelerimizden bahsedecek olursak, en iyi hd porno video arşivine sahip bir siteyiz. "The Best anal porn videos and slut anus, big asses movies set..." hd porno faketaxi