[0.7.10] date field are not (correctly) handled

[NiNiX]

Versions: 0.7.10 (and 0.7.8) with Sencha Touch 2.1.1
Browser: Chrome 24 (Win8 / Desktop)
Severity: critical (data corruption)

Using 'syncstorage' proxy in store, it seems that date field value are not correctly sent to the remote server when synchronizing new records (date for updated records are not sync at all, but I will open a new ticket for this case). When re-importing remote data in "empty" local storage (e.g. after a first login on a new device), date values for imported records are invalids (empty objects {}). So I supposed that date data are definitively lost.

Test case (I modified the 'examples/touch/todo' example for this test case)

MyApp.model.Todo line 29 (changed the 'timestamp' field type to DATE)

[...]
    {
        name: 'timestamp',
        type: 'date'
    }
[...] 

MyApp.view.Edit:: SaveRecord line 75 (remove '.getTime())

[...]
    record.set("completed", (completed == 1));
    record.set("timestamp", new Date());
[...] 

MyApp.view.Todos line 59 (displaying the timestamp)

[...]
    itemTpl: [
        '<div class="completed{completed}">{task}-{timestamp}</div>'
    ],
[...] 

- Create a new task and save it: the list item will show a valid date (when tracing in MyApp.view.Edit:: SaveRecord, the record timestamp is a valid date). But based on the SIO debug log, sent value for the new record timestamp is {} (empty object)

DEBUG: Transport.send {
"service":"SyncRpcService",
"from":"device-**********",
"msg":{
    "method":"putUpdates",
    "args":[{
        "dd":{"userId":"**********", "databaseName":"todos", "generation":"0", "version":2},
        "rd":{"deviceId":"device-***********","replicaId":"b6"},
        "csv":"b1-66754329-5.b3-67258192.b6-67444510-5",
        "updates":"[
            [\"b6-67444510\",\"_oid\",\"b6-67444510\",\"b6-67444510\"],
            [\"task\",\"test2\",\"b6-67444510-1\"],
            [\"details\",\"test2\",\"b6-67444510-2\"],
            [\"completed\",false,\"b6-67444510-3\"],
            [\"timestamp\",{},\"b6-67444510-4\"],
            [\"id\",\"b6-67444510\",\"b6-67444510-5\"]
        ]"
    }],
    "corr-id":6
},
[...]
}

- Logout (and clear browser data to be sure to remove the store local copy).
- Login again and wait for sync: the list item will show title-[object Object]. Based on the SIO debug log, imported value for timestamp is {}:

DEBUG: SyncProxy.applyUpdate: (b6-67444510 . _oid = 'b6-67444510' @ b6-67444510) accepted, creating new record 
INFO: SyncProxy.applyUpdateToRecord: (b6-67444510 . task = 'test2' @ b6-67444510-1) accepted 
INFO: SyncProxy.applyUpdateToRecord: (b6-67444510 . details = 'test2' @ b6-67444510-2) accepted 
INFO: SyncProxy.applyUpdateToRecord: (b6-67444510 . completed = 'false' @ b6-67444510-3) accepted
INFO: SyncProxy.applyUpdateToRecord: (b6-67444510 . timestamp = '{}' @ b6-67444510-4) accepted 
INFO: SyncProxy.applyUpdateToRecord: (b6-67444510 . id = 'b6-67444510' @ b6-67444510-5) accepted

[jason.cline]

Thank you for the detailed test case to re-produce. We are investigating and will get back to you as soon as we know more.

[jason.cline]

Hello,

As a general practice we have avoided using the date field type. Instead storing the date as either a number:

date.getTime()

or as a formatted/parseable date using the ISO standard.


We would recommend that you modify your code and not use the date field type with sync stores while we work on a better support for that field type.

In the examples we provide (todo and shared data sync) we use date.getTime() so that the values can be sorted relative to one another. If your timestamp is for display purposes only then you can either convert the number into a date object then format it or you can simply store the date as a pre-formatted string.

[NiNiX]

Hi Jason,

Thanks for your advises (can't find where in documentation you warn about not using date field?). I finished my app (JDI for the HTML5 contest) so I don't look for an alternative solution, I just want to share my experience with Sencha IO and help you to enhance it.

I started my project without SIO and use a local store. All my code was based on date manipulation and display (and it worked fine). When switching to SIO, I tried to use integer or string to deal with SIO limitations, but (after many hours of debug) I had to heavily hack my model and fix some of your code to get this to work.

I needed support for null date, not sure that integer field can support null with SIO and did not want to have this special case: if (field == 0) means null. Using string seems OK, but there was the sorting issue (my field should be sorted as date). To workaround this, I defined my own Ext.data.Types (based on the string one):

Ext.data.Types.DATETIME = {
    convert: function(value) {
       return (value === undefined || value === null)
            ? (this.getAllowNull() ? null : '')
            : String(value);
    },
    sortType: Ext.data.SortTypes.asDate,
    type: 'datetime'
}; 

Then, I did not want to convert to/from date everywhere in my code when accessing/updating my records. I though about just implementing the encode, decode and convert methods of my new data type, but SIO failed to handle it correctly. So I HACKED the get() and set() methods of my model to perform implicit conversion. And again, SIO decided to not synchronize my field (found later where was the issue in your code).

So yes, I know that date are not totally supported in SIO and that it's preferable to not use it. But I think that you should fix this and transparently support date (and null date). Now, I still have two other date related bugs to report, are you interested in it?

[jason.cline]

Hello,

Again thank you for your detailed feedback. We are working on a more robust fix for the date issue.

We don't cover the date issue in the documentation. I'll add it in the next release; assuming we don't fix the issue and provide full support for dates in sync stores.