View Full Version : [3.2] Ext.ux.PersistentStore: auto-save records to HTML5 DOM storage

4 Jun 2010, 8:54 AM
I desperately wanted the ability to have a Ext Store that fetches data from a server, but can cache that data on the client so it persists across page reloads. This improves performance and reduces demand on the server providing the data. The problem is Ext's current data architecture can't do a lot of the things that are needed to persist Record objects properly, so this UX has several pieces to it to enhance Ext's architecture. Hopefully some if not all of these enhancements will make it into the main distribution, so I'm including lots of the justification for the design.

The first problem is that although DataReaders support using convert functions to turn raw data into objects (like Date), there is no way to do the reverse and turn an object back into a serialized form that could be re-read by the same convert function later. As an example of this limitation, if you attempt to build a store with a date field and attach a JsonWriter to it, the JsonWriter will send the date back in the default JavaScript format. Putting custom classes on Records gets even messier!

The solution to this is to add another custom function to data types that does the inverse operation of convert, which I call write(). Here's the enhancement to the DATE type that makes this work (note I've made some other enhancements here that rely on Ext.ux.DateFormat (http://www.extjs.com/forum/showthread.php?99179-3.0-Ext.ux.DateFormat-Ext.ux.TimeZone-Java-like-dates-and-timezones); it would be rather trivial to write different versions of these to use the builtin Date.parse() and Date.format() functions):

Ext.apply(Ext.data.Types.DATE, {
convert: function(v) {
var df = this.dateFormat;
return null;
return v;
if(Ext.isObject(df)) {
return df.parse(v);
if(df == 'timestamp'){
return new Date(v*1000);
if(df == 'time'){
return new Date(parseInt(v, 10));
if(this.dateTimezone) {
return new Ext.ux.DateFormat(df, this.dateTimezone).parse(v);
return Date.parseDate(v, df);
var parsed = Date.parse(v);
return parsed ? new Date(parsed) : null;
write: function(value) {
var df = this.dateFormat || "c";
if(! Ext.isObject(df)) {
df = new Ext.ux.DateFormat(df, this.dateTimezone);
return df.format(value);
Once we have a write() function to call, something needs to be able to call it! My first attempt at this was an enhancement of DataWriter.toHash(), but relying on DataReaders and DataWriters for Record serialization and deserialization had a bunch of problems:

A Store may or may not have a DataWriter defined on it
A Store's DataReader may be expecting a format that can't be persisted to DOM storage (XML)
DataReaders and DataWriters are designed to be attached to Stores, and Stores are coded to only interact with a single DataReader and DataWriter combination.

To solve these problems I chose instead to enhance the Record class itself, so a Record can produce a raw encodeable object representation of its data which can then be read back in to rebuild the original record. This is the equivalent of implementing Serializable in Java.

Here's the serialize() method, which is pretty straightforward:

* Produces an object containing all the serialized data in this Record
* @param mapNames Pass true to have the serialization perform field name
* mapping according to the mapping property of the field definitions
* @param writeAllFields Pass false to only retrieve fields which have changed
* @return A flat serialized object containing all data in the record
Ext.data.Record.prototype.serialize = function(mapNames, writeAllFields) {
var map = this.fields.map,
data = {},
raw = (writeAllFields === false && this.phantom === false) ? this.getChanges() : this.data,
Ext.iterate(raw, function(prop, value){
if((m = map[prop])){
var v;
// Call the appropriate write function if one is defined for
// this type. Otherwise just use the value itself.
if(Ext.isFunction(m.write)) {
v = m.write(value);
} else if(Ext.isObject(m.type) && Ext.isFunction(m.type.write)) {
v = m.type.write.call(m, value);
} else {
v = value;
data[(mapNames && m.mapping)? m.mapping: m.name] = v;
return {
id: this.id,
phantom: this.phantom,
data: data,
dirty: this.dirty,
modified: this.modified
deserialize() is a static method that gets added to the Record subclass/type by Ext.data.Record.create. This is the override of that factory method:

(function() {
var orig = Ext.data.Record.create;
Ext.data.Record.create = function(o) {
var recType = orig(o);
* Construct a record from raw data
* @param obj An object containing the raw data for the record
* @param mapNames Pass true to perform field name mappings
* according to the "mapping" property on the field definition
* of this record type
* @return A Record object of the proper type initialized with
* the given data
recType.deserialize = function(obj, mapNames) {
var values = {};
recType.prototype.fields.each(function(f) {
var v = obj.data[mapNames && f.mapping? f.mapping: f.name];
values[f.name] = f.convert((v !== undefined)? v: f.defaultValue, obj.data);
var rec = new recType(values, obj.id);
Ext.apply(rec, {
phantom: obj.phantom,
dirty: obj.dirty,
modified: obj.modified
return rec;
return recType;
As an aside, we can now easily fix that bug in DataWriter.toHash() that causes Dates to be sent back to the server in the default Javascript format. It might also be possible to refactor DataReader or JsonReader to use deserialize as part of its read operations.

Ext.override(Ext.data.DataWriter, {
toHash: function(rec, config) {
var data = rec.serialize(true, this.writeAllFields).data;
// we don't want to write Ext auto-generated id to hash. Careful not to remove it on Models not having auto-increment pk though.
// We can tell its not auto-increment if the user defined a DataReader field for it *and* that field's value is non-empty.
// we could also do a RegExp here for the Ext.data.Record AUTO_ID prefix.
if (rec.phantom) {
if (rec.fields.containsKey(this.meta.idProperty) && Ext.isEmpty(rec.data[this.meta.idProperty])) {
delete data[this.meta.idProperty];
} else {
data[this.meta.idProperty] = rec.id
return data;
Once all those framework changes are made, declaring Ext.ux.PersistentStore is pretty easy:

* A special Store that automatically saves a copy of its contents to the
* browser's local DOM storage and repopulates itself from storage when
* instantiated.
* @param domStorageKey The key to use in local storage. If unspecified,
* defaults to the value of storeId if present. If neither domStorageKey nor
* storeId are present, persistence is disabled.
Ext.ux.PersistentStore = function(config) {
Ext.ux.PersistentStore.superclass.constructor.call(this, config);
if(this.storeId && this.domStorageKey === undefined) {
this.domStorageKey = this.storeId;

"load": this.persistData,
"save": this.persistData,
"clear": this.clearPersistedData

Ext.extend(Ext.ux.PersistentStore, Ext.data.Store, {
// Private. Saves all data in the store to DOM storage.
persistData: function() {
if(! this.domStorageKey || typeof localStorage === "undefined" || this.isDestroyed === true) {
var raw = [];
this.each(function(r) {
raw.push(r.serialize(false, true));
localStorage.setItem(this.domStorageKey, Ext.encode(raw));
// Private. Loads all data from DOM storage.
retrieveData: function() {
if(! this.domStorageKey || typeof localStorage === "undefined" || this.isDestroyed === true) {
var r = localStorage.getItem(this.domStorageKey);
if(! r) {
// Nothing in storage. Ignore.
try {
r = Ext.decode(r);
if(! Ext.isArray(r)) {
throw "Not an array";
} catch(e) {
// Data in storage was invalid. Clear it and stop.
var recs = [];
for(var i = 0, len = r.length; i < len; i ++) {
recs.push(this.recordType.deserialize(r[i], false));
this.totalLength = recs.length;
* Clears all the data this store has cached in local storage
clearPersistedData: function() {
if(! this.domStorageKey || typeof localStorage === "undefined" || this.isDestroyed === true) {
I just finished it this morning, so peripheral cases aren't heavily tested yet.