PDA

View Full Version : [code] Ext.data.ObjectReader



GArrow
25 Sep 2007, 4:55 PM
I have been doing some work to connect Ext to my application, which uses DWR to expose Java functions to JavaScript. I plan to post a series of threads to provide this code and to address various Ext issues that arose while developing it.

I found it necessary to create the following DataReader subclass, which converts objects into records. It was written to drop into the Ext.data namespace and the comments are based off the example and language in ArrayReader.

Two incidental observations. First, it seems wrong for ArrayReader to be a subclass of JsonReader. It gains nothing by this other than the implementation of read(), which is filled with json-specific logic and thus unusable anyway. I would suggest that ArrayReader should derive from DataReader directly. This leads to the second observation, that the read() method of a Reader appears to be of questionable value. To that end, I added a Note to the documentation for it, which reads:


Note: This implementation is intended as a convienience to simplify writing DataProxy subclasses and to guide implementations into common field names to return result data. If these fields are unnatural for a proxy, it may call readRecords() directly and handle this functionality itself.

It might be more sensible to remove it entirely, and require DataProxy classes to handle everything other than the basic conversion of blobs into records using their implementation-specific knowledge about how and where meta-data is stored.


/**
* @class Ext.data.ObjectReader
* @extends Ext.data.DataReader
* Data reader class to create an Array of {@link Ext.data.Record} objects from an array of objects
* based on mappings in a provided Ext.data.Record constructor.<br><br>
* <p>
* Example code:
* <pre><code>
var RecordDef = Ext.data.Record.create([
{name: 'name', mapping: 'name'}, // "mapping" property not needed if it's the same as "name"
{name: 'occupation'} // This field will use "occupation" as the mapping.
]);
var myReader = new Ext.data.ObjectReader({
id: "id" // The field that provides an ID for the record (optional)
}, RecordDef);
</code></pre>
* <p>
* This would consume data like this:
* <pre><code>
[ {id:1, name:'Bill', occupation:'Gardener'}, {id:2, name:'Ben', occupation:'Horticulturalist'} ]
</code></pre>
* @cfg {String} id (optional) The field that provides an ID for the record
* @constructor
* Create a new ObjectReader
* @param {Object} meta Metadata configuration options.
* @param {Mixed} recordType The definition of the data record type to produce. This can be either a
* Record subclass created with {@link Ext.data.Record#create}, or an array of objects with which to call
* Ext.data.Record.create. See the {@link Ext.data.Record} class for more details.
*/
Ext.data.ObjectReader = function(meta, recordType){
meta = meta || {};
Ext.data.ObjectReader.superclass.constructor.call(this, meta, recordType||meta.fields);
};
Ext.extend(Ext.data.ObjectReader, Ext.data.DataReader, {
/**
* This method is only used by a DataProxy which has retrieved data from a remote server.
* <p>
* Data should be provided in the following structure:
* <pre><code>
{
objects : [ { }, { }... ], // An array of objects
totalSize : 1234 // The total number of records (>= objects.length)
}
</code></pre>
* The 'objects' field is required; other fields are optional.
* @param {Array} response An object containing an array of objects and meta data.
* @return {Object} records A data block which is used by an {@link Ext.data.Store} as
* a cache of Ext.data.Records.
* <p>
* Note: This implementation is intended as a convienience to simplify writing DataProxy
* subclasses and to guide implementations into common field names to return result data.
* If these fields are unnatural for a proxy, it may call readRecords() directly and handle
* this functionality itself.
*/
read : function(response){
if( undefined == response.objects ) {
throw {message: "ObjectReader.read: Objects not available"};
}
var result = this.readRecords(response.objects);
if ( undefined != response.totalSize )
result.totalRecords = response.totalSize;
return result;
},

/**
* Create a data block containing Ext.data.Records from an an array of objects.
* @param {Object} objects An array of objects.
* @return {Object} records A data block which is used by an {@link Ext.data.Store} as
* a cache of Ext.data.Records.
*/
readRecords : function(objects){
var records = [];
var recordType = this.recordType, fields = recordType.prototype.fields;
var idField = this.meta.id;
for(var i = 0; i < objects.length; i++) {
var object = objects[i];
var values = {};
for(var j = 0; j < fields.length; j++){
var field = fields.items[j];
var v = object[field.mapping || field.name] || field.defaultValue;
v = field.convert(v);
values[field.name] = v;
}
var id = idField ? object[idField] : undefined;
records[records.length] = new recordType(values, id);
}
return {
records : records,
totalRecords : records.length
};
}
});

jack.slocum
25 Sep 2007, 6:46 PM
Thanks for sharing the code. I am sure there will be plenty who find it extremely useful. You may wish to post it in the X forum where others might find it a little more easily.

In regards to your comments:
The read() method is required because data can come from different sources. E.g. json and arrays from responseText and xml from responseXML. The proxy doesn't need knowledge of this as it is just the transport. One methodof transport (e.g. HttpProxy) can handle multiple data formats (e.g. json and xml) but a Reader only handles a single data format.

At that point I do agree it appears that JsonReader has outgrown ArrayReader and it's possibe to remove the subclass. I will take a look at it.

mystix
25 Sep 2007, 7:16 PM
[ moved to Ext.ux from General ]

GArrow
26 Sep 2007, 6:49 AM
The read() method is required because data can come from different sources. E.g. json and arrays from responseText and xml from responseXML. The proxy doesn't need knowledge of this as it is just the transport. One method of transport (e.g. HttpProxy) can handle multiple data formats (e.g. json and xml) but a Reader only handles a single data format.

My experience runs counter to this. Let me try to explain how I am thinking of it. Let's say that there are a dozen different technologies for talking to sources of data, Tech1..Tech12. Because this is JavaScript, there are only a limited number of (obvious) ways that these could represent data: arrays of objects, arrays of arrays and strings (json). (Of course, it is possible, if less likely, that they could use other representations, like arrays of encoded XML strings, but let's ignore that for the moment.)

However, it is very likely that each of these technologies will require a different proxy implementation, as the proxy is the thing that is responsible for talking to the source of data. At present, Ext has some fairly generic proxies, but one can relatively easily imagine additional proxies, such as mine for talking to DWR, to talk directly to services like Oracle or IMAP or LDAP or DNS or to another AJAX system or simply to other JavaScript code, such as Google Gears. In this scenario, one would have:
OracleProxy
IMAPProxy
LDAPProxy
DNSProxy
DWRProxy
GearsProxy
etc...

Now, unless all of the systems behind these proxies are designed with the intent of working with existing Ext readers (which surely won't happen), it is unlikely that the proxy will conveniently wind up with a response object that matches what a given reader class expects. In the case of my code, that would mean having a 'totalSize' field; in the case of JsonReader that would mean having a 'responseText' field that contains json data.

So, it is surely the responsibility of the proxy to make sure that such an object structure exists. For example, if the (admittedly contrived) DNS-for-JavaScript service decided to use json, but stored it in a 'jsonData' field, then DNSProxy would be responsible for, at the least:

reader.read({ responseText : response.jsonData });

This would only work if the metadata was stored in the manner that JsonReader expects. If the metadata were entirely separated in a 'jsonMetaData' field, then the DNSProxy would have to do one of three things:

Evaluate both jsonData and jsonMetaData into JavaScript objects and them combine them into the structure expected by JsonReader.read() and json-encode that.
Evaluate both jsonData and jsonMetaData into JavaScript objects and them combine them into the structure expected by JsonReader.readRecords() and then call that.
Write their own Reader class, which would happen to also use json encoding, but reuse/share no code with JsonReader.


Well, that number three is the obvious good choice here shows two things.

The JsonReader isn't very reusable.
I have never used the JsonReader, or I would have realized it would make a bad example case.


So, once again, using ArrayReader this time:

So, if the DNS-for-JavaScript service happens to provide the proxy with the following response containing 2 of 40 records:

{ dnsRecords : [['foo', '1.2.3.4'], ['bar', '1.2.3.5']], total : 40 }

It would clrearly make sense for DNSProxy to use ArrayReader to convert the array in 'dnsRecords' into Record objects, but there is no way (since read() is not implemented) for an ArrayReader class to know about the 'total' field and use it. The DNSProxy class has to take care of that, with:

var result = reader.readRecords(response.dnsRecords); // An ArrayReader
if ( undefined != response.total ) result.totalRecords = response.total;
return result;

While I can appreciate that the Proxy should not need to know details about the internals of Records, like that there is a field 'totalRecords', there seems to me no way to implement readers that are both highly reusable and simultaneously understand how to find and use the meta-data from numerous service proxies.

So, I suggest that the readers, at least the ones with generic-looking names like 'ArrayReader', only do very generic things (and perhaps not even provide a working read() method), and leave it to their subclasses to implement read() or to proxy classes to implement any other logic needed (Such as in the case where one needs to make multiple requests, because the service cannot provide the meta-data in the same response as the data!)

So, what I am thinking would (temporarily ignoring all backwards compatibility for clarity) look like

ObjectReader -- implements simpleminded readRecords; read() throws "not implemented"
ArrayReader -- implements simpleminded readRecords; read() throws "not implemented"
JsonReader -- implements simpleminded readRecords; read() throws "not implemented"
XMLReader -- implements simpleminded readRecords; read() throws "not implemented"

DWRReader -- subclasses ObjectReader; implements read() that knows about meta-data
ExtJReader -- subclasses JsonReader; implements read() that does what JsonReader.read() does in the existing distribution

HTTPProxy -- using XMLReader, calls readRecords, does not expect or care about metadata
DWRProxy -- using DWRReader, calls read(), which knows where meta-data is
FooProxy -- using ArrayReader, calls readRecords, and makes other network calls to fetch meta-data

So, in practice, it would not make sense to change JsonReader as I describe, nor would there be any real benefit in removing the XMLReader's trivial read() implementation. But it would be possible and reasonable to re-parent ArrayReader to DataReader (since no one could possibly be using its read() call inherited from JsonReader) and implement a read() that throws "unimplemented", and then do the same with the ObjectReader code above (Which I have now realized I should subclass as DWRReader to implement a read() for my DWR-specific purposes).

GArrow
26 Sep 2007, 4:51 PM
After the rambling comments I posted here previously, I have been reviewing and finalizing some of my class documentation, and this has led me to a better understanding (I think) of the original intent of the Reader/Proxy/Connection classes.

The Reader Contract

The Reader is meant to implement half of a contract between the data provider (generally a server, but potentially anything one can access with JavaScript) and the Ext system, by converting foreign data into Ext Records. The Proxy class is meant to be, as was said, a transport. And so long as one can cause the data provider to adhere to the contract of the Reader -- such as by providing an object with a 'responseXML' member -- this can appear to be the case. But, when a data provider does not respond with data in that format, then either the data provider needs to be modified (which may be impractical if the server is not under the control of the developer using Ext), or the Proxy has to be responsible for transforming the data from a format that is available into one that a Reader expects.

The Connection class is where this dependency has been hidden, by itself doing what I have been suggesting is the job of a Proxy class -- structuring data in the format expected by a Reader class.

In order to use a Store, One has to provide a Proxy and a Reader. So, these are the core/necessary/responsible classes. The Connection class can/should be considered to be part of the HttpProxy, which is(?) the only proxy that uses it -- and Connection appears to be so HTML/HTTP-centric that it could not be used for any other Proxy.

So, given that a Proxy, or a Proxy with the help of a Connection or some data provider, must arrange for data to be structured in the way a Reader's read method expects, it seems to me highly dubious that the read method actually adds any value. Because ultimately it is just decoding data from a format that the proxy itself just put the data into. Thus, if you start with:


[['foo', 123],['bar', 456]]

And a Proxy puts it into an object like:


{ responseArray: [['foo', 123],['bar', 456]] }

And the Reader (here, "ArrayReader" perhaps) immediately extracts the field and passes it to readRecords:


[['foo', 123],['bar', 456]]

It seems like there may be no value being added. If the Proxy (or code it uses) has to structure the data so the Reader can read it, then perhaps it might as well simply call readRecords in the first place.

Providing an object that contains 'responseXML' is minimally useful for ensuring that the correct Reader is being used. But, it is just as reasonable to suggest that one just pass the data (json string, array of objects, array of arrays, DOM node, etc) to readRecords, and let it blow-up if it's the wrong kind of data.

Reusability

I would maintain that there is no good way for a Reader, if it is to be generic and reusable, to extract metadata from a response. This is wholly the job of the Proxy (or code it uses). It could be put into a subclass of the Reader, but there does not seem to be any real advantage to that, as that subclass would itself be no more reusable than the Proxy that uses it. Moreover, a Proxy that depends on a specific Reader to extract metadata would be more fragile than one that only depends upon a Reader to understand the data format.

For example, one can imagine two subclasses of ArrayReader that each have read methods which know how to extract totalRecords data. One expects the value to simply be the first entry in the array:


[40,['foo', 123],['bar', 456]]

The other expects the value to be in the length field of an object at the first entry in the array:


[{length: 40},['foo', 123],['bar', 456]]

Neither of these subclasses of ArrayReader is likely to be very useful, other than to whatever Proxy class happens to be retrieving data in the particular format. It would be more sensible, and simpler, for these Proxies to simply extract and handle the metadata themselves, passing only the real records to an ArrayReader object's readRecords method.

Store Record Data Cache

I asked this elsewhere, but while implementing a Reader, I noticed a lack of documentation about what it is actually supposed to return. I would suggest that instead of simply returning an anonymous object which happens to contain the fields records and totalRecords, it would be sensible to have a RecordCache class, and to construct and return one of those. This would define (and document) what the Store needs to get, as well as providing a clean interface for creating one.

Something like:


new RecordCache(records);
new RecordCache(records, totalRecords);
recordCache.setTotalRecord(count);

Nothing fancy; just enough to define and document.