PDA

View Full Version : Correct way to load a single store from multiple similar JSON sources?



daubman
21 Nov 2011, 11:22 AM
What is the correct way to populate a store (that will be providing data to a grid view) from multiple similar (but not identical) JSON sources?

On the server side (that I do not directly control and thus cannot 'fix') there are three endpoints providing JSON data via ajax to my client.

The client is a simple grid panel that has ~15 columns.

Each of the three different data source (yes, all from same domain) provide records with (more than) these 15 fields. However, here are the differences among them:

- Each datasource will have a different mapping for some of the fields (the structure of the JSON is not the same)
- Each datasource will have a different root and record specification for the JSON Reader
- Each datasource will (obviously) have a different url

I could handle the first two issues by adding custom 'convert' functions to each field that differs across sources, however, I don't know how to solve the third issue (different URLs). Also, this does not seem ideal.

I imagine this isn't quite a one-off situation (multiple similar but non-identical data-sources that all provide data that should be combined in the same store, and, more importantly, displayed in the same grid).

Has anybody run across this before? If so, have you solved it? Is there a recommended approach?

Am I approaching this wrong? Is there another way to add the data provided from several different JSON sources within the same grid? :-/

Thanks,
Aaron

tobiu
21 Nov 2011, 11:48 AM
well, definitely not a fun to code szenario.

on store level you have the method:


setProxy( String/Object/Ext.data.proxy.Proxy proxy ) : Ext.data.proxy.Proxy
Sets the Store's Proxy by string, config object or Proxy instance

Parameters
proxy : String/Object/Ext.data.proxy.Proxy
The new Proxy, which can be either a type string, a configuration object or an Ext.data.proxy.Proxy instance
Returns
Ext.data.proxy.Proxy
The attached Proxy object


you can also create 3 different stores for your usecase and fill a 4th one (that is connected to the grid) depending on your logic.

the last way is to create a custom store / proxy extension.

friend
21 Nov 2011, 12:03 PM
There's usually a one-to-one correspondence between a grid, it's store and the URL invoked by the store; having said that, there's no simple, integrated feature which will address your problem (due to the inherent async nature of Ajax calls). The crux of your problem is as you stated:


On the server side (that I do not directly control and thus cannot 'fix').

I once solved a similar problem by issuing an Ext.Ajax.request() to the first URL and inside the 'success ' block, I cached the response data (to a previously defined store). While still inside the 'success' block of the first request, I then issued another Ajax request which called the second URL, etc., etc.

This is a psuedo-synchronous way of executing multiple, interdependent Ajax requests.

daubman
21 Nov 2011, 12:12 PM
Tobias, I'm not sure I get how manually setting the proxy would help? I'd still be set with all of my set of issues, right?

Also, managing three separate stores and feeding a common store would work, but would result in multiplication of data storage - e.g. as the number of records I'm dealing with grows, the amount of memory used by the browser would grow by 3-times that... which is definitely not desired.

friend, I think we're solving slightly different problems. For my example, any one of the three JSON services provide complete/sufficient data to add a new row to the grid - so I don't need to retrieve any additional data... I just don't have a good way of displaying the complete rows from each of these different data sources since the formatting (and url) of all of them is different, even though the data contents have a lot of overlap.

in pseudocode:
Datasource 1 fields:
name, mapping: d1.user.name
id, mapping: d1.user.numericId

Datasource 2 fields:
name, mapping d2.users.userName
id, mapping: d2.users.myId

Datasource 2 fields:
name, mapping: d3.userRecord.userdata.name
id, mapping: d3.userRecord.userdata.id

a single record from any source will contain the needed name and id fields, they are just mapped differently, and also come from different URLs.

daubman
21 Nov 2011, 12:40 PM
Ok, to make this question more general, how would you create a simple Grid Panel app serving the purpose of aggregating prices of a particular product from several vendors.

Scenario:
Using JSONP, you need to access product information from amazon.com, buy.com, etc...
The values returned all contain the data you need for your grid. However, the mapping into that data is different for each provider.

How do you populate a Grid Panel with something simple like:
Vendor, Item, Description, Price

from all of these vendors so you can sort on price and find the lowest cost option from among all vendors? Another term for this might be "how do you normalize data from different sources to fit into a single store?"

...this *must* have been done before :-/

jay@moduscreate.com
22 Jan 2012, 7:23 PM
The short answer is that the Ext JS data layer was never meant to cover this type of request.

The data proxy is only designed to read from one data provider (client, memory, server). The data model is only designed to a static set of data (fields) mappings from any inbound proxy.

How I would achieve something like this is to write an abstraction layer for a particular data store to fetch data from multiple data sources, and merge the data into one linear set of objects for it to consume with the help of the model and reader.