Direct or REST which is the best? :-)

[dj]

So if i need to use REST for this purpose why should i use Direct and how should i use it because i would then need 2 points of access for clients which can be hard to maintain at some point...

Yes, you would have two different access vectors but that does not have to mean that you have to maintain these two independently. It should be fairly easy to add Ext.Direct's router as a (fairly transparent) layer between the RESTful interface of your application and your client side javascript. So you only have to maintain the RESTful interface. Nevertheless it is added complexity but it also brings you added benefits. Consider what's more important in your case.


PS: Ext.Direct and Ext.Direct's RemotingProvider are not the same. Batching requests is a feature of the RemotingProvider. Ext.Direct is a quite abstract thingy; Comet/Polling provider for it are in the works, a REST provider would be easy to do. So whenever I praise the benefits of Ext.Direct I actually mean the benefits of its remoting provider.

[Nalfein]

Simplicity and browser caching I understand and I (at least with browser caching) agree with. Could you elaborate on why one would loose scalability?

Using REST you can without extra work split your load between multiple machines using just URLs. No central server or a load balancer is needed for that. In a well-designed REST API, when traversing is done using links embedded in resource representation, you can change URL scheming very easily and make your application distributed without much effort.

Persistent connections are a bit ambiguous in this domain so to clarify that we are talking about the same thing: HTTP 1.1 Persistent Connections: keeping the TCP connection open after one HTTP request finished so that further HTTP requests can be made without the overhead of establishing a TCP Connection first. Header involved Connection: keep-alive and Keep-Alive: 123.

Yes. Another great feature of HTTP 1.1 is HTTP pipelining: "The pipelining of requests results in a dramatic improvement in page loading times". Unfortunately there are yet some servers that don't support this and therefore some browsers including Firefox has pipelining disabled by default. You can safely turn it on (described here) and set network.http.pipelining.maxrequests is 2. You can experiment with higher values but some webpages may not work correctly if they limit number of simultaneous requests per user.

But sending multiple XHRs as opposed to only one batched XHR is nevertheless slower because: (a) of the added HTTP overhead (Header, HTTP roundtrip)

That is right, but the overhead can be very small and forgivable compared to simplicity and easiness of debugging. To make it small you use persistent connections and caching.

An Ext JS may group requests logically and send them one after another. If caching is done properly and they are really sent in a persistent connection the perceived response time should be similar.

(b) even modern browser only support 2 or 6 parallel TCP connections to one domain. Pipelining the requests through available TCP connections brings another overhead.

Where is the overhead? You have two channels compared to a one when we use Ext.Direct.

(c) Content encoding algorithms (i.e. gzip/deflate) can only see and process the individual response. That's not as efficient as gzip'ing the batched response (one dictionary, synergy effects).

I would like to see some numbers. Even if we get 2% smaller messages, we are adding Ext.Direct's code.

I cannot quantify the overhead of (a), (b) and (c) but I did see a significant speed improvement after implementing Ext.Direct's RemotingProvider in one of my apps so I assume that it is significant.

I think that that happened because you haven't optimized your REST services properly. It is much more difficult than setting up an Ext.Direct point. You need to use caching and make sure that your requests are sent with appropriate headers. For example, if you miss Content-Length or it is calculated incorrectly, the connection would be dropped. A nice script to test cacheability: http://www.mnot.net/cacheability/

[dj]

Using REST you can without extra work split your load between multiple machines using just URLs. No central server or a load balancer is needed for that. In a well-designed REST API, when traversing is done using links embedded in resource representation, you can change URL scheming very easily and make your application distributed without much effort.

Agreed. Although, you will run into problems (solvable with some effort with e.g. JSONP) when connecting your ExtJS frontend with such a service. Keyword: Same-domain-policy of XHRs.

(b) even modern browser only support 2 or 6 parallel TCP connections to one domain. Pipelining the requests through available TCP connections brings another overhead.

Where is the overhead? You have two channels compared to a one when we use Ext.Direct.

I don't really see that overhead anymore either. In fact more parallel connections can speed up the requests because they can be processed in parallel on the server. (Can be done also with a sophisticated Ext.direct.RemotingProvider router in a more enterprisy environment, e.g. a Java application server or a Rails application with a thread pool that processes the combined requests in parallel. ... but that's rather complex to do)

(c) Content encoding algorithms (i.e. gzip/deflate) can only see and process the individual response. That's not as efficient as gzip'ing the batched response (one dictionary, synergy effects).

I would like to see some numbers. Even if we get 2% smaller messages, we are adding Ext.Direct's code.

It can get far more effective than 2%. The gain highly depends on the data you are sending. In the test I describe below the gain from combining all messages compared to sending them individually is a decrease in size by 90%. Anything is possible between 0% and 90% depending on your data.

I think that that happened because you haven't optimized your REST services properly. It is much more difficult than setting up an Ext.Direct point. You need to use caching and make sure that your requests are sent with appropriate headers. For example, if you miss Content-Length or it is calculated incorrectly, the connection would be dropped. A nice script to test cacheability: http://www.mnot.net/cacheability/

The web application with which I observed this significant performance improvement after implemented the Ext.Directs Remoting API is a Ruby on Rails application. As you might know, Ruby on Rails has ETag handling build-in. Setting max-age or future Expires header (so completely get rid of the GET requests) is not an option for me. The users of that application always want the freshest data. So the If-none-match-header has to be enough.


As I don't really can explain why I saw this significant speed improvement, I came up with a simple demo. It's a slightly pimped /examples/restful/ that you can find with your Ext sources. The enhancements are:

• Up to 20 Grids are rendered. Each has its own grid store. (But they share the same fake-db)
• Optional Ext.direct.RemotingProvider API on top of the RESTful one.
• Optional gzip-encoding of the API calls.
• Rudimentary timing code for timing the XHR load operation.

Have a look for yourself: http://ext.wirsind.info/rest-vs-direct/ (source code included)

The loading times vary - even in the same browser. I haven't investigated why that's the case but I assume that the timing mechanism used ((new Date()).getTime()) isn't the best and varying server load makes the rest.


In my tests in Firefox, Opera and Safari (each Mac&Windows, newest stable versions) the performance of the two different APIs is more or less the same. Sometimes the performance of the REST version is slightly better.

Google Chrome and Internet Explorer (6, 7 and 8) seam to not like the many parallel XHRs of the REST version. Often the XHRs time out - if they don't time out, loading times are ca. 10 times higher than the loading times of the Direct version. This could be a server issue (e.g. the testing server is quite slow and has little RAM thus it is configured for very few max parallel FCGI server) so please do also test the code on your own server if you can.


It would be great if you could have a look at the code and tell me any optimizations I missed. But then, when it is so hard to make a REST API that flawlessly works with an ExtJS frontend and plugging an Ext.direct.RemotingProvider router into your application is so easy...

[durlabh]

Nice sample!

[thesilentman]

Hi @misha,

if you decide using Ext.Direct, take a look at this list of providers:

My personal favorite is this one.

Cheers

[squalo]

Hello
I use a RESTful PHP interface to a MySQL Databse.
CRUD works fine so far. But how can I manage, that 'inserts' or 'updates' ar not done, if some criteria fails?
I know how to do this with SQL , but how is it possible to get this into my ExtJs grid?

"Insert" and "Update" API, which I use, doesn't have return values.
And if I make an extra php function for an first lookup-query (like I would do in Ext.direct),
how can I manage such a call in the well defined REST URLs?

Appreciating any hints.
Thx.