Using CommonJS Transport Proposal

[thomasf]

A recent Article on Ajaxian brings me to this topic again, as we have implemented some kind of require/include dependency system here at work for extjs. I can see that there are many approaches and it is not good to reimplement the wheel over and over again. So my question is: Are there any plans to use some kind of Module Loading and dependency loading approach in ExtJS? Are there any plans to include something in the core of ExtJS so developers do not have to think of that problem. I have read it over and over again in the forums, but i want to know it from the Dev team if there are any plans to use something like http://requirejs.org/ (which would be my favorite).
What do other developers say?
Please read the information from the links before posting.

Kind regards
Thomas

[joeri]

Yes, ExtJS needs this. If Ext is to grow in feature set (and it will), we need a way to avoid having the initial payload become any bigger (it's already too big). This means Ext itself needs to be packaged up in a similar way as dojo, with a strong module loading mechanism, and that the code we write on top of it also needs to be able to leverage this module mechanism.

[thomasf]

I would really like to know if that is planned for ExtJS? If the code for ExtJS would be organized in such a way we can simply "autoload" not included classes at runtime. It would also allows us to map the xtypes to the appropriate file and so one could autoload scripts on demand.

For example:
Ext.chart.BarChart could be mapped to 'Ext/chart/BarChart.js' and could have the xtype 'Ext/chart/BarChart' so the ComponentMgr could easily map those xtypes to files and therefore load those classes on demand.
Such an approach has some very useful advantages:

• It is much clearer for new developers how to write, name and organise there written code.
• It gets easier to use user extensions / plugins / components because they have their own "namespace" and therefore named that way so it the do not collide with oder ones.
• Easier to deploy a custom set of needed features of ExtJS
• Easier to optimize (combining and minimizing) code. Either deployment stage or on-the-fly per request

It should be possible and even easy to use RequireJS with ExtJS. What do you say?

[edspencer]

This is a complex issue and one we need to think carefully about. The expected performance benefits often do not emerge from this technique due to the fact that network access is not free (e.g. there is latency and overhead involved in each http request).

In a large number of cases you're better off just bundling the component in one large request at the beginning and saving all the http overhead, latency and inefficient use of packets that come with lazy-loading.

For example if your network latency is 100ms and your network bandwidth is 5mbps (reasonable estimates for an office environment), then loading a 500kb file would take 900ms total. On the other hand, if you were to load just the essentials up front (call it 250kb), then load the 10 components you use on demand (say they're 20kb each), you've got a 500ms initial load time followed by 10 more requests that take 132ms each, totalling 1820ms.

Obviously the numbers above are rather arbitrary, and depend a lot on your network's performance characteristics, but it's clear that this isn't just an open and shut case. Ideally the minimal set of code would be built into an application-specific build of Ext JS and delivered on page load, providing a good middle ground between both strategies.

[thomasf]

This is a complex issue and one we need to think carefully about. The expected performance benefits often do not emerge from this technique due to the fact that network access is not free (e.g. there is latency and overhead involved in each http request).

In a large number of cases you're better off just bundling the component in one large request at the beginning and saving all the http overhead, latency and inefficient use of packets that come with lazy-loading.

Of course, you are right with your arguments. But it is not only about on-demand or lazy loading and saving some Kbs. Its also about how to organize the files. As i said above it would allow developers to organize their code exactly like ExtJS does and it could be loaded the exact same way with the same approach. On the other hand it is also easier to combine and optimize the code you need if it is organized like suggested and if you do so, there is then no HTTP overhead, BUT we have the benefit of a clean structure, easier debugable code, easier extensibility and the possibility to load scripts on-demand, which of course should not and is not my main argument.

Please see also http://requirejs.org/ Site for more details on optimizing

Kind regards

[joeri]

On typical office networks you're not going to see much difference either way. It makes sense to have "big file" builds in that case, because you're not going to benefit much regardless of how you organize your code.

The problems I'm seeing is with servers hosted "far away" from the user. For example, our web apps are going to be rolled out to mining expiditions in australia. Generally these have lousy internet connections (high latency and low bandwidth). If you have to load 300+ KB up front, on a 20 KB / sec line, that's a long wait. Much better to load 100 KB, show a page to the user, have them click, load another 100 KB, have them click again, load another 100 KB. The user may end up waiting longer overall, but the application will "feel" faster.

While I recognize that it's not a clear-cut case of "do this, and the app will load faster", it can often be important to have a choice of where to split the payload. Currently we don't have any choice.

Beyond that, even if the core ExtJS library is going to remain a single "big file", I really, really, need on-demand loading for the rest of the functionality. The goal of the apps I'm building out is to end up with many megabytes of javascript code. Even on office LAN's you have to use on-demand loading in that case. I've built my own on-demand loader, and it's serving me fine, but it doesn't make sense that everyone that wants to use Ext to build the sorts of apps it is really designed for has to build the same sorts of extensions.

[Mike Robinson]

To me, the most appropriate solution would seem to be ... not demand-loading as suggested, but rather, "custom builds."

For instance: if my application does not use charts, and it never will, then I don't need to deploy an ExtJS payload along with it which includes any of this functionality.

Ideally, a tool would be provided that could parse an entire source-tree, pick out the various Ext components and calls that occur in the code, and make an ExtJS deployment file that contains only the functionality needed.

[mschwartz]

This is a complex issue and one we need to think carefully about. The expected performance benefits often do not emerge from this technique due to the fact that network access is not free (e.g. there is latency and overhead involved in each http request).

In a large number of cases you're better off just bundling the component in one large request at the beginning and saving all the http overhead, latency and inefficient use of packets that come with lazy-loading.

For example if your network latency is 100ms and your network bandwidth is 5mbps (reasonable estimates for an office environment), then loading a 500kb file would take 900ms total. On the other hand, if you were to load just the essentials up front (call it 250kb), then load the 10 components you use on demand (say they're 20kb each), you've got a 500ms initial load time followed by 10 more requests that take 132ms each, totalling 1820ms.

Obviously the numbers above are rather arbitrary, and depend a lot on your network's performance characteristics, but it's clear that this isn't just an open and shut case. Ideally the minimal set of code would be built into an application-specific build of Ext JS and delivered on page load, providing a good middle ground between both strategies.

I think this is correct.

Unfortunately, ExtJS is client-side only, so there's not much you guys can do other than show us example code in a few server-side languages...

Saki's thread about ripping through the .jsb file and generating script tags for each of the ExtJS source files for debugging is just the start of what can be done server-side.

To take it one step further, the same kind of logic could be done to join all the .js files together and deliver it in one payload. Though ext-all-debug.js is basically this.

Where it gets interesting is if you can use something other than the .jsb, like an array of Ext core scripts to include. Some subset of ext-all that is perfectly suited for your app. Add to that array through some automagic mechanism all the Ext core scripts that the ones in the array depend on. Concatenate all the scripts dictated by the file names in the array and you have a custom ext-small.js kind of file.

Doing just one request for that concatenated .js file, and doing gzip on the server, and it's a pretty small payload, all things considered.

Obviously you can go even further by running the concatenated file through a minifier. And one more step would be to cache that resulting file so you don't have to do the collect dependencies/join/minify steps very page access.

[joeri]

The way I see it, you can have two viewpoints:
1. Everything should be rolled up into one big file, using server-side or design-time build scripts.
2. There has to be a split payload.

The first solution for me is fine for single-purpose apps, and doesn't scale for suites.

When you're using a split payload, the question is reduced to "how many parts do you split across, where do you split, and how do you load?" The way I see it, splitting the payload, for suite-style apps, is an inevitability. When you're splitting things up, having an on-demand loader infrastructure is an inevitability. Therefore, it's a no-brainer for me that this should be part of extjs itself.

What remains up for debate is whether extjs itself must be cut into pieces. The answer to that is "it depends", like with so many things. But at least we should start with having an on-demand loading infrastructure in the library so it's possible to split things up without having to do it all custom-coded.