PDA

View Full Version : Providing compressed (gzip) .js file



liotrox
6 Dec 2006, 4:35 AM
Hi all,

there is a way to put in the server a mylib.js.gzip file and reference the same in the HTML
code with a <script> tag like:

<script source="mylib.js.gzip" type="????" >

I know that server-side sending appropriate HTTP header you can send compressed content what I want to avoid is to create a server-side code to send this gzipped content.

Thanks
Angelo

brian.moeskau
6 Dec 2006, 8:56 AM
You can place gzipped files on the server, but the purpose is for the browser at the other end of the line to unzip it before rendering. You would never actually reference the file as a .gzip file in code -- it will always have been unzipped by the browser by that time. You'll still have to handle stuff on the server side correctly to make this work (either checking HTTP headers as you mentioned, or possibly configuring your web server depending on what your setup is -- I think Apache can be made to handle this automatically with a plug-in, while IIS is a bit more of a pain).

liotrox
7 Dec 2006, 2:36 AM
You can place gzipped files on the server, but the purpose is for the browser at the other end of the line to unzip it before rendering. You would never actually reference the file as a .gzip file in code -- it will always have been unzipped by the browser by that time. You'll still have to handle stuff on the server side correctly to make this work (either checking HTTP headers as you mentioned, or possibly configuring your web server depending on what your setup is -- I think Apache can be made to handle this automatically with a plug-in, while IIS is a bit more of a pain).

It is clear that the .js.gzip files will be put on the server. What I want is to transfer, on the network, compressed files and not to decompress the same at server-side with an Apache/IIS plugin.

The browsers are already capable to decompress on-the-fly the .gip content but currently I know that they need some HTTP header in the HTTP answer to manage it correctly.

So I was asking if there is a way in the HTML source code to reference directly a .gzip files.

Thanks
Angelo

brian.moeskau
7 Dec 2006, 7:04 AM
So I was asking if there is a way in the HTML source code to reference directly a .gzip files.

And I was answering no :)

Choleriker
7 Dec 2006, 4:40 PM
Web.config in <system>:


<httpModules>
<GZIP>
<add>
</httpModules>

And the compression-filter:



using System;
using System.Collections.Generic;
using System.IO;
using System.IO.Compression;
using System.Text;
using System.Web;

namespace tecbehind
{
/// <summary>
/// http://www.microsoft.com/belux/msdn/nl/community/columns/desmet/compression.mspx
/// </summary>
public class HttpCompressionModule : IHttpModule
{
public HttpCompressionModule()
{

}

#region IHttpModule Members

void IHttpModule.Dispose()
{

}

void IHttpModule.Init(HttpApplication context)
{
context.BeginRequest += new EventHandler(context_BeginRequest);
}

#endregion

bool useCompressionParsed = false;
bool _UseCompression = false;

public bool UseCompression
{
get
{
if(useCompressionParsed)
return _UseCompression;

try
{
_UseCompression = bool.Parse(System.Configuration.ConfigurationManager.AppSettings["UseRawCompression"].ToLower());
}
catch
{
}
finally
{
useCompressionParsed = true;
}
return _UseCompression;
}
}

void context_BeginRequest(object sender, EventArgs e)
{
if (UseCompression)
{
string desName = HttpContext.Current.Request.Url.AbsolutePath.ToLower();
if (desName.EndsWith(".js") || desName.EndsWith(".css"))
return;
//
// Get the application object to gain access to the request's details
//
HttpApplication app = (HttpApplication)sender;

//
// Accepted encodings
//
string encodings = app.Request.Headers.Get("Accept-Encoding");

//
// No encodings; stop the HTTP Module processing
//
if (encodings == null)
return;

//
// Current response stream
//
Stream baseStream = app.Response.Filter;

//
// Find out the requested encoding
//
encodings = encodings.ToLower();

if (encodings.Contains("gzip"))
{
app.Response.Filter = new GZipStream(baseStream, CompressionMode.Compress);
app.Response.AppendHeader("Content-Encoding", "gzip");
#if DEBUG
app.Context.Trace.Warn("GZIP compression on");
#endif
}
else if (encodings.Contains("deflate"))
{
app.Response.Filter = new DeflateStream(baseStream, CompressionMode.Compress);
app.Response.AppendHeader("Content-Encoding", "deflate");
#if DEBUG
app.Context.Trace.Warn("Deflate compression on");
#endif
}
}
}
}
}


But i know, there is even a simple way in IIS settings to do that too. Google knows all :)

Michael

[/code]

Animal
8 Dec 2006, 7:16 AM
I'm doing this for my project. I can't tell how small the GZipping makes it because the Fiddler tool only shows the first chunk of chunked encoding.

My build uses Ant to concat all js files into the clases directory under the name "Aspicio-debug.js", and then creates "Aspicio.js" by minifying it. This goes into the web app's classpath, so that a ClassLoader can access it.

I have a servlet which accesses that using the current ClassLoader, checks the modified date against the "If-Modified-Since" header, and outputs the bytes through a GZIPOutputStream.

I set the Content-Type to "text/javascript", and the Content-Encoding to "gzip", set the Last-Modified appropriately, and off it goes...

Here's how I build the concatenated, minified file in Ant. If anyone can make it better, please post your code. The problem is that the YAHOO base files need to be included in a certain order. If you just use a <fileset> with includes, it uses directory order, not the order of your includes.... so I use multiple <concat> tasks to get the order-critical files in, and then another <concat> excuding those.




<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="no">
<fileset dir="${basedir}/WebRoot/js">
<include name="yahoo.js"/>
</fileset>
</concat>
<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="yes">
<fileset dir="${basedir}/WebRoot/js">
<include name="event.js"/>
</fileset>
</concat>
<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="yes">
<fileset dir="${basedir}/WebRoot/js">
<include name="dom.js"/>
</fileset>
</concat>
<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="yes">
<fileset dir="${basedir}/WebRoot/js">
<include name="logger.js"/>
</fileset>
</concat>
<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="yes">
<fileset dir="${basedir}/WebRoot/js">
<include name="connection.js"/>
</fileset>
</concat>
<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="yes">
<fileset dir="${basedir}/WebRoot/js">
<include name="animation.js"/>
</fileset>
</concat>
<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="yes">
<fileset dir="${basedir}/WebRoot/js">
<include name="dragdrop.js"/>
</fileset>
</concat>
<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="yes">
<fileset dir="${basedir}/WebRoot/js">
<include name="treeview.js"/>
</fileset>
</concat>
<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="yes">
<fileset dir="${basedir}/WebRoot/js">
<include name="container.js"/>
</fileset>
</concat>
<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="yes">
<fileset dir="${basedir}/WebRoot/js">
<include name="menu.js"/>
</fileset>
</concat>
<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="yes">
<fileset dir="${basedir}/WebRoot/js">
<include name="yui-ext.js"/>
</fileset>
</concat>
<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="yes">
<fileset dir="${basedir}/WebRoot/js">
<include name="fcl.js"/>
</fileset>
</concat>

<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="yes">
<fileset dir="${basedir}/WebRoot/js" includes="*.js">
<exclude name="yahoo.js,event.js,dom.js,logger.js,connection.js,animation.js,
menu.js,dragdrop.js,treeview.js,container.js,fcl.js,yui-ext.js"/>
</fileset>
</concat>


<java fork="true" jar="${basedir}/jars/custom_rhino.jar" output="${basedir}/WebRoot/WEB-INF/classes/Aspicio.js">
<jvmarg line="-Xmn100M -Xms500M -Xmx500M"/>
<arg line=" -opt -1 -c ${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js"/>
</java>


I have the Dojo compressor jar "custom_rhino.jar" in a "jars" directory in my build area. You can download that from Dojo.

Once you have the single file on your classpath, the servlet is just:



package com.aspicio.servlet;

import java.io.InputStream;
import java.net.URL;
import java.net.URLConnection;
import java.util.Date;
import java.util.zip.GZIPOutputStream;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import com.aspicio.util.Utils;

public class GetAspicioScript extends HttpServlet
{
public void doPost(HttpServletRequest req, HttpServletResponse resp)
throws ServletException
{
doGet(req, resp);
}

public void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException
{

// Extract the last cached date from the request.
// If present, this will be in RFC822 format eg: "Fri, 08 Dec 2006 09:30:32 GMT"
Date lastAccessDate = null;
String lastAccessDateStr = request.getHeader("If-Modified-Since");
if (lastAccessDateStr != null)
lastAccessDate = Utils.parseRFC822Date(lastAccessDateStr);

try
{
URL u = Thread.currentThread().getContextClassLoader().getResource("Aspicio.js");
URLConnection uc = u.openConnection();

// If the If-Modified-Since header was sent, see if we can save bandwidth.
Date modified = new Date(uc.getLastModified());
if (lastAccessDate != null)
{
if (!modified.after(lastAccessDate))
{
response.setStatus(HttpServletResponse.SC_NOT_MODIFIED);
return;
}
}

// It *has* been modified since they last got it, so send it.
response.setContentType("text/javascript");
response.setHeader("Last-Modified", Utils.rfc822Format[0].format(modified));
response.setHeader("Content-Encoding", "gzip");

// Copy the bytes from the soure file into the GZIPOutputStream which
// is wrapping the servlet's OutputStream
InputStream in = uc.getInputStream();
GZIPOutputStream out = new GZIPOutputStream(response.getOutputStream());
byte[] buffer = new byte[16384];
int bufferFill = in.read(buffer);
while (bufferFill != -1)
{
out.write(buffer, 0, bufferFill);
bufferFill = in.read(buffer);
}
out.close();
in.close();
}
catch (Exception e)
{
e.printStackTrace();
throw new ServletException(e.getMessage(), e);
}
}
}


Where my Utils class contains



public static final SimpleDateFormat[] rfc822Format = new SimpleDateFormat[]
{
new SimpleDateFormat("EE, d MMM yyyy HH:mm:ss z", Locale.getDefault()),
new SimpleDateFormat("EE d MMM yyyy HH:mm:ss z", Locale.getDefault()),
new SimpleDateFormat("d MMM yyyy HH:mm:ss z", Locale.getDefault()),
new SimpleDateFormat("EE, d MMM yyyy HH:mm:ss", Locale.getDefault())
};

public static Date parseRFC822Date(String rfcdate)
{
for (SimpleDateFormat d : rfc822Format)
{
try
{
synchronized (d)
{
return d.parse(rfcdate);
}
}
catch(ParseException e)
{
}
}
return null;
}

Choleriker
8 Dec 2006, 9:21 AM
Hey animal,

is that the only lib provided by dojo you are using or do you using the dojo widgets and libs? What are your experiences with dojo?

Mines are: dojo is a cool library but if you are trying to get huge projects running with dojo, the libraries are to huge of size and very very slow.

Michael

Animal
8 Dec 2006, 9:24 AM
I've found Dojo to be huge, buggy, undocumented and basically dodgy. I've seen comments in the code like "// Don't understand this" and lots of bad practice. It basically does not work in FF.

brian.moeskau
8 Dec 2006, 10:08 AM
So back to the OP's question, there are many solutions to gzip, but they basically all involve setting up a server side solution of some sort -- HTTP handler and/or IIS configuration for .NET, servlet in Java, Apache configuration, etc. The gzip'ed content always gets unzipped by the browser while reading through the HTTP request -- you wouldn't ever refer to the gzip'ed file in your code in a script tag with a .gzip extension as in your example. In order for this to work, you have to instruct the browser on what content-type you're sending and check to make sure it's supported. AFAIK, there's no way around doing something on the server to handle it.

Animal
11 Dec 2006, 3:28 AM
Investigating Rereading the Ant manual, I've simplified the code.

It turns out, using a <filelist> element allows you to specify file order, so, I first concat in all the base YAHOO files, then yui-ext, then the order-dependent application js files, then all the rest with an excludes list to ensure previously concatenated files aren't included again:




<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="no">
<filelist
dir="${basedir}/WebRoot/js"
files="yahoo.js,event.js,dom.js,logger.js,connection.js,animation.js,
dragdrop.js,treeview.js,container.js,menu.js,
yui-ext.js,
fcl.js,ListManager.js,InputField.js"/>
</concat>
<concat destfile="${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js" fixlastline="yes" append="yes">
<fileset dir="${basedir}/WebRoot/js"
includes="*.js"
excludes="yahoo.js,event.js,dom.js,logger.js,connection.js,animation.js,
dragdrop.js,treeview.js,container.js,menu.js,
yui-ext.js,
fcl.js,ListManager.js,InputField.js"/>
</concat>


<java fork="true" jar="${basedir}/jars/custom_rhino.jar"
output="${basedir}/WebRoot/WEB-INF/classes/Aspicio.js">
<jvmarg line="-Xmn100M -Xms500M -Xmx500M"/>
<arg line=" -opt -1 -c ${basedir}/WebRoot/WEB-INF/classes/Aspicio-debug.js"/>
</java>

Animal
14 Dec 2006, 7:50 AM
Using the latest Firebug, I see that the 1569K concatenated file which contains all our javascript is 312K when gzipped.

jack.slocum
14 Dec 2006, 8:25 AM
Holy crap. 1569K? You must have every JS library ever written included. :)

Animal
14 Dec 2006, 8:40 AM
Most of the YAHOO base which comes to 949K, plus yui-ext which is 404K, and then our own utilities and widgets.

Our UI is by no means finished either.

jack.slocum
14 Dec 2006, 8:59 AM
Is that uncompressed or something?

Animal
14 Dec 2006, 9:07 AM
Yes. Compressed using the Dojo compressor it's 851K