Like must web developers I'm always looking for was to improve the techniques that I already have.  This week I was thinking there must be a better way of combining and compressing css and js files to clients, especially core library file that don't change all that often.

I was already combining multiple js and css file into a single stream using the standard output buffering compression built into PHP, but these files where being combined and compressed everything they where requested. There had to be a better way to reduce both server load and improve caching on client browsers.

Then I though why can't I keep the combined & compressed files on the server and stream them out instead.....

After a bit of research I had decided on the following tools and techniques..

1. Add better Cache Control headers (age, expires and last modified, etag)
2. Use gZip encodng if browser supports it
3. Use gzencode to compress and cache the contents of the files
4. Logic to update cache if file changes

Workflow & Logic

First up was to work out the logic for controlling and auto updating the cached version if any of the files where updated

Compressing and Combined Files

Now I was already compressing my out via the built in output buffering compression, so there is no difference in the size of the download file here, but I did need to add a check to see if the browser supported compression and to bypass the cache and stream the file straight to the client.

Using PHP to combine multiple files into a single stream is fairly straight forward using the readfile function in PHP But what I'm proposing doing here is to load the contents of a single file or multiple files into a single variable, then compress this content using the gzencode function and finally writing this compressed content to a new file in the sites cache, so it can be simple read just like the original file would normally be.

Example Compressing File Content

Example Compressing Multiple Files

As an example of how important it is to compress your output when sending css and javascript content to the client, I've setup a couple of javascript examples, consisting of a combined mootools library to simulate a web page and a web application that uses ExtJS. I posted the results below...

An fairly average site will usually attract around 10,00 unique visitors which will approximate to about 40k~50k page views. Using the web example above, this could result in a saving of up to 2Gb of extra traffic that does not have to be download load from that server, and while this does reduce your bandwidth costs, it also increases concurrent number of connections your server's link can handle before it becomes saturated.

But also this affects your end user's experience, by making them wait longer to download as page, or in terms of a web application longer for you application to download, so they can do what they need to do.

In these examples I was able to reduce the transfered file size by almost 80% and that's huge saving with very little effort...

Caching File on Server

Now that the combined files have been compressed and cached on the server, very subsequent request for that javascript or css file can simply be streamed from it's cached version instead of re-compressing the output each time it is request.

As you can see from the result above (btw my virtual server tops out as 50 concurrent connections), the simple act of compressing and caching the compressed file has yielded some great results..

On the small files it has reduced the time taken to complete the 2000 requests by to less than half of what it was when compressing the files on the fly, It also has more than doubled the number of concurrent requests the server could process per sec

On the larger files the results are even better with time taken to process the 2000 request down the less than a third of what it was previously and tripling the number concurrent requests per sec being processed.

At the end of the day what does it mean.... It means on the same hardware and server configuration we can now support 2~3 time more requests for our css and javascript files and also reducing the waiting time for the end users to download each page.

Better Controlling the Browser's Caching

coming soon......

Download & Documentation

coming soon.....