CSS Sprites have been around for a while, and keep coming up as a way to get blazing fast speed to your site.  Steve Souders just showed off SpriteMe! at Velocity '09 (speaking of — why use the CSS Sprite Generator or other server-based tools when you can use <canvas> and toDataURL and generate sprites on the fly?).  However, there are a number of misconceptions about CSS Sprites, the main one being that they have no downsides.

The basic idea of CSS Sprites is to combine a number of images used on your site into a single image, thus reducing the number of HTTP requests that need to be made to your site.  The image is rendered using a CSS background and background-position (which, incidentally, means that your markup becomes more complex; the image to use is specified in CSS, not in a plain <img> tag).

The biggest problem with CSS sprites is memory usage.  Unless the sprite image is carefully constructed, you end up with incredible amounts of wasted space.  My favourite example is from WHIT TV's web site, where this image is used as a sprite.  Note that this is a 1299x15,000 PNG.  It compresses quite well — the actual download size is around 26K — but browsers don't render compressed image data.  When this image is downloaded and decompressed, it will use almost 75MB in memory (1299 * 15000 * 4).  If the image didn't have any alpha transparency, this could be maybe optimized to 1299 * 15000 * 3, though often at the expense of rendering speed.  Even then, we'd be talking about 55MB.  The vast majority of this image is blank; there is nothing there, no useful content whatsoever.  Just loading the main WHIT page will cause your browser's memory usage to go up by at least 75+MB, just due to that one image.

That's not the right tradeoff to make for a website.

Another (but much less important) downside is that if a sprite-using page is zoomed using the full-page zoom feature that's present in many browsers, the browser may need to do extra work to get the correct behaviour at the edges of these images — basically, to avoid adjacent images in the sprite from "leaking in."  This isn't a problem for small images, but can be a performance hit for bigger ones.

There are cases where CSS sprites are clearly beneficial; the main example is combining a bunch of small similarily-sized images into a single one.  For example, a bunch of 16x16 icons used as indicators for various things on your site, or a bunch of 32x32 icons used as category headers or similar.  But combining images with vastly different dimensions, especially when there are skinny and tall images in the same sprite as wide and short ones is almost never a good idea.

However, sprites do improve page load time (at least initial page load — on subsequent page loads, the browser hopefully has the images cached).  What's the alternative?  Unfortunately, there isn't one, yet.  Many browsers have support for offline manifests already; it might be possible to extend that to allow downloading one file (like a jar/zip file) that contains a manifest of resources and equivalent URLs that are contained inside it.  But any such work won't show up on the web for some time yet.

So, when deciding whether to sprite or not to sprite, keep in mind that there are factors in play other than raw page load performance.  As a general rule of thumb, if most of your sprite doesn't contain real image content, you should probably avoid using it.  Also, keep an eye out for future solutions that will preserve both the page load speed, as well as avoiding the negative memory and performance impact of sprites.