Internet Explorer 9 was officially announced today at the Microsoft Professional Developers Conference. Timed to coincide with this is a blog post from the IE team, An Early Look At IE9 for Developers. It’s chock full of interesting information, but I want to pick on them for a particularly misleading graph. I don’t think it was done intentionally, but it’s still worth thinking about because…

1. Data & details matter
2. This chart’s design is covering up some impressive stats.
3. The other chart on the page is pretty interesting.

The accused party:

According to the IE team, this “shows the relative performance of different browsers on the same machine running the SunSpider test.” In other words, relative Javascript performance. Also note the tooltip states that this shows IE9 is competitive “with FF3.6, Chrome4, and the nightly webkit build.”

Certainly, it looks that way. IE9 looks insignificantly larger than its contemporaries. But does this chart really proves that IE9 is only a bit slower than the competition?

Nope.

There is a difference between what this chart purports to show, and what it actually shows. The inclusion of IE7 and IE8 distort the range wildly, so the results we are meant to compare (IE9, FF3.6, Chrome4, Webkit) are scrunched together near the bottom. This makes any meaningful comparison of those data points rather ineffective.

If you think I’m full of crap and this is a perfectly fine chart, consider an alternative, more truthful approach. I’ve done my best to base this on the data from the original, but the range distortion is such that these numbers may be somewhat off. I’m also throwing out the scale in favor of a relative measurement, since the original point of this chart was to compare relative performance.

Hey, that’s different! So, is IE9 competitive with the other browsers? I would hazard to say that “competitive” to Firefox 3.6 is possible since FF looks to be about 85% of the whole.

Chrome and Webkit are another story. As someone that has spent odious amounts of time lately thinking about performance, I’ve been thrilled when I get a 10-15% reduction in execution speed. IE9 is going to have to manage a ~50-55% reduction just to get where Chrome 4 and the Webkit are at this very moment. If I were on the IE9 team, that would be a pretty sobering realization.

That makes “competitive” a bit of a stretch. By comparison, the world record for the mile-run (3:43) is roughly 40% of my current best, but I doubt I’m in serious contention for the 2012 Olympics just yet.

Earlier, I said the original was covering up some interesting data – let’s revisit that. Even though it is meant to show relative JS performance in the latest browsers, the chart is much more impressive as a testament to how far IE9 has already come. Again, consider this alternative:

There are still nits to pick with this chart, but it makes clear just what an achievement the IE9 team has pulled off even if they shipped today. IE9 is already in the neighborhood of 5-6x better in script performance than IE8, and around 15x better than IE7. That’s awesome. As developers push increasingly more work onto the client, we need every bit of performance we can get.

So, review time. What have we learned?

1. There is a new version of Internet Explorer in the works.
2. The IE team has made some great progress, but they still have a lot of work ahead of them.
3. Even if you don’t fudge the numbers on your graphs, you can still inadvertently make them misleading.
4. Subsets of large data sets often illustrate much more than the whole.
5. I am a prickly bastard who thinks about charts too much (In my defense, I’ve spent a lot of time in the past two months creating tables and charts for clients)

For more information on how to make a great chart, I can’t recommend The Visual Display of Quantitative Information by Edward Tufte enough. It is a remarkable book.

Gold stars for everyone, including the IE team. 5x is damn impressive, even if it’s not yet par with the other browsers.

-Scott