Many readers have asked us to replicate our browser
speed tests on a Mac, to see the difference in platforms and
performance. So we snagged a new MacBook Pro when nobody was looking and
tested the latest browsers on it.
While this is a pretty
good match to the tests we just finished on Windows,
we're adding in a Firefox 4 pre-release build and the final version of
Opera 10.6—which, all told, probably isn't that different from
the beta, given the short time distance between them. We used the same browser
testing parameters we're regularly using for Windows browsers, with a
few exceptions.
In the case of cold and warm starts, and nine-tab loading, I
could not find an equivalent of my favorite Windows timer, made
by Rob Keir, that can stay on top of any window and be activated by a
custom keyboard shortcut. If you know of such a Mac timer app with those
features, by all means—tell us in the comments or by email. Instead, then,
I had to rely on the Ultimate Stopwatch app for Android, made by Richard
Hyndman. I used one finger to tap the "O" key to launch items with
Command+O, while simultaneously tapping the timer app on my Nexus One, and
tried my best to be consistent.
As for the testing platform, it's a MacBook Pro, bought the day
of the latest hardware "refresh," with the specs shown at left here. It
was kept plugged in for the tests, and running on a LAN cord for page
loading measurements.
One other quick thing
I'm doing different this time around—I'm sharing my data. So if you wanted
to see my actual testing results in numbers—to check my math, craft your
own much more stylish graphs, or what have you—you can view
them at this Google Docs spreadsheet. Note: Link has changed since
first posting.
Now, onto the good
graph-y stuff! Click any of the images below for a larger, wider
view.
Boot-Up and Warm
Loading—Winner: Safari 5!
It's not all that surprising that Apple's software engineers
would have the best shot at getting fast start-ups and smooth integration
down pat. What is surprising, though, are all the other results.
Compared to the Windows
results, everything's almost inverted—Chrome takes longer to start up,
and Chrome beats out Firefox.
Tab Loading—Winner:
Chrome!
I'd have expected Safari 5 to fare better at loading
multiple tabs on a Mac than on Windows, but, alas. It did actually load
the tabs after a while, though, as compared to a freeze-out on Windows.
Chrome continues to be the savviest at tab loading—though there's no
Internet Explorer 8 here to weird out the results.
JavaScript—Winner: Opera
10.6!
I don't entirely trust this result, or, at least, question
whether it's the Dromaeo
test we're using, or Opera's JavaScript coders, or a little of both. Opera
10.6 jumps from an also-ran in beta form to a miles-ahead winner in final
form—unless it's just far, far faster on a Mac. This is, of course, just
one test of certain benchmarks, and not a full picture. Still, beyond
Opera's weird jump, Chrome, Safari, and Firefox performed at about par
with their Windows counterparts.
DOM/CSS—Winner (By a Nose):
Safari 5!
Safari and Opera trade places in Mac testing, while
Chrome keeps pace with second place.
Memory Use (Without
Extensions)—Winner: Safari 5!
One big caveat in this test is that Chrome's developers note
that the browser "seriously over-reports memory usage"—it's a bug report
they link right from the about:memory page we're using to
fairly measure multi-process memory usage. That said, I don't think Chrome
was going to knock down the far-and-away efficiency of Safari, or probably
even catch up to Firefox in second place.
Memory Use (With
Extensions)—Winner: Firefox 3.6.6!
We would have liked to test out a similar set of
extensions on Firefox 4's pre-released, but even after using a
don't-worry-about-compatibility tweak, our test build refused to let us
try three of the five extensions in our testing
guidelines. So we stuck with what we could text, and Firefox again
performed well with five representative add-ons loaded. Many of our
commenters will note that Firefox seems to leak memory over time—we're
brainstorming a testing process that would fairly catch this. Suggestions?
Drop them in the comments.
The Scores
We've tallied up how the
six browsers performed in five tests—cold/warm starts, nine-tab load,
JavaScript, DOM/CSS, and memory use—and we're posting the results here.
It's not scientific, but we tried to be fair in calling winners, and
granting ties when results were so close as to be negligible to the end
user. Here's how these browsers fared, ordered in rank, out of a total of
30:
- Chrome 5.0.375.70 (Stable): 23
- Safari 5 (6533.16): 22
- Chrome 6.0.447.0 (Dev): 21
- Opera 10.6: 18
- Firefox 4.0b2 (Pre-Beta Candidate): 15
- Firefox 3.6.6: 13
Compare that to the
final scores in our Windows tests, and you'll get an inkling that not
all browsers are created equal, especially across operating systems.
As many commenters have
noted, these are just numbers and tests—they don't take into account
standard features you may love about a browser, add-ons you can't get
anywhere else, or a general good vibe you get from a certain browser. We
just like to see how things stack up from an objective perspective, so we
know what to expect when browser makers claim their newest build is,
inevitably, the "fastest ever."
There you have it—our first batch of Mac-based browser results. As noted
above, you can
see
the raw data for yourself, if you'd like. Come up with better graphs
than ourselves? Have a suggestion for better Mac testing tools? We're all
ears in the comments.
No comments:
Post a Comment