Browser Benchmarks: Firefox10 Vs. Chrome 16 (VISTA)

Source Technovore

A few days later than planned, but here at last, is the NEW test of browsers. It’s been about 6 months since I last tested them, and all of them have moved on. Well, all except for Internet Explorer 9, which handily gives us a nice benchmark for last time.

We will be testing Firefox, Opera, Internet Explorer9, Chrome, and Safari. Compared with the results of last year, not long after Firefox5 came out, how have things changed?

First things first, I was somewhat goaded into this test. A Firefox fan challenged me, during a discussion in the UK Pirate Party IRC channel to look at Firefox again, when Firefox10 came out. Being an open-minded person, I agreed to look at it.

Vista-based tests were all performed Wednesday, Feb 1 2012 on the standard Vista test machine (specs here). The only hardware change was the addition of an external 1Tb USB HDD, which was disconnected for the tests. The machine was run on a user account set specifically for the testing, and the same order and method used on each browser.

Browser

Version

Release Date

Notes

Chrome

16.0.912.77

January 23  2012


Opera

11.61

January 24 2012


Internet Explorer

9.0.8112.16421

March 14 2011

32-bit version

Safari

5.1.2

November 29 2011


Firefox

3.6.26

January 31 2012

Not used in Memory Test

Firefox

5.0

June 21 2011

Not used in Memory Test

Firefox

10.0

January 31 2012


The inclusion of Firefox 5 was to allow a comparison with earlier benchmark tests along with IE9. In addition, the Long Term Release version of Firefox (currently at 3.6.26) was tested at the benchmarks, but not in the memory test, as it is no longer possible to run two versions at the same time.

Peacekeeper Test.

The peacekeeper test has changed since the last time I benchmarked, and the new version aims to test things in more stable way, and run on more devices. The Firefox 5, and IE9 results can be compared with the previous test’s results if a past comparison is desired.

The results are as follows

Yet again, Chrome is the clear leader. Opera has dropped back a bit from Chrome – they used to be fairly close – meanwhile Firefox and Safari are still pretty close. Internet Explorer, because it’s now 11 months old, is dropping back. By contrast, three of the other four had versions less than a month old, while Safari is 3 months old, and that shows.

If we look at the score breakdown (not graphed this time) it’s clear to see why Chrome’s in the lead. In pretty much every test, it was the front runner, with Opera the only one able to beat it, on two tests – the HTML5 canvas ripple-test, where it was 2 frames per second faster; and DOM operations where not only Opera, but Safari took Google’s client to task. The huge lead for chrome, though, comes from its long-time strength, data parsing, where it was 3.5x faster than all the others. It’s also the only client to handle all the video playback codecs.

It’s a clear win for Chrome here.

Sunspider

As always, we check out the javascript engine with the sunspider 0.91 test.

Sunspider 0.9.1 (shorter are better)

Here we can see the results. Some definite progress has been made by Mozilla, and it’s certainly catching the IE browser quickly. Opera has actually slowed down some, back to the same sort of performance of 2 years ago. Perhaps it was a testing error, but we will have to wait and see.

HTML5

Using the tests from html5tests.com, all browsers picked up some points, except IE9. Surprisingly, that also includes FireFox5, which gained 10 points from its last test back in the summer despite the browser being completely untouched. I can only attribute that to the test being altered to conform to the new html5 specs.

While only Chrome managed to break the 300 point mark last time, this time all except IE9 and the older Firefox’s made it. Opera is where Chrome was, while firefox has just nudged ahead for once. Chrome is way off in the distance though as we can see here.

Memory

One of the more frequent criticisms of both Chrome and Firefox has been their memory usage. To test the real-world memory usage, an 8-tab assortment of pages was used, which are surprisingly system-heavy. They were:

The memory test in progress
This gave a balance of data-heavy pages, and active content pages. Browsers were run simultaneously, to ensure they all had the same content. Each browser had the first tab set up, then every browser had the second, with all actions being the same. The figures were collected from Chrome’s memory manager, but were checked (which is one reason this took a week to get out) using figures from Process Explorer.

The breakdown is as follows.

Surprisingly, Chrome wasn’t the worst one, as most people would assume. Internet Explorer was worse in every department, and Opera was also worse in all-but-one group. The surprise though has to be Safari. It was far and away the lightest on the system. However, if I had tried another 8 tabs (and when devising this test, myself and Illunatic did come up with about 20 in total, including some NSFW) we might see a difference.

Conclusion

Which is the best browser for Vista? It really is hard to beat Chrome. Not only for performance, but for adopting the new technologies, it’s way ahead of the rest. Doubtless it was the reason Mozilla has attempted to emulate Google with the rapid release schedule. Opera is also hanging in there too. If speed isn’t the top priority, but reliability and built-in features are, or if you’re just not happy with Chrome/Google’s privacy position, then it’s certainly the one to go for.

Firefox, while it’s improving, still requires too much in the way of outside software to be made ‘full features’, and while it’s a criticism that can also be thrown at Chrome, it doesn’t have Chrome’s speed, while its security is starting to look a little ragged. The 3.6.x release is also showing slight improvement, perhaps as much as they can squeeze out of it without a radical code overhaul, and the point of continuing the 3.6 path is that it’s NOT an overhaul.

Safari is competent, and a choice if you’ve got memory limitations, yet it’s just ‘competent’ and not ‘Good’ or ‘great’ at anything in particular. As for Internet Explorer, what was decent 11 months ago is starting to look a little long-in-the-tooth now, even despite its Sunspider score. While it’s certainly better than previous versions, Microsoft needs to update things more often, at least every 6 months. However, with Internet Explorer 10 already in development, it’s unlikely to happen.

Windows7 Test coming very soon DONE

Raw Data

Browser

Chrome

Opera

Safari

IE

Firefox

Firefox

Firefox

Version

16.0.912.77

11.61

5.1.2

9.0.8112.16421

10.0

5.0

3.6.26

Position

1

2

4

5

3

6

PEACEKEEPER

Overall Score

2629

111912

1410

1121

1484

1143

641

Rendering

65.10

68.52

58.17

36.99

48.49

36.65

28.61

RenderGrid01

fps

140.17

160.46

149.19

66.76

177.00

174.59

94.39

renderGrid02

fps

118.83

133.28

127.71

66.73

116.31

113.58

58.26

renderGrid03

fps

18.14

24.23

13.02

7.78

5.54

4.76

5.82

renderphysics

fps

59.43

42.55

46.14

53.99

48.46

19.12

20.90

HTML5 Capabilities

of 7

7

5

3

3

6

6

3

webglSphere

fps

58.04

NO

NO

NO

60.79

61.22

NO

VideoPosterSupport

Yes

Yes

Yes

Yes

Yes

Yes

Yes

videoCodecH264

Yes

NO

Yes

Yes

NO

NO

NO

videoCodecTheora

Yes

Yes

NO

NO

Yes

Yes

Yes

videoCodecWebM

Yes

YES

NO

NO

Yes

Yes

NO

workerContrast01

ops

2405.60

1738.97

3658.45

3750.75

Workercontrast02

ops

3783.73

2555.20

6486.82

3906.66

Gamingspitfire

fps

50.12

51.69

51.74

53.28

55.60

55.17

53.28

HTML5 Canvas

23.82

25.58

9.15

10.28

23.00

17.76

6.39

experimentalRipple01

fps

37.46

39.51

14.34

16.58

36.32

27.86

10.30

experimentalRipple02

fps

15.14

16.57

5.84

6.37

14.56

11.32

3.96

Data

56369.14

16677.73

17978.98

17325.7

17706.15

13937.04

9876.45

arrayCombined

ops

9543.0

3721.0

5669.0

4169.5

4587.0

3871.0

3887.0

arrayWeighted

ops

332964.5

74750.5

57019.5

71994.24

68347.0

50178.5

25095.0

DOM operations

ops

9077.41

9674.29

9239.96

3541.38

7034.93

5362.16

3503.7

domGetElements

ops

512657.5

132719.0

345499.0

125628.14

304024.0

290994.5

104686.5

domDynamicCreationCreateElement

ops

15289.0

12068.79

23581.5

1437.5

14572.0

5860.5

6319.0

domDynamicCreationInnerHTML

ops

15600.5

26414.5

19405.0

3994.5

22368.5

7170.0

12180.0

domJQueryAttributeFilters

ops

3976.5

10986.5

4313.5

1681.0

2977.0

2711.5

1511.5

domJQueryBasicFilters

ops

1302.0

1561.5

1456.5

1118.5

1014.0

919.04

539.23

domJQueryBasics

ops

3458.5

2253.5

2696.5

994.5

2210.5

1787.5

992.0

domJQueryContentFilters

ops

2174.0

16522.0

2245.5

960.0

876.5

1096.5

748.5

domJQueryHierarchy

ops

5137.5

16522.0

5236.5

4204.5

3840.5

3304.0

1809.0

domQueryselecto

ops

17112.0

17302.0

15588.5

16091.5

18959.5

18573.0

9020.5

Text Parsing

158377.91

90367.15

62998.29

75744.14

51880.51

40048.46

17119.27

stringChat

ops

55575.5

40492.0

21290.0

44892.0

46347.5

47124.0

22805.5

stringDetectBrowser

ops

357355.5

340031.0

184300.0

195694.72

202913.5

141160.0

114453.0

Stringfilter

ops

35279.5

19264.0

32562.5

25987.0

2652.0

3012.5

164.42

stringValidateForm

ops

782809.5

439897.0

212463.5

171232.88

258577.5

84657.0

77276.0

stringWeighted

ops

181681.5

51649.7

36554.5

63775.51

58280.0

60727.5

44337.0

SunSpider

ms

HTML5 Test

Basic

/475

374

329

302

141

332

296

183

Bonus points

13

9

9

5

9

9

4

Memory Usage

Private

Kb

471864

560704

100576

548724

278056

Shared

Kb

17393

21072

28436

35144

30060

Total

Kb

489257

581776

129012

583868

308116

Virtual Memory

Private

Kb

494024

614784

111748

619404

266784

Mapped

KB

184436

16520

17284

272128

118360