Benchmarked: Firefox Javascript on Linux and Windows - and it's not pretty

TuxRadar

As you might have guessed from our domain name, TuxRadar.com, we're big fans of Linux. But being a fan of Linux doesn't necessarily make you a Linux fanboy - the kind of person who blindly ignores anything negative about their passion of choice as if that somehow made it better.

In fact, we think more Linux users need to admit there are some places where Linux isn't quite as good as its competitors. And one place where that's certainly true is in web browsing. Don't believe us? We did some simple JavaScript benchmarks of Firefox 3.0 using Windows and Linux to see how it performed across the platforms - and the results are pretty bleak for Linux.

You might be wondering, "why JavaScript?", and there's a simple reason: online apps such as Google's Gmail and Google Docs, Apple's Me.com, and of course ever-growing (and increasingly popular) JavaScript libraries such as jQuery, Prototype and Scriptaculous. These rely on browsers executing a huge amount of JavaScript, which means browsers have had to put a lot of working into optimising their JavaScript performance.

Proof of this can be seen by the number of JavaScript benchmarks that have appeared over the last few years - in fact, the WebKit team, the V8 team and the Mozilla team have all produced their own benchmarks, so no one can argue that JavaScript performance isn't important.

We wanted to find out just how well the Linux version of Firefox faces up against its Windows equivalent, so we ran the benchmarks from all three browser makers to make sure we got a full range of results. Some specifics you probably want to know:

  • We used Firefox 3.0.6 running on Windows XP SP3 and Fedora 10. Both were running in 32-bit mode.
  • Our test machine was a quad-core Intel Core 2 running at 2.66GHz, with 4GB of RAM and 1.5TB of hard drive space.
  • The three tests we used were: SunSpider, V8 Benchmark 3 and Dromaeo (using "Recommended Tests").
  • We have marked in red where Linux performance falls behind Windows for a given test, and vice versa. In short, red numbers are the ones that scored lower.

SunSpider

Uses the SunSpider benchmark from WebKit.org. Results in milliseconds; lower is better.

Firefox Windows vs Firefox Linux: SunSpider test

SunSpider (lower is better)

Windows 

Linux 

Difference 

Total 

2478.6 

2786.6 

308.00 

3d 

282.4 

344.4 

62.00 

Access 

410 

402.6 

7.40 

Bitops 

312.8 

266 

46.80 

Controlflow 

32.2 

34.4 

2.20 

Crypto 

168.6 

168.8 

0.20 

Date 

212.4 

326.4 

114.00 

Math 

288.8 

321.6 

32.80 

Regexp 

181.8 

226.6 

44.80 

String 

589.6 

695.8 

106.20 

V8 Benchmark Suite

Uses the V8 Benchmark Suite v3 benchmark from Google. Results are measured against a reference system with score 100; higher is better.

Firefox Windows vs Firefox Linux: V8 test

V8 (higher is better)

Windows 

Linux 

Difference 

Total 

241 

182 

59.00 

Richards 

243 

193 

50.00 

DeltaBlue 

298 

238 

60.00 

Crypto 

164 

166 

2.00 

RayTrace 

272 

158 

114.00 

EarleyBoyer 

317 

246 

71.00 

RegExp 

193 

122 

71.00 

Dromaeo

Uses the Dromaeo benchmark from Dromaeo.com. Results measured in test runs a second; higher is better.

Firefox Windows vs Firefox Linux: Dromaeo test

Dromaeo (higher is better)

Windows 

Linux 

Difference 

Total 

50.33 

36.97 

13.36 

3D Mesh Transformation 

94 

70.64 

23.36 

3D Raytrace 

56.52 

44.88 

11.64 

AES Encryption/Decryption 

29.44 

30.31 

0.87 

Arrays 

122.17 

86.72 

35.45 

Base 64 Encoding and Decoding 

40.42 

37.28 

3.14 

Bitwise And 

126.97 

101.82 

25.15 

Code Evaluation 

168.66 

115.89 

52.77 

Compute Bits in Byte 

32.07 

44.4 

12.33 

Compute Bits in Byte (2) 

38.02 

53.36 

15.34 

DNA Sequence Alignment 

83.23 

68.94 

14.29 

DNA Sequence Counting 

35.01 

27.28 

7.73 

DOM Attributes 

67.89 

39.93 

27.96 

DOM Attributes (Prototype) 

95.81 

60.28 

35.53 

DOM Attribute (jQuery) 

73.21 

46.25 

26.96 

DOM Events (Prototype) 

58.45 

36.57 

21.88 

DOM Events (jQuery) 

42.97 

27.11 

15.86 

DOM Modification 

77.79 

55.5 

22.29 

DOM Modification (Prototype) 

37.38 

24.31 

13.07 

DOM Modification (jQuery) 

55.21 

39.58 

15.63 

DOM Query 

283.52 

170.43 

113.09 

DOM Style (Prototype) 

59.7 

40.28 

19.42 

DOM Style (jQuery) 

40.94 

25.79 

15.15 

DOM Traversal 

15.92 

9.51 

6.41 

DOM Traversal (Prototype) 

56.27 

33.83 

22.44 

DOM Traversal (jQuery) 

28.37 

17.33 

11.04 

Date Formatting 

107.57 

72.26 

35.31 

Date Formating (2) 

76.08 

43.85 

32.23 

DeltaBlue Constraint Solving 

39.89 

33.73 

6.16 

Fannkuch 

45.14 

58.28 

13.14 

MD5 Hashing 

78.67 

75.85 

2.82 

N-Body Rotation and Gravity 

45.62 

31.68 

13.94 

Partial Sum Calculation 

139.91 

103.61 

36.30 

Prime Number Computation 

41.75 

45.57 

3.82 

Prime Number Computation (2) 

42.8 

46.27 

3.47 

RSA Encryption/Decryption 

8.11 

8.12 

0.01 

RayTracer 

2.58 

1.63 

0.95 

Recursive Number Calculation 

94.87 

89.82 

5.05 

Regular Expressions 

19.33 

15.12 

4.21 

Richards Benchmarks 

64.14 

51.77 

12.37 

Rotating 3D Cube 

72.84 

62.25 

10.59 

SHA1 Hashing 

74.52 

72.64 

1.88 

Script Unpacking 

7.74 

9.51 

1.77 

Spectral Norm of a Matrix 

69.99 

69.44 

0.55 

String Parsing and Searching 

4.12 

2.87 

1.25 

Strings 

130.57 

116.57 

14.00 

Tag Cloud Creation 

62.17 

43.17 

19.00 

Traversing Binary Trees 

57.03 

42.3 

14.73 

Trigonometric Calculation 

76.27 

83.65 

7.38 

Validate User Input 

51.26 

41.41 

9.85 

 

Answers to some predictable comments

Why didn't you use Firefox 3.1?

We tried using a nightly build of Firefox 3.1 to see how performance might change in the future, but it locked up while running the Dromaeo tests so we opted to leave it for now. To be fair, the browser is still in beta, so it wouldn't really be a good test.

 

What compile options did you use for Firefox?

We, almost certainly like the majority of Linux users, don't compile Firefox ourselves - we used the stock build from Fedora i686. You could probably squeeze more performance out of Firefox by compiling it yourself using some specific options, but then that would hardly be a real-world test, would it?

Seriously, if you think the excuse "but you didn't compile it yourself!" is worthy, STFU and GTFO.

 

Did you test Vista?

No. We used Vista in our last benchmark, and it wasn't terribly pleasant, so we're trying XP instead. Some Linux fans might complain that Vista is (reported to be) slower than XP and thus would yield lower results, but according to our site logs 67.48% of our visitors use XP so benchmarking on Vista would also not be a fair real-world test.

 

Why is Linux slower?

A quick Google can show you that Firefox/Linux performance problems are nothing new, but usually these lie in graphical areas such as slow page refreshes or scrolling. With infamous threads such as this one around, and OpenOffice.org still performing much better on Windows than it does on Linux, it seems more likely to us that the performance problems seen above are less Mozilla's fault and more down to general performance issues on Linux.

Clearly this is an area that's always under construction - people like Michael Meeks take performance seriously, and we're optimistic for the future. Still.

 

I ran those benchmarks myself and Linux was 1,000,000 times faster than Windows!

Uh-huh. Please don't post this kind of nonsense unless you provide the hard figures from the benchmarks and your system specs and your distro - something like the output from "uname -a" ought to do it.

 

These benchmarks are artificial and thus worthless - you eeediots!

Actually, they were designed to test the kinds of things that web pages really do. The Dromaeo test is particularly impressive, building upon Prototype, jQuery and more to produce tests that are designed to reflect real-world JavaScript usage.

 

Hah! This is final proof that LinSUX is a WASTE OF TIME and all LinSUX users are teh GHEY!!!1

Our other box is your Windows box. Now go away.

 

This is irrelevant - I don't care how fast JavaScript is!

That's fine - maybe you have a really fast computer, maybe you avoid JavaScript-heavy sites, or maybe you're just really patient. But performance matters to us, and it matters to a lot of other people too - particularly when we're talking about the same software on two competing operating systems.

Think about it: if Firefox is slow because Mozilla's Linux code is weak, then it's a small (if irritating) problem - perhaps the Mozilla developers are more interested in optimising for Windows users simply because there are more of them. But if it's slow because or GCC or some other more general reason, then it's likely to affect a much larger part of your desktop. If you really don't care about performance, why are you reading this in the first place?

Remember, It's hard to be sure whether it's Firefox or Linux that's to blame, but we're hopeful someone will pick up the baton and give Firefox on Linux a kick in the backside to get it moving faster regardless of what the cause is.

 

This is irrelevant - Linux is better because it's freeeeeeeee!

Yes, we know. There's a reason we use Linux, after all. But given how many computers ship with Windows pre-installed and without an option to get it Windows-less, Windows is effectively $free too.

And yes, we also know you can try declining the EULA and clawing cash back from the manufacturer, but very, very few people do that. The rest of us buy a computer with Windows, blank it, then put Linux/BSD/MikeOS on.

 

You must hate Linux to post this kind of FUD

We admit the results aren't pretty, but we invite you to reproduce the tests yourself and see how you do - we've provided all the links, so please feel free to report your results below.

Update: even Wine beats Linux Firefox - click here to read

You should follow us on Identi.ca or Twitter


Your comments

Known slowness

"it seems more likely to us that the performance problems seen above are less Mozilla's fault and more down to general performance issues on Linux."

Firefox and OpenOffice are well known for being sluggish on Linux, it would be more interesting testing the alternatives to them before saying the problem *is* Linux. Everybody knows that bad ports bring bad results in performance, and I think mozilla was always negligent about Linux, because their windows user base is much bigger.

Opera..?

Hi there. As I was reading this, I had a thought. What about Opera on both Windows & Linux? I know when I use Opera on my Ubuntu system it's always much faster than Firefox. I'd be curious to know how the Windows vs Linux versions compare. :)

Wrong reference to optimization thread

I don't think that referencing the optimization work on OpenOffice is a good idea on this type of benchmarking.

Meeks was working on load/startup time and not on normal operation speed of OO.o

FreeBSD

Hi,

Have you thought to test i.e. FreeBSD?

I doubt that Mozilla are

I doubt that Mozilla are giving their best to improve performance on Linux really. After all, a monitory of their users are on it.

A couple points ...

1. I use Fedora 10 as my default desktop for work and play, and I know that Fedora bloats firefox up so I usually compile it myself. Perhaps a fairer test to use the stock Linux build from Mozilla.

2. A lot of the Windows graphics capabilities are build right into the Windows kernel so it would have an advantage, but we all know that it is less secure.

mathematical illiteracy

How about giving fractional or percentage differences
in performance rather than simple differences.
2700-2600 =100 is not the same as 150-50=100,
but your tables make no distinction.

test with other browser

as previously said make the same test with opera could be interesting, i, and apparently many, have ever see firefox really really slower than others in javascript area, so the opera test could be nice before saying it's a global linux performance issue.

maybe it is, but i don't think.

anyway thanks, this cross OS test was nice :)

Use stock builds

A much better test would be to download the Mozilla build of Firefox and use that on Fedora, rather than rely on possible variance in distribution packaging. You're likely using the Mozilla build for Windows, so that would be comparing apples to apples.

That said, I've noticed the Firefox performance issues on my Debian box so I don't expect a repeated benchmark to yield significantly different results.

Runs faster to me.

I'm using Mandriva Linux and Firefox/OpenOffice run faster (much faster actually) than those same apps in Windows.

Mandriva compiles with code optimized for 686 and higher processors.

Probably you are using a Linux distro optimized for 386 (Debian, ¿Ubuntu?,...). That fact alone can slow down most software. Specially the X windowing system.

Firefox != Linux

Why do you assume Firefox and Linux are the same thing? There are other browsers for Linux too! Opera is a really good one, though it's not open source. But it's available for Linux.

EXA?

Hi there,

What kind of X acceleration were you using (XAA/EXA/UXA/None)?
What graphics module were you running?

I want to be certain that you were running a Firefox benchmark and not a 2D graphics benchmark.

Frank

Duh.

Microsoft spends billions of dollars on Windows.
Windows has much better optimizing compilers.
Windows has far more developers dedicated to writing and optimizing their video drivers.
Mozilla.org dedicates far more resources to the Windows version of Firefox than the Linux version.
Windows has a lot more users, justifying all of the above.

Those are all absolutely true statements, and I would be amazed if Windows wasn't faster than Linux in many things.

That said, I personally find Linux to be a more productive environment than XP for the work that I do. I have a lot of ram in my dual boot XP/Linux work computer and when I am doing development work, I regularly use it all. Under Linux, when you have a database server, web server, compiler, etc all running in the background and using most of the system memory, you can STILL switch over to Firefox and browse the web. When I use Windows, I have to go get a coffee when the CPU and/or memory are taxed.

When you consider the other advantages Linux has (free as in freedom not just price, agile development model, better security, etc) I just don't see a single threaded javascript test being much of a deciding factor. Yes Linux could be faster, but you are going to have to throw a lot of people and money at the problem if you want to make Linux faster at every conceivable benchmark. Good luck with that.

NVidia/ATi/Intel graphics chipset?

I'm with Frank on this one - what graphics card/drivers were you using? And while you are checking that one out, also check which Xorg driver you are using. It can make a HUGE difference - there are plenty of Phoronix benchmarks to show just how different the various card/drivers can be.

This is old news

It doesn't matter how laud the mad penguins scream.
Firefox and Thunderbird experience on windows is much better.
For 99% of normal users, on the desktop, Linux is slower, shakier and more paranoid (as opposed to the utterly false "faster, more stable and more secure").
So please, all screaming penguins shut up, put up and keep working on it(quietly).

Graphics

The Linux graphics stack is the likely culprit here. Luckily, there's plenty of work going on right now, all aimed at sorting out this situation.

Graphics??

How in the heck would 2D graphics performance affect something done without any graphics at all - all these benchmarks are just number crunching. If you think it's acceptable to tell someone, "welp, your Firefox is slow... because you're using NVidia/ATi/Intel!", then you're wrong. Even if it's true, it would only imply some other flaw elsewhere in the system because it worked fine on Windows.

"Probably you are using a Linux distro optimized for 386"

They specifically say i686 Fedora was used.

Not a great comparison

As an engineer who's done a fair amount of work in performance measurement, I don't think this was a very good test. Far more variables need to be tested.

First, different distros need to be tested, not just Fedora. Also, distro-provided Firefox builds should be tested alongside direct downloads from mozilla.com. Windows users don't have distro-provided builds, remember.

A different browser needs to be tested besides Firefox. Opera seems like a likely choice, since it also has versions for both OSes. It wouldn't hurt to also provide results for IE and Konqueror, even though they aren't cross-platform, just to show how they fare in comparison to the other two.

Vista needs to be tested as well, in addition to XP.

Finally, different video cards should be tried: Intel, Nvidia, and ATI, to determine if any bad drivers are to blame.

You simply can't provide any meaningful comparison with only two data points as this article has done. It's nearly worthless. Yes, it takes a lot of work to test all these different combinations, but if you're not going to put in the work to do something properly, then don't do it at all. Don't do a half-ass job.

Openjdk in Fedora

I couldn't find in the article whether you were using the default openjdk in Fedora or the current (jre) from Sun.
Fedora's default is openjdk.
If your not using the same kit, it's not a valid test.
Like others though, I have noticed poor java performance in Linux verses Windows.

I have made another test

Very recently I, out of curiosity, tested how long does it take to start the same Windows XP in Virtual Box on Slackware 12.2 and on Windows XP3. There was no measurable difference, 35 seconds each. This is true number crunching.

I recall some other published in the Net test of different Ubuntu versions. Individual tests were 2 times faster on one Ubuntu than on another.

Thus, I guess the performance differences you discovered may be due to Linux, Fedora, Gnome, GCC or whatever, least likely Linux. The article claims that it is about "Firefox Javascript on Linux and Windows". It is a lie since the article is about "Fedora Firefox on Fedora and Mozilla Firefox on Windows", and it is something very different.

As for the the <<Seriously, if you think the excuse "but you didn't compile it yourself!" is worthy, STFU and GTFO.>> argument, I see it as a smart attempt to mask the true problem your test reveals: do it yourself Linux is very different from what Fedora or Ubuntu users get.

On Slackware 12.2 XFCE4 Firefox 3.0.6 feels much faster than on Windows XP SP3. So what?

Slamd64 and Slackware (both 12.2)

On Slackware and Slamd64 (64bit Slackware) kernel versions 2.6.27 (on both) Firefox 3.0.5 and 3.0.6 benchmarks kill MS XP and Vista by a huge margin. Also comparing fedora Firefox to Mozilla Firefox is not a fair comparison.
This tells me one thing neither are your tests on Fedora or XP fair.

Well, you've certainly

Well, you've certainly stacked the deck haven't you?

"We, almost certainly like the majority of Linux users, don't compile Firefox ourselves - we used the stock build from Fedora i686."
"...according to our site logs 67.48% of our visitors use XP so benchmarking on Vista would also not be a fair real-world test."

This is nonsense. You've cherry-picked your favorite version of windows based upon it's performance and user uptake. Then why did you choose a 32-bit version of Linux when 0% of 'real-world' users would do so? A normal user would be using the x86_64 64-bit version on a 64-bit chip. Not only that, but you've picked an end-of-lifed OS for your Windows contender and not Microsoft's current 'state-of-the-art.' Why? The next version of Windows won't be based upon Windows XP, it'll be based upon Vista (I realize they're related) but Vista is the current implementation.

Tsk, tsk.

Good test.

Thanks for taking time to do this test. Of course, the peanut gallery has plenty of suggestions. . . looks like they should get off their butts and run some of the tests _they_ suggest. Instead of just taking potshots at you. --AA

64 bit vs 32 bit

My experience under both Linux (CentOS) and OpenBSD, which presumably have very little in common OS-wise is that 32-bit Firefox lags far behind 64-bit Firefox, especially in javascript intensive apps (esp Gmail). I'd be interested in seeing those apps benchmarked, (also possibly comparing 32-bit and 64-bit Windows versions). In addition to seeing benchmark comparisons, I'd be very curious if anyone could give an explanation for the disparity (assuming that it is borne out in your benchmarks), as I am at a complete loss for explaining how the distinction could be so clear.

Other benchmarks

I will be very interested to see other benchmarks on firefox.

I have long being suspecting the CSS rendering by firefox on Linux isn't the same as it's on Windows. I have been writing CSS styles for various browsers. It is a painful process to deal with all the differences various browsers have in rendering CSS styles. Obviously, it is quite expected to see the lack of some CSS styles or wrongly rendered styles on IE. But I was shocked to see differences between firefox on linux and firefox on windows. I mean these two are the same thing, aren't they?

Linux kernel itself MIGHT be the culprit here

Lately in the kernel revisions and especially in kernel Fedora 10 ships there have been some pretty bad performance sucking regression(s) related to the scheduler. These are already known and well documented - but solving this serious bug is very difficult because they are so far not able to pinpoint it to a commit or series of commits.

Nice analysis

I think this kind of post has good value. It uses specific tests to provide specific claims. A good place to work from. AFAIK Firefox is open source so if Mozilla won't devote time to making it perform on Linux, it could always be forked.

If these problems are Linux or X issues this test will make it easier to find out.

Denial is not just a river in Egypt

Web browsing on Linux obviously sucks rocks. EVE Online just dropped their unsellable Linux client. Netbooks are dumping their Linux options. Ubuntu is losing money at an astronomical rate. Red Hat is shifting from Linux to JBoss as their primary software.

Face it folks. Linux is dead. Long live Microsoft and Apple, the only true innovators in computing.

Netbooks would use Android (Linux ) with Google (Chrome) browser

Mr.Egyptian.. use your favourite search engine to find out where netbooks are heading before posting. You are already mummified.

Firefox improving less of Firefox's performance on Linux has nothing to do with GNU/Linux's performance.

A proper article from a real benchmarking journalist would not have mis-directive claims in the title / article. Yes you have stated truth in a very sick fashion. Suits the most incompetent non-technical losers only.

Oracle runs faster on linux

At a former employer we did extensive testing between Windows and Linux running Oracle 9x and 10x database. Oracle always ran 10% to 33% faster on Linux.

your linux benchmark scores are puzzling

First let me say that I admit that there are things that run faster/better on other OS's than Linux. With that said, I use only Linux, and cannot test on Windows or any other OS other than Linux.

MY laptop specs are far less cutting edge than your system specs were:

Intel(R) Pentium(R) M processor 1600MHz
1 Gb Memory
Old IDE disk
Firefox version 3.0.6
kernel version: 2.6.27-gentoo-r7-23-dec-2008

My scores are this:
Score: 92.8
Richards: 69.2
DeltaBlue: 116
Crypto: 79.0
RayTrace: 87.5
EarleyBoyer: 128
RegExp: 89.9

So, my score on the google test is less than half of yours, and lower is better.

My SunSpider test also blows away both your windows and linux score, too:

RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 10235.8ms +/- 7.1%
--------------------------------------------

3d: 1282.0ms +/- 27.0%
cube: 510.6ms +/- 45.6%
morph: 328.4ms +/- 40.1%
raytrace: 443.0ms +/- 43.8%

access: 1440.8ms +/- 15.8%
binary-trees: 191.0ms +/- 44.9%
fannkuch: 566.2ms +/- 39.8%
nbody: 450.2ms +/- 32.9%
nsieve: 233.4ms +/- 36.7%

bitops: 1224.8ms +/- 11.6%
3bit-bits-in-byte: 230.0ms +/- 54.3%
bits-in-byte: 335.6ms +/- 25.2%
bitwise-and: 293.6ms +/- 49.9%
nsieve-bits: 365.6ms +/- 30.6%

controlflow: 117.0ms +/- 44.6%
recursive: 117.0ms +/- 44.6%

crypto: 737.4ms +/- 11.1%
aes: 329.4ms +/- 22.6%
md5: 253.6ms +/- 48.4%
sha1: 154.4ms +/- 62.3%

date: 1037.0ms +/- 16.7%
format-tofte: 652.2ms +/- 28.0%
format-xparb: 384.8ms +/- 42.9%

math: 1067.6ms +/- 24.2%
cordic: 438.8ms +/- 26.4%
partial-sums: 379.4ms +/- 39.5%
spectral-norm: 249.4ms +/- 43.9%

regexp: 752.6ms +/- 16.7%
dna: 752.6ms +/- 16.7%

string: 2576.6ms +/- 16.8%
base64: 344.4ms +/- 31.5%
fasta: 653.2ms +/- 18.0%
tagcloud: 475.2ms +/- 29.3%
unpack-code: 783.2ms +/- 33.0%
validate-input: 320.6ms +/- 20.0%

I cannot begin to guess why my old clunker is outperforming your system, and I beleive you did your benchmarks in a fair way. But obviously there is something wrong because I had 12 browser windows open, and 3 other apps running when I ran this test (over wireless Internet) and I blew your higher end system out of the water.

Comparing Firefox 3.0 (Fedora 10) against Firefox 3.0 (Mozilla)

Did some runs on my T60p laptop on three builds of firefox. First, testing the Fedora build against the Mozilla build. The FROM column is Mozilla - the TO column is Fedora 10. Both are 3.0.6. Looks like there is trouble in the unpack code in the Fedora build.

TEST COMPARISON FROM TO DETAILS

=============================================================================

** TOTAL **: *1.06x as slow* 4991.8ms +/- 1.2% 5288.2ms +/- 3.7% significant

=============================================================================

3d: ?? 549.8ms +/- 1.8% 557.6ms +/- 6.1% not conclusive: might be *1.01x as slow*
cube: 1.05x as fast 215.4ms +/- 1.1% 204.2ms +/- 2.8% significant
morph: *1.06x as slow* 153.8ms +/- 1.6% 162.6ms +/- 3.8% significant
raytrace: ?? 180.6ms +/- 5.5% 190.8ms +/- 15.9% not conclusive: might be *1.06x as slow*

access: ?? 647.0ms +/- 1.4% 650.4ms +/- 2.8% not conclusive: might be *1.01x as slow*
binary-trees: 1.06x as fast 142.6ms +/- 4.4% 134.0ms +/- 4.7% significant
fannkuch: *1.02x as slow* 230.4ms +/- 0.6% 235.2ms +/- 0.7% significant
nbody: ?? 193.2ms +/- 1.4% 195.4ms +/- 1.4% not conclusive: might be *1.01x as slow*
nsieve: ?? 80.8ms +/- 11.7% 85.8ms +/- 23.0% not conclusive: might be *1.06x as slow*

bitops: - 470.6ms +/- 0.6% 469.4ms +/- 0.7%
3bit-bits-in-byte: 1.03x as fast 111.2ms +/- 2.0% 107.6ms +/- 1.9% significant
bits-in-byte: 1.03x as fast 127.4ms +/- 1.6% 123.6ms +/- 0.6% significant
bitwise-and: *1.03x as slow* 104.4ms +/- 1.6% 107.2ms +/- 2.6% significant
nsieve-bits: *1.03x as slow* 127.6ms +/- 1.6% 131.0ms +/- 1.9% significant

controlflow: 1.02x as fast 131.2ms +/- 1.2% 128.0ms +/- 1.4% significant
recursive: 1.02x as fast 131.2ms +/- 1.2% 128.0ms +/- 1.4% significant

crypto: ?? 365.6ms +/- 4.0% 369.4ms +/- 1.1% not conclusive: might be *1.01x as slow*
aes: ?? 117.0ms +/- 1.7% 119.4ms +/- 3.2% not conclusive: might be *1.02x as slow*
md5: ?? 123.6ms +/- 2.5% 126.2ms +/- 1.8% not conclusive: might be *1.02x as slow*
sha1: - 125.0ms +/- 10.6% 123.8ms +/- 1.5%

date: ?? 565.2ms +/- 2.1% 578.6ms +/- 2.2% not conclusive: might be *1.02x as slow*
format-tofte: ?? 335.0ms +/- 2.8% 342.0ms +/- 3.5% not conclusive: might be *1.02x as slow*
format-xparb: *1.03x as slow* 230.2ms +/- 2.7% 236.6ms +/- 1.0% significant

math: 1.03x as fast 536.6ms +/- 1.5% 522.6ms +/- 0.6% significant
cordic: 1.06x as fast 212.0ms +/- 1.2% 200.0ms +/- 2.3% significant
partial-sums: ?? 191.8ms +/- 2.9% 192.8ms +/- 3.0% not conclusive: might be *1.01x as slow*
spectral-norm: 1.02x as fast 132.8ms +/- 0.8% 129.8ms +/- 2.6% significant

regexp: *1.07x as slow* 378.6ms +/- 3.1% 406.0ms +/- 16.8% significant
dna: *1.07x as slow* 378.6ms +/- 3.1% 406.0ms +/- 16.8% significant

string: *1.19x as slow* 1347.2ms +/- 1.5% 1606.2ms +/- 12.9% significant
base64: *1.14x as slow* 145.0ms +/- 2.0% 165.4ms +/- 11.7% significant
fasta: - 345.8ms +/- 2.3% 342.2ms +/- 6.7%
tagcloud: *1.06x as slow* 265.6ms +/- 1.8% 281.0ms +/- 6.5% significant
unpack-code: *1.50x as slow* 416.2ms +/- 4.8% 622.6ms +/- 30.6% significant
validate-input: *1.12x as slow* 174.6ms +/- 3.3% 195.0ms +/- 4.5% significant

Now, that's interesting but Firefox 3.0.6 lacks PGO and Firefox 3.1 is now at Beta 2. Lets take a look at that comparison.

FROM is Firefox 3.0.6 (Mozilla build)
TO is Firefox 3.1-beta2 (Mozilla build)

TEST COMPARISON FROM TO DETAILS

=============================================================================

** TOTAL **: 1.95x as fast 4991.8ms +/- 1.2% 2556.8ms +/- 4.0% significant

=============================================================================

3d: 1.83x as fast 549.8ms +/- 1.8% 300.2ms +/- 1.5% significant
cube: 2.09x as fast 215.4ms +/- 1.1% 103.2ms +/- 3.1% significant
morph: 2.97x as fast 153.8ms +/- 1.6% 51.8ms +/- 5.2% significant
raytrace: 1.24x as fast 180.6ms +/- 5.5% 145.2ms +/- 1.5% significant

access: 2.17x as fast 647.0ms +/- 1.4% 297.8ms +/- 5.6% significant
binary-trees: - 142.6ms +/- 4.4% 140.4ms +/- 11.7%
fannkuch: 2.49x as fast 230.4ms +/- 0.6% 92.4ms +/- 2.8% significant
nbody: 4.90x as fast 193.2ms +/- 1.4% 39.4ms +/- 2.8% significant
nsieve: 3.16x as fast 80.8ms +/- 11.7% 25.6ms +/- 11.2% significant

bitops: 7.98x as fast 470.6ms +/- 0.6% 59.0ms +/- 8.6% significant
3bit-bits-in-byte: 55.6x as fast 111.2ms +/- 2.0% 2.0ms +/- 0.0% significant
bits-in-byte: 10.3x as fast 127.4ms +/- 1.6% 12.4ms +/- 5.5% significant
bitwise-and: 27.5x as fast 104.4ms +/- 1.6% 3.8ms +/- 14.6% significant
nsieve-bits: 3.13x as fast 127.6ms +/- 1.6% 40.8ms +/- 11.1% significant

controlflow: - 131.2ms +/- 1.2% 131.0ms +/- 1.3%
recursive: - 131.2ms +/- 1.2% 131.0ms +/- 1.3%

crypto: 4.04x as fast 365.6ms +/- 4.0% 90.6ms +/- 14.2% significant
aes: 2.07x as fast 117.0ms +/- 1.7% 56.6ms +/- 22.6% significant
md5: 5.19x as fast 123.6ms +/- 2.5% 23.8ms +/- 4.4% significant
sha1: 12.3x as fast 125.0ms +/- 10.6% 10.2ms +/- 20.0% significant

date: 1.14x as fast 565.2ms +/- 2.1% 493.8ms +/- 5.2% significant
format-tofte: 1.22x as fast 335.0ms +/- 2.8% 275.2ms +/- 5.3% significant
format-xparb: 1.05x as fast 230.2ms +/- 2.7% 218.6ms +/- 8.6% significant

math: 4.59x as fast 536.6ms +/- 1.5% 117.0ms +/- 25.6% significant
cordic: 3.05x as fast 212.0ms +/- 1.2% 69.4ms +/- 42.7% significant
partial-sums: 5.33x as fast 191.8ms +/- 2.9% 36.0ms +/- 2.4% significant
spectral-norm: 11.4x as fast 132.8ms +/- 0.8% 11.6ms +/- 9.6% significant

regexp: 1.65x as fast 378.6ms +/- 3.1% 229.8ms +/- 35.0% significant
dna: 1.65x as fast 378.6ms +/- 3.1% 229.8ms +/- 35.0% significant

string: 1.61x as fast 1347.2ms +/- 1.5% 837.6ms +/- 1.5% significant
base64: 5.41x as fast 145.0ms +/- 2.0% 26.8ms +/- 13.7% significant
fasta: 2.00x as fast 345.8ms +/- 2.3% 173.2ms +/- 3.7% significant
tagcloud: 1.06x as fast 265.6ms +/- 1.8% 249.4ms +/- 4.4% significant
unpack-code: 1.47x as fast 416.2ms +/- 4.8% 283.0ms +/- 1.7% significant
validate-input: 1.66x as fast 174.6ms +/- 3.3% 105.2ms +/- 5.2% significant

So things are definitely improving. Be interesting to see a comparison for Windows using Beta-2 - that's not something I can do.

"Puzzled User": Bigger is better on V8 benchmark.

Dear Puzzled User:

Lower values are _not_ better on V8 benchmark.Your old clunker is, ahem, clunking.

64 bit ?

Sunspider (total only - lower is better)
win 32 bit = 2936.4ms, win 64 bit = 3602.0ms, linux 64 bit = 3383.6ms

V8 (total only - higher is better)
win 32 bit = 201, win 64 bit = 156, linux 64 bit = 157

Dromaeo (total only - higher is better)
win 32 bit = 43.41runs/s, win 64 bit = 35.78runs/s, linux 64 bit = 30.69runs/s

So while the totals are slightly misleading, FF under linux beats FF under windows in 2 out of 3 tests.
(if you compare non optimised versions rather than cherry picking)
Dual Xeon Quad 2.33GHz, 12 GB RAM, 250 GB drives
Fedora 9 x86-64
Win XP pro 64
All copies of FF from Mozilla, not distro.

FTW

Real world...

Many comments deny it but Firefox (and OpenOffice) is extremely sluggish on Linux and barely usable compared to Windows. Basically, if Firefox didn't have Firebug, I would switch to Konqueror and for OpenOffice, I switched to using TeX because it's so much faster (even MS Office is faster than OO).

I don't know the reasons (I suspect bad ports) - tested on various Ubuntu distributions and Debian Sid for three years (and on different computers too).

Example: I have a webpage with light jQuery js and a few images (real world webpage). In Windows: loads in two seconds. Opera on Linux: loads in two seconds. Konqueror: loads in two seconds. Firefox: completely freezes for five seconds and then loads the page.

THE

I think the fact that WINE runs it faster proves it isn't an inherit limitation of performance in Linux itself. Since wine routes w32 function calls to their equivalent native Linux functions, it still is technically being executed "natively". It shows that Linux itself isn't the culprit. The PGO compiler optimization is probably the best and most plausible reason for this disparity.

Windows 7??

Jeez, XP is ancient and still doing better. I wonder what FF on WIn7 Beta would look like? Better still IE8/Win7 vs. FF/Linux would be interesting.

Anyone can post results for

Anyone can post results for 64bit Linux Firefox vs wine (which is 32bit even on 64bit archs)

32-bit vs 64-bit

Is it possible you're running a 32-bit Firefox on 32-bit Windows, and 64-bit Firefox on 32-bit Linux?

An app compiled for 64-bit (Firefox on Linux) will often be slower than one compiled for 32-bit because the instructions are usually bigger, leading to less efficient use of the CPU instruction cache, and the data values are usually bigger (int is twice as big), so the CPU data cache is also less efficient.

profiling ftw

As expected in a firefox performance thread, a lot of comments have suggested that either graphics or moz's UI layer are at fault. Well, I'm here to say that just this once, graphics isn't the problem. The intel graphics driver and X on my G45 system was down at .05% of CPU usage running the dromaeo.com tests. 99% of the time (and the CPU is pegged for the whole test) is in JS and mozilla. I don't know moz enough to be sure about the moz UI layer suggestions seen here, but it looked like about 60% of the time was outside of the JS interpreter itself and in mozilla's interactions with it.

It's easy to test yourself. Go get sysprof. No, not oprofile, sysprof. Bother your distro until they include it, because this is the best performance debugging tool you'll use. Install the debug symbols for moz from your distro. Click "start", start the benchmark, and click "profile" when you're done. Then click on the lowest high-cpu function on the left side, then click through on the right to browse where the CPU is going. Or just save it to a file and include it with the bug report you file upstream (we love that).

It's just for JavaScript

Generally speaking, performance of Firefox under Linux and Mac OS X is better than under Windows. I can clearly feel the difference.

IE over firefox

I m very disappointed to use IE over firefox on windows because it is much faster eventhough i really dont want to

I knew something was up!

I knew something was up! I can use Windows XP or Vista and Mozilla Firefox and browse digg.com, gmail.com, etc. MUCH smoother and faster than I can on Firefox. Even Wine Firefox was faster.

Sad, sad, sad.

patent attorney

If anyone wants to comment to me on this, I am at genecavanaugh@gmail.com.

I really have no choice in operating systems; I must keep my client's data secure, and that means linux.

However, I often "borrow" a Vista machine, or an XP machine, and do "normal stuff" - stuff I think is common to nearly all users.

My linux machine is always faster than the latest Vista machine, and about on a par with the XP machines (usually "latest machine" again, latest update, etc.).

AND, most important, all the breakins and data leaks on all the machines I know of are Windows, never linux.

Ubuntu and Vista?

Suggestion: repeat the benchmarks running Firefox in Ubuntu and Vista.

That way we'll see how Firefox perform in other distro, and in the future Windows versions.

ffbull - JDK is Java. These

ffbull -

JDK is Java. These benchmarks are JavaScript. JavaScript comes with the browser and needs no installation.

For those who say "Apples to Apples is running the Mozilla version" then you're obviously putting constraints on the tests results to just complicate your argument. Linux isn't just used by people who manually install stuff anymore. Most people use repositories such as Yum, Apt-get, CNR, etc.

Firefox ships with Fedora as it does Ubuntu, and 95% of your Linux users are going to use the one built in unless there's problems, then they'll hope an update from Fedora/Ubuntu respectively fixes it.

I'm very happy with this article as it has some solid evidence of what many Linux users have been screaming about Firefox's performance in Linux.

Thanks for the testing and even more thanks for the commotion. Its what the Linux community needs!

-Tres

LOL

Nice joke there with MikeOS.

Not that I've actually had the chance to try it.

Linux OS is still as fast than Microsoft own OS.

Okay, there seems to be few non-valid assumtions on the article.

First few facts. The Linux kernel, it is monolith. Monolith kernel is same as operating system. Linux kernel == operating system. False believe is that Linux kernel != operating system and that should be forgotten right away.

Second, Wine, it is not emulator, it is API swapper. It change the Win32 calls to Linux OS system calls.

If Linux can run Mozilla Firefox almost as fast than Windows OS, then Linux OS can run Internet Browser as fast than Windows. There is nothing what can proof that Linux OS is slower to run Internet Browsers than Microsofts own OS.
Only, and only then, if Linux + Wine + Firefox (win32) would be ran same speed as the Linux + Firefox (native), then we could make conclusions that Linux OS is slower than NT.

Because Firefox is slower when run as native on Linux OS, the problem exist top of the Linux OS (kernel) and the Internet Browser (firefox) can be the fault itself, but that would need proofs.

I believe, that problem exist between Linux OS (kernel) and the Internet Browser (Firefox), because when using wine, Linux OS runs the Firefox just fine.

But, I doupt that part of the problem can be found from Mozilla too. They have not anymore supported Linux same way as they did earlier when Firefox was still called other names. Since then, they have mainly supported Windows users, the marketing is for Windows users and the development is for Windows users. All plugins comes first for Windows platform. And other platforms needs more own developers to bring out nice plugins.

So, as being a fan of the Linux OS and OS scientist, I do not like that people spread lies that Linux kernel is just a kernel and operating system is Linux + something else (example of GNU, do not even start that political propaganda!). Because then people does articles and tests what just starts the flamewars and the fanboyis of "linux" (actually fanboyis of Linux distributions (!= operating system) who wants own choise to be best) comes up just blaming something, while actually being true, while the article itself being wrong.

Problem is that people assumes that software system is same as the operating system. Operating System is nothing more than small software what runs all the applications and offers them and all software libraries a platform to work on the computer hardware. It does not include anything fancy like Internet Browsers, C-libraries, mediaplayers or texteditors..

So in simple, if the Linux + Wine can run Internet Browser as fast than the compared OS, then Linux OS really is as fast and not slower. You can even edit system libraries or even code own small application, what would slow down internet browser. Even the stupist person knows then that it is the one small application what slows down the browser, not the operating system.

This case was nice and "well" done. Thanks for that. But about scientific test, I hope that you do not blame the operating system from actions, even when you self proof that to be wrong conclusion.

This article rises lots of questions, what is slowing down the Firefox, clearly Linux OS (kernel) can run all just fine, but problem exist somewhere higer of the Operating System. We just need to find out that what causes it and then fix it!

It's The Compiler

Our Windows build uses profile-guided optimization and our Linux build does not. Also, MSVC generally produces faster, smaller code than gcc. That is easily enough to explain the differences you're seeing here. It also explains why running the Windows build under WINE is faster.

Almost all Linux Firefox users are actually using a distro packaged build. So hassle your distro to use PGO if they aren't already.

robert@ocallahan.org

Oh, just to allay conspiracy theorists: far more Mozilla developers use Linux than Windows. Partly because it takes about 20 minutes to build Firefox on a good Linux box vs an hour on Windows. And also of course because of free software.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Post new comment

CAPTCHA
We can't accept links (unless you obfuscate them). You also need to negotiate the following CAPTCHA...

Username:   Password:
Create Account | About TuxRadar