Real-World Firefox 3 Memory Usage Leads the Field

An anonymous reader writes "The author developed a program to snapshot memory usage per process every 3 seconds on Windows. Using this he recorded 3 hours of memory usage for five different browsers under real-world usage scenarios: Safari 3.1, Firefox 3, Flock 1.2 (a browser based on Firefox 2), Opera 9.5, and Internet Explorer 8. A million data points indicate that Firefox 3 has a surprising advantage over the other browsers tested. These are real-world tests and not contrived benchmarks."

Wonder what Firefox 2 looked like ...

Interesting test - pretty amazing how FF3 basically flatlines at around 120 MBytes for over 2 hours of usage ... would have been interesting if the same methodology could be used with FF2 to see how much of an improvement FF3 is over that and how well the leaks were fixed. [komar.org]

Re:Wonder what Firefox 2 looked like ...

Cue Iraqi spokesman

There are no memory leaks. All memory usage is as we intend it to be. Any reports of leaks are lies by people who do not understand our page caching system. The infidels will never take Baghdad.

Re:Wonder what Firefox 2 looked like ...

The article states FF3 is an improvement over FF2, without offering data points for FF2. However, it also mentions Flock is based on FF2, so I'm guessing they've assumed the Flock data is representative for FF2.

What cap?

Setting a cap on memory usage isn't a good solution, IMHO -- using well-designed memory handling that proactively frees memory seems to me to be a far better solution than a cap and garbage collection model.

I haven't seen any mention of a cap, even if that's a natural conclusion based on the flat line that one can observe in the graphs.

If you check this [pavlov.net] fairly lengthy explanation of how memory usage was improved in FF3 you'll see that it is mostly attributed to reduced fragmentation and leaks, and smarting caching, just as you are advocating.

db

Re:Wonder what Firefox 2 looked like ...

That won't do those without 120 MB of memory to spare much good. Setting a cap on memory usage isn't a good solution, IMHO -- using well-designed memory handling that proactively frees memory seems to me to be a far better solution than a cap and garbage collection model.

(-1, Uninformed)

Firefox has no global cap on memory. It will dynamically configure it's caches (to some extent) based on the available RAM. It would be a stupid design to leave lots of RAM free, and reload stuff over the net. It also proactively frees memory, in most cases, although it sometimes delays a little, because it knows that you might turn around and reuse all of that memory you just stopped using. The GC is just for JavaScript (required by design) and for DOM nodes which end up being circularly referenced (which is unavoidable).

Finally, 120MB is not a lot of RAM. Notice that the other browsers are using similar amounts of RAM.

Regards
-Jeremy

What time of day did he do his tests?

I find that for certain hours of the day (namely in the evening) My memory usage skyrockets. It probably has to do with the increased number of images I am loading :D

But what memory metric was taken?

IIRC the memory displayed in process manager isn't necessarily the memory requested/used by the program, but merely what Windows has allocated, partially based on the applications requirements and partially based on what Windows _thinks_ the program needs.

As such there's room for applications to look like they're using more memory than they are which can lead to misleading stats. If this test has only taken into account the memory windows has allocated it doesn't necessarily act as a measure of how efficient the program is at least, just how good it is at playing Window's memory management system.

Re:But what memory metric was taken?

Ok, I see your point. Let's pretend Browser X is using dirty tricks with Windows's memory management system to shrink down how much memory is allocated to it. Browser Y is not doing that and appears to be less efficient.

Well and good, but it's irrelevant. The remains that Browser X is taking less memory from Windows's pool of resources. It doesn't matter how Browser X is doing it or how efficient Browser X being with the memory internally, it is a solid truth that Browser X is using occupying fewer system resources than Browser Y.

It's really a moot point, because it's unlikely that the developers of Browser X knew any "cheats" that would let them use substantially less memory than every other browser out there.

HTTP 503'd (aka /.ed)

Service Not Available.

At the time of posting this, there were like, 10 comments in the thread. Assuming that only 10% of all /.ers RTFA, that means that the site can support only 1 simultaneous user.

Bah.

If they didn't compare with Dillo/lynx, it's meaningless. Also, already slashdotted.

Terrible reference

Between:

These aren't stress tests, and I probably never went over 4 windows in each browser, with at most 3 tabs in each window.(Emphasis mine)
and
.the individual numbers should not be compared to each other...

...how is this supposed to be taken seriously? "Contrived benchmarks" at least provide consistent and reliable results. They may not provide a completely accurate picture of real world browsing, but it's a hell of a lot better than this anecdotal "test".

Re:Terrible reference

This was my first question too. Real-world testing is all well and good, but how controlled was it? What assurances do we have that his results really paint the picture he claims and not something else?

Note: The site is down so I haven't read the article yet. I'm guessing it fails to address this concern?

Re:you do know what "contrived" means right?

Tests need to be repeatable, and when they are repeated they need to provide consistent results. If you can't provide consistent results on each subsequent test, you need to provide solid rational on why the results were outside your margin of error. Saying "Oh, I might have had one or two youtube videos up at the time" is not solid rational.

Scripts that visit the exact same pages, for the exact same time, do the exact same things across all browsers provide consistent, quantifiable results. Since everyone's browsing behavior is different no script will ever provide "accurate" results for real world usage. But then again, those scripts could be closer to my real world usage than this guys anecdotal test. Get it?

If slashdotted

Final memory usage in MB
Safari 636.9
Firefox 3 111.8
Flock (Firefox 2) 191.9
Opera 9.5 190.6
Internet Explorer 194.4

How did they measure memory consumption?

As the server is (already!?) down, I didn't yet have a chance to RTFA. So perhaps it is in the article somewhere, but I couldn't help wondering: how did they actually measure memory usage?

I'm asking because, these days, that pretty much amounts to rocket science.

Different operating systems report memory usage differently, even between different versions of the same OS (yes, I'm looking at you, Vista vs. XP). If they used "top" or its equivalent, it matters a lot whether they looked at real usage, virtual memory size (can be huge but that doesn't say anything) or what-have-you. Some OS's cheat quite a bit in what memory is reported as being "free" or "available", as well. Then we get to questions like "does it include the size of shared libraries", if not, is that fair if the libraries are really only used by that one application? Etc. etc.

So I'm not saying memory using doesn't matter (it very much does), it's just hard to measure it exactly. And, any attempts at doing so, should be documented precisely.

Re:How did they measure memory consumption?

The OS was Vista, and the program was written in .Net to use the function PrivateMemorySize64 [microsoft.com]. MSDN says it returns "the amount of memory which cannot be shared with other processes". It also says it's the same as the "Private Bytes" value in taskmon. Probably it means that it's the amount of memory the process received from mallocs (or rather GlobalAllocs/LocalAllocs/HeapAllocs), and which can't be assigned to some other process.

It's worth noticing that the guy bothered with a GUI and an interactive filtering option for such a simple program. I wonder whether he ever heard of CLI, because it looks like a perfect fit for this kind of program.

By the way, why not post CoralCDN links (append .nyud.net to hostname) instead of direct links when the site in question is small and likely to be Slashdotted?

Memory?...what about speed?

While it's admirable that it's the leanest of the bunch, if I have 2GB of memory and over half of that is unused at the moment, do I really care if my browser uses 25MB instead of 40MB? I would think the speed with which the browser (and subsequent windows) opened, as well as how quickly it loaded plug-ins and other embedded media, would be of more importance.

Definitely important

In the case of Firefox, memory usage has ranged from 25MB to $MEMORY_AVAILABLE. Which sucks no matter how much you have.

This doesn't mesh with my experience

My wife and I share a computer. She uses mostly uses Firefox, I mostly use Opera. This is on a 64-bit Ubuntu Hardy.

I have noticed no difference in her memory usage since we upgraded to FF3. I used to regularly have to kill her browser every once in a while (maybe once or twice a week) because it was eating up all the RAM. Since we upgraded to FF3, I can see no difference in memory usage.

For example, right now FF is using 300MB resident, Opera is using 100MB. Flashblock is installed on both browsers. Granted, that's not a terribly good test considering we've been browsing to different sites, but I've found that those numbers are fairly stable. FF usually levels off in the 300-500MB range, and Opera in the 100-150 range.

YMMV.

Re:This doesn't mesh with my experience

64-bit firefox eats a LOT more of memory. The windows versions are 32-bit only.

Re:Yah, but how reliable?

I haven't had Firefox 3 crash for me yet (although I've only been using it since Download Day). I have noticed that it no longer gets hung up processing javascript the way Firefox 2 often did.

Re:Yah, but how reliable?

This is most likely related to the Flash plugin. The second suspect would be the Java plugin. For me Firefox never crashed on a website without Flash and Java, but I had a few crashes due to Flash bugs.

Re:Lools IIS can't hold its own

"The term "slashdotted" has become ubiquitous with smashing a webserver due to high traffic.
Most webservers are *nix based (though admittedly IIS is gaining ground)."

So what you are trying to say is that Slashdotting is becoming more and more popular?

Re:...contrived benchmarks.

<keynote style="Steve">
Safari on OS XI is going to be 400% faster. It's going to look 700% rounder, and integrate seamlessly with your ego. It will make you 1500% more smug, no matter how smug you were before.

Firefox ? Not smug.
IE8 ? *chuckles* next slide.
Opera ? They still have square corners, what does that tell you about their priorities ?

It's so awesome we had to give it a new name: Snow Safari.
(*applause*)
</keynote>

Re:Careful...

' They're trying to charge money ofr something everyone else gives away for free, and with the latest browsers they're running out of legitimate advantages to boast."

Please change that template, it is 2008 already.