Ars Analysis Calls Windows 7 Memory Usage Claims "Scaremongering" 334
Posted
by
timothy
from the nothing-but-dihydrogen-monoxide-to-drink dept.
from the nothing-but-dihydrogen-monoxide-to-drink dept.
Via newsycombinator comes a reaction at Ars Technica to the recently reported claims of excessive memory use on machines running Windows 7. From the article: "I installed the XPnet performance monitoring tool and waited for it to upload my data to see what it might be complaining about. The cause of the problem was immediately apparent. It's no secret that Windows 7, just like Windows Vista before it, includes aggressive disk caching. The SuperFetch technology causes Windows to preload certain data if the OS detects that it is used regularly, even if there is no specific need for it at any given moment. Though SuperFetch is a little less aggressive in Windows 7, it will still use a substantial amount of memory—but with an important proviso. The OS will only use memory for cache when there is no other demand for that memory."
So (Score:5, Informative)
Though SuperFetch is a little less aggressive in Windows 7, it will still use a substantial amount of memory—but with an important proviso. The OS will only use memory for cache when there is no other demand for that memory.
I really wonder when people will get this. In the earlier thread I saw people commenting that Windows 95 didn't need so much memory and so on..
To state it again. This is not RAM memory you need, use or have purpose for. IF you do need it, it is zeroed-out and free'd to application in like 30ms (one frame in usual FPS games).
If you have fast memory, do use it to it's full extend.
Re:So (Score:1, Informative)
30ms? (Score:3, Informative)
Really? 30ms. Shit that's slow.
This IS RAM we're talking about here, right? y'know nanosecond stuff, 10^-9 not 10^-3 seconds.
Re:So (Score:3, Informative)
#1) Linux _doesn't_ have all the best and greatest technology built in.
#2) It even does have some really crappy technology in between.
#3) guess what the "-/+ buffers/cache" line in the output of "free" means
#4) guess what the "buffers" and "cached" columns of "free" means
#5) guess what prefetch/preload is.
Sheez... anyone gets access to a keyboard these days
Re:So (Score:2, Informative)
People aren't taking the time to learn the meaning of the 'available' memory stat in the Task Manager.
Based on my experience, it's usually very close to the total you get when adding 'free' memory and 'cached' memory together.
'Available' in this case means that, as parent suggests, Windows will free it for use as soon as it's needed.
Re:If only the cache were actually -good- (Score:3, Informative)
As I understand it, it doesn't cache data, it caches applications (and I think also fonts and other often-used things). So startup time for your web browser, e-mail client, IM client, and any other applications you use often will be much faster. For example, Google Chrome loads almost instantaneously on my system, from a cold start. It won't keep a memory cache of things that applications do, and hence it won't speed up compilation, rendering, etc.
For that matter, 2GB of RAM isn't a whole lot to cache with when you think about it. The sweet spot for Vista/Win7 is really around 4GB or higher for SuperFetch to really shine.
Re:he's a native speaker (Score:2, Informative)
This is completely offtopic, but you seem to come with this message in every thread so I answer you one time here - I am from sweden, not uk.
Been using 7 awhile now (Score:3, Informative)
Seems to use 1.3 gig no matter what I do. It's got 4 gig total. Boots in less than 2 minutes. It makes mistakes a lot faster than the old machine, which was just shy of a brick when I switched.
Re:So (Score:5, Informative)
Yep, same for Linux. My Linux boxes use ALL the memory available even if I do not run many applications on it. The left over memory SHOULD be used as buffers/cache. If Windows 7 seems to use more memory from a newbie point of view, it might be because it does things like it should better than previous versions. I can't tell for sure since I have never tried win 7.
See this 4 GB Linux machine below, it only has ~49 MB of "absolutely free" memory and uses ~449 MB of swap.
In realty, it has ~2842 MB of "available memory" since it uses ~2792 MB of buffer/cache.
Using buffer/cache makes the system order of magnitude faster. If programs need that memory, the OS will give to them and use less buffer/cache.
free
total used free shared buffers cached
Mem: 4133252 4083380 49872 0 26852 2766248
-/+ buffers/cache: 1290280 2842972
Swap: 1999800 449244 1550556
Re:30ms? (Score:0, Informative)
Furthermore, RAM is not used when it is allocated. When app allocated memory, it only allocates address space. You could easily have 2:1 or 5:1 ratio of allocated memory space to used RAM. RAM is only used when application starts to use that memory.
In addition, there is READ-ONLY memory (non-writable, executable) that is shared between applications.
This is another reason why having more than 2G of RAM on a 32-bit OS is dumb. OS runs out of address space and your 4G ram can easily be cut down to 2G usable. For heavy multi-tasking applications, where you have lots of specialized hardware (ie. graphic cards, DMA accessible RAID, etc) I wouldn't be surprised if a 32-bit OS would have problems managing more than 2G ram.
Re:he's a native speaker (Score:2, Informative)
Re:30ms? (Score:2, Informative)
This IS RAM we're talking about here, right? y'know nanosecond stuff, 10^-9 not 10^-3 seconds.
Yes, and nanoseconds (10^-9) multiplied by the number of memory locations to clear (10^6 when you're talking multi-MB chunks of memory) gets us right back in the millisecond (10^-3) range. Which is just a blink of the eye for us humans, btw.
Linux does that (Score:5, Informative)
Linux uses available memory for cache, and rather aggressively. All available memory can be filled with cached file blocks. This happens routinely on systems which have big randomly-accessed files open, like databases.
There's nothing wrong with this, except that, once in a while, Linux hits a race condition in prune_one_dentry, causing an "oops" crash, when there's an unblockable need for a memory page and something is locking the file block cache.
This is one of the Great Unsolved Mysteries of Linux. Linus wrote about it in 2001 [indiana.edu] ("I'll try to think about it some more, but I'd love to have more reports to go on to try to find a pattern.. "). As of 2009, this area is still giving trouble. [google.com] The locking in this area is very complex. [lwn.net]
Re:If only the cache were actually -good- (Score:3, Informative)
Programs and compilers have nothing to do with disk read/write caches, this is handled at the OS level.
Programs ask to read/write from/to disk and they have no idea where the OS get the data from.
You and the GP might be confusing CPU cache that has its own dedicated memory and disk cache that uses the OS RAM.
Re:Tits on a bull (Score:2, Informative)
At boot, Windows is doing other things not prefetching. Turn superfetch on and monitor your active processes at boot, you'll see that the superfetch PID is sittng all the way at the bottom of everything. Its IO is negligible as is its CPU usage until AFTER everything else has loaded and your system has gone back to twiddling its bits for a few hundred thousand processor cycles.
Re:30ms? (Score:5, Informative)
You might want to grab a copy of Process Explorer sometime, and look at the stats it reports. you'll notice that windows actually spends idle time pre-zeroing ram, so that this is already done, in more than enough amounts. If your system is slammed, i could see having to pre-zero the pages, just before use, however, but it's not like it's not something that couldn't be done while waiting for other I/O operations to complete (since your system is slammed anyway :) )
My laptop currently has 2.8 million pages zeroed atm (it has 8gb, and I don't have much running right now, so there's not a lot to cache.)
that was supposition (Score:5, Informative)
The only data in the article you refer to was captured data from XPnet that said that >90% of RAM was in use in Windows 7 machines. There was no data saying it led to swapping, that was supposition.
http://www.computerworld.com/s/article/9158258/Most_Windows_7_PCs_max_out_memory [computerworld.com]
Re:This is one of those great mysteries of life (Score:4, Informative)
if everyone is so afraid of their computer memory being used to the fullest, why do these people install so much of it?
Most users remember back to at least the 90s. You had to install enough ram to do what you needed (load the OS and program(s) - WinNT had the audacity to require 8MB to run well). There was no caching of any useful sort, so your free memory was really a measure of how many programs you could load. Programs, like Photoshop, added scratch files to overcome the physical RAM limits, but at a horrible performance penalty should have to actually use it. "Free RAM" became synonymous with "how many things you could do or open simultaneously."
All modern operating systems have moved on, but people haven't been educated about this. They remember how bad it was when they ran out of memory, and panic when the OS reports it's almost full. Honestly, it would be far better if MS would have reported the cached memory differently. I don't really care how much memory is used as superfetch cache most of the time - I'm more concerned with the total active usage. My netbook "only" has 2GB, but I do some 24/96 audio recording with it, and will occasionally work with photoshop images, so I am concerned if I have less than 500-600MB free when I open a session as I'm likely to exceed the physical RAM. I can read the data, so it's not a big deal, but others freak out about it.
Re:30ms? (Score:1, Informative)
Re:So (Score:5, Informative)
You're as bad as the guys who wrote that article in the first place. If you don't know how Windows works, please don't talk about it.
Hell, its not just windows. All operating systems do this.. and to be quite frank, programmers of all kinds should have cache techniques well understood. So the GP is neither a windows guru nor a decent programmer. The odds are very good that hes just an I-use-software geek, rather than someone who knows anything about computers.
Re:So (Score:5, Informative)
Source: Wikipedia [wikipedia.org]
Their work, says Horvitz, was able to predict which applications users would open by time of day and also by day of the week.
Source: InfoWeek [informationweek.com]
Re:So (Score:5, Informative)
You can press F1 while in task manager and then search for a particular metric, e.g. "available memory". This produces results that seem moderately useful, for example:
Under Physical Memory (MB), Total is the amount of RAM installed on your computer, listed in megabytes (MB). Cached refers to the amount of physical memory used recently for system resources. Available is the amount of memory that's immediately available for use by processes, drivers, or the operating system. Free is the amount of memory that is currently unused or doesn't contain useful information (unlike cached files, which do contain useful information).
For more details about particular counters you can check the Windows Internals book, or Memory Performance Information [microsoft.com] on MSDN. Also, many counters in task manager have similar or identical perfmon counters, and perfmon has its own help (IIRC there's a "show description" option in the counter selection dialog)
.
Re:Windows 7 and Vista Lie about memory usage (Score:5, Informative)
Not necessarily. Many programs commit large chunks of memory in case they need it later but only use a small portion initially. This simplifies program logic because you don't have to free and reallocate the buffer when you need more space, deal with potential reallocation failures etc. Or a program might want to specify a larger-than-default stack commit size to make sure it doesn't hit a stack overflow if it tries to extend the stack while the system is temporarily out of commit (most services and other system critical processes do that). Or it might map a copy-on-write view of a file, in which case commit is charged for the entire view but no extra physical memory is used until the program actually writes to the pages. Etc etc... The end result of this is that you can't really say anything conclusive about physical memory usage by looking at commit charge
Commit charge is a virtual memory metric. It's great for detecting memory leaks and deciding how big your pagefile needs to be, but not so great for understanding physical memory usage. Often it might seem like there is a correlation between commit charge and physical memory, but you can also find systems that are very low on available RAM yet have plenty of available commit, and vice versa.
Task manager now shows used physical memory (defined as Total - Available). Available memory is the most straightforward way to understand whether your system needs more memory or not, and this is why in Vista/Win7 it was chosen as the main indicator of "memory usage".
Idle memory is worthless. (Score:3, Informative)
Ars (Score:2, Informative)
While catering to the Apple fan-base a little too much for my tastes -- hits it right on.
It is scaremongering.
.
Cache is there for a reason -- idle RAM is useless RAM. Now, do I think that Windows allocation mechanism for idle RAM is right on? Of course not. It could use some tuning. It could have used some tuning since XP -- but is idle RAM stupid? Yes.
Anything that can update the pipeline between process/thread interaction with data before it has to go to disk -- is preferred.
However, Windows needs to be smart about what it caches and keeps in the pre-fetch. Frequently used and accessed rather than conditional stale.
That's the point.
Re:So (Score:5, Informative)
The function you're looking for is VirtualLock [microsoft.com]. You may also look into increasing the process's minimum working set with SetProcessWorkingSetSize [microsoft.com]. This requires SeIncreaseBasePriorityPrivilege.
A process that is scanning through a file is supposed to use the FILE_FLAG_SEQUENTIAL_SCAN hint so that the cached pages are recycled first, but that doesn't always happen. It also doesn't help that csrss will ask the kernel to minimize a process's working set when its main window is minimized.
Re:Tits on a bull (Score:3, Informative)
Sort of.
The issue with ext4 had to do with dirty cache not being written to disk in a timely manner. This meant that you could have disk writes you thought had already happened and certain actions could get out of sync because of they way they'd been written.
The cache in this case is not dirty(in fact it's read only) so there's no risk of losing data (since most of the time the data on the disk and the data in memory are identical and when they're not the disk is right).
Re:So (Score:5, Informative)
Starting with Vista, working sets of GUI processes are no longer emptied when the main window is minimized.
For the standby cache recycle problem, Superfetch can help a lot. First of all, it can detect when apps do things like read lots of files sequentially without using FILE_FLAG_SEQUENTIAL_SCAN (or when they do this through a mapped view) and deprioritize these pages so they don't affect normal standby memory. And if useful pages still end up being recycled (e.g. because some app temporarily consumed lots of memory), Superfetch can re-populate them from disk later.
Re:PAE? (Score:1, Informative)
Except PAE is supported on all Pentium Pro and later Intel CPUs.
I don't know where you get the idiotic idea that it only works on 64-bit hardware.
Re:It's a matter of definitions (Score:2, Informative)
Except it simply doesn't do that as you can easily demonstrate with test case programs. Memory used for cache is used but not reserved, so if a program needs it, it will simply free some of the cache and give it to the program. No swapping required. Swapping programs in order to maintain the cache would be idiotic, but Windows doesn't do this.