A Little Hick-up Getting Into gdb
I had a little fun in actually getting debugging to work under a VMware appliance. As soon as I did a gdb <path>/soffice.bin that would be the end of that the VM would slowly die, the host OS would die and finally the client OS would panic. The root cause of this is that gdb needs a lot of memory to load the soffice.bin symbol tables. I ended up needing to allocate 640 Kbytes to the VM to run stably. I could then start to trace the soffice.bin and my first task was to hunt down a nasty memory leak (which is the subject of a separate post). Both running the VM and needing to watch soffice.bin memory usage got me into task manager and interpreting its columns.
Now I though that I understood how VM architectures ran and since the Windows one was architected by same guys who did the Digital VMS one, I thought that I should understand this pretty well. However, I was wrong — at least as far as interpreting the stats goes. Time for Googling the great knowledge base, which threw up this article An introductory guide to Windows Memory Management. It turns out that Task Manager does not label its columns at all well. If you use perfmon specifically to look at the process counters for a given process, there is a list of counters that relate to memory management, and some of these have equivalent columns that you can display on taskman. Thee are:
| Perfmon | Taskman | Linux ps e | Description |
| Virtual Bytes | – | RSS/%MEM | This is the total memory address space used by the process. Now this contains a mix of memory types. Must code and read-only data is directly mapped from disk, so the relevant pages in the disk file are the master copy, though clearly if the processor is to reference and use locations it must fault these pages into its working set in memory. Read only pages can be discarded when necessary because the page can always be reloaded from the original disk file when required. Pages that have been modified are a different matter in that in most cases any original mapped file is read-only and updated pages must then be written somewhere else. This somewhere is the PageFile. |
| Private Bytes | VM Size | DRS | And this is where the Private Bytes size comes in: this is the total read-write memory that is private to this process. |
| Pool Bytes | Memory that read-write that may be shared between process is maintained in Pool memory, and this is split into two categories: that which can be paged off to the PageFile and that which must be retained in working memory. In practice these two are tiny compare to Private Bytes, for example. | ||
| Working Set | Mem Usage | RSS | The working set is the subset of the Virtual Bytes which is currently in memory. Note that this can be a mix of Private, Pool and read-only memory. Also note that in general the actual memory needed is a lot less than the sum of the working sets. This is because the majority of the code in an process is shared (DLLs etc) and used my many processes running one the machine. |
The Virtual Memory system uses a variety of algorithms to tune the memory mix and set working sets, but broadly when a process needs a page in memory that it doesn’t have, a page fault occurs and the memory is loaded from disk. The more a process is faulting, then the more working set is allocated to it. At the same time working set is slowly pinched back from all processes on a least recently used basis so that processes can have their inactive pages discarded (if readonly) or rolled out to the PageFile if read-write. In doing this the Operating system seeks to make best use of memory and minimise the total page fault rate in doing so.
Hence a process such as VMware needs a lot of Virtual Bytes (basically the size of the VM being emulated plus a relatively small overhead for the VM monitor). However much of this will be read-only bytes: guest mapped from the sectors of the virtual hard disk which is itself mapped onto a large disk file on the host. Hence the Private Bytes or VM size will be relatively small. If the host is short of memory then is will tend to compress the memory used by the VM faulting out a lot of the memory in the virtual machine which is representing the clients physical memory. In this situation if you change the pattern of usage of the guest VM you can then get double faulting occurring — that is the guest OS will be faulting in stuff into what it thinks of as its physical memory but this is in turn causing VMware to page fault in the host environment.
On the other extreme you can get quiescent processes which have a lot of read-write memory, but their working sets have been trimmed right back. An example here on my machine is SQLserver, which is started by default but is then largely dormant hence it has a 32Mbyte VM size but only a 5Mbyte Mem Usage (working set. On my machine, the biggest hog is any VMware client that I open, followed by Outlook, Winword, then Soffice.