references.txt

SMAPS, MAPS, and MEMINFO Documentation
By Amy Zhang
Last edited: 7/11/12

/proc/<pid>/maps

A process's maps file contains currently mapped contiguous virtual memory
regions and their access permissions (Linux Documentation).
It is the memory map showing which addresses currently visible to that process
are mapped to which regions in RAM or to files (Wikipedia).

format = [address permissions offset device inode pathname]
- address: beginning and end addresses of the chunk used for the mapping in the
           process's address space
- permissions: 4 spots following the format "rwx[p/s]"
	- r: read
	- w: write
	- x: executable
	- p: private, unshared region (copy on write)
	- s: shared region
	- -: one of these properties is disabled

NOTE: the following only apply if the region was mapped from a file, otherwise
these fields appear as 0s
- offset: the offset of where the mapping starts in the file it was mapped from
- device: major:minor (hex) device numbers where the file lives
- inode (index number): number to identify where the inode is located in the
                        file system
	  - mappings with inode number = 0 do not have an inode associated with the
      memory region (often the case for BSS - uninitialized data;
      these mappings are anonymous, meaning they aren't associated with a disk
      file (Stack Overflow))
- pathname: name of the file the region was mapped from
	- If the mapping is not associated with a file:
		- [heap] = the heap of the program
		- [stack] = the stack of the main process
		- [vdso] = the "virtual dynamic shared object", kernel system call handler
    - other [TODO: there might be others? Find them]
	  or if empty, the mapping is anonymous. (Documentation/filesystems/proc.txt)

/proc/<pid>/smaps

Smaps reports the various types of memory use for each page mapping in a given
process. It's designed to be a precise snapshot of a moment. Listed below are
the categories of interest and what (we think) they represent.

According to Chris's Wiki smaps does the following:
For each VMA mapping that gets listed in smaps, the kernel walks all of the PTEs
associated with it and looks at all of the known pages. Each PTE is then counted
up.

- page mapping: same result as if you just typed /proc/<pid>/maps

NOTE: all memory is reported by number of pages and is per mapping (not for the
entire process - ideally that is one of the outputs our script would provide)

- Size: how much address space the mapping covers - if you added up the sizes
        for each mapping, you would get the entire size of the process's
        address space.
- Rss (resident set size): private + all shared (amount of mapping currently
                           resident in RAM) - the full PTE size (Chris's Wiki)
- Pss (proportional): private + shared memory/number of processes using it;
                      this is the proportional amount of the RSS

- Shared_Clean: 100% of shared_clean memory
- Shared_Dirty: 100% of shared_dirty memory

- Private_Clean: clean pages only used by the given process
- Private_Dirty: dirty pages only used by the given process

- Referenced: pages that have been used recently (referenced or accessed pages)

- Anonymous: amount of memory that does not belong to any file (even mappings
             mapped from a file can contain anonymous pages), these are usually
             mappings created by an allocator program like malloc.

- Swap: to our knowledge, ChromeOS doesn't currently support swap, but it
        represents how much would-be-anonymous memory is also used out on swap

- KernelPageSize: 4kB for this system
- MMUPageSize (memory management unit pagesize): also often 4kB

- Locked: if the mapping has been locked into memory via mlock()* then 'locked'
          reports the same value as PSS, otherwise reports 0
*from man pages: mlock() and mlockall() respectively lock part or all of the
calling process's virtual address space into RAM, preventing that memory from
being paged to the swap area - what does this mean for us if ChromeOS doesn't
implement swap?


/proc/meminfo (Source: CentOS)

Gives general memory information about the system using the following categories
(measured in kBs):

- MemTotal: total amount of physical RAM, in kilobytes
- MemFree: amount of physical RAM left unused by the system
           (sum of lowfree+highfree)

- Buffers: amount of physical RAM used for file buffers

- Cached: amount of physical RAM used as cache memory
- SwapCached: amount of swap used as cache memory

- Active: total amount of buffer or page cache memory in active use. This is
          memory that is either currently in use or has been recently used and
          is usually not reclaimed for other purposes.
- Inactive: total amount of buffer or page cache memory that is free and
            available. This is memory that has not been recently used and can be
            reclaimed for other purposes.

- HighTotal: total amount of memory not directly mapped into kernel space
             (can vary based on the type of kernel used)
- HighFree: free memory not directly mapped into kernel space
- LowTotal: total amount of memory directly mapped into kernel space
            (can vary based on type of kernel used)
- LowFree: free memory directly mapped into kernel space.

- SwapTotal: total amount of swap available
- SwapFree: amount of swap not in use

- Dirty: amount of memory waiting to be written back to the disk
- Writeback: amount of memory actively being written back to the disk

- Mapped: amount of memory that has been used to map devices, files, or
          libraries using the mmap command

- Slab: memory used by the kernel to cache data structures for its own use

- Committed_AS: amount of memory estimated to complete the workload. This value
                represents the worst case scenario value, and also includes swap
                memory. It's an estimate of how much RAM you need to avoid an
                OOM occurrence for this workload.

- PageTables: amount of memory dedicated to the lowest page table level

- VmallocTotal: amount of total allocated virtual address space
- VmallocUsed — amount of used virtual address space
- VmallocChunk — largest contiguous block of available virtual address space

ADDITIONAL INFORMATION
If we compare the values we obtain using the script to those in about:memory in
Chrome, they are a little bit different. We think this is because Chrome uses
/proc/<pid>/stat to gather memory information while we're using smaps. Also,
Firefox's about:memory has an interesting representation of their usage - could
take a look at this to get some ideas.

Some other things we want to look at: compare RSS with private RSS and what the
RSS is per mapping. VMsize is another commonly reported entity.
It "counts the total address space used by a process. However, this is not
accurate because it counts pages that are not mapped in to memory" (Bmaurer).

Keep in mind that processes with dirty memory will be more difficult to dispose
of since we would have to perform a write back to save the data that has been
written. Perhaps this is something we want to consider for our process
terminating heuristic.

SOURCES
Linux Documentation (http://kernel.org/doc/Documentation/filesystems/proc.txt)
Bmaurer blog: http://bmaurer.blogspot.com/2006/03/memory-usage-with-smaps.html
Chris's Wiki: http://utcc.utoronto.ca/~cks/space/blog/linux/SmapsFields
Stack Overflow:
http://stackoverflow.com/questions/9922928/what-does-pss-mean-in-proc-pid-smaps
CentOS: http://www.centos.org/docs/5/html/5.2/Deployment_Guide/s2-proc-meminfo.html
Man pages (maps)
Wikipedia