Producing Reports With rcapstat
Use rcapstat to report resource capping statistics. Monitoring Resource Utilization With rcapstat explains how to use the
rcapstat command to generate reports. That section also describes the column headings in
the report. The rcapstat(1) man page also contains this information.
The following subsections use examples to illustrate how to produce reports for specific
purposes.
Reporting Cap and Project Information
In this example, caps are defined for two projects associated with two users.
user1 has a cap of 50 megabytes, and user2 has a cap of
10 megabytes.
The following command produces five reports at 5-second sampling intervals.
user1machine% rcapstat 5 5
id project nproc vm rss cap at avgat pg avgpg
112270 user1 24 123M 35M 50M 50M 0K 3312K 0K
78194 user2 1 2368K 1856K 10M 0K 0K 0K 0K
id project nproc vm rss cap at avgat pg avgpg
112270 user1 24 123M 35M 50M 0K 0K 0K 0K
78194 user2 1 2368K 1856K 10M 0K 0K 0K 0K
id project nproc vm rss cap at avgat pg avgpg
112270 user1 24 123M 35M 50M 0K 0K 0K 0K
78194 user2 1 2368K 1928K 10M 0K 0K 0K 0K
id project nproc vm rss cap at avgat pg avgpg
112270 user1 24 123M 35M 50M 0K 0K 0K 0K
78194 user2 1 2368K 1928K 10M 0K 0K 0K 0K
id project nproc vm rss cap at avgat pg avgpg
112270 user1 24 123M 35M 50M 0K 0K 0K 0K
78194 user2 1 2368K 1928K 10M 0K 0K 0K 0K The first three lines of output constitute the first report, which contains the
cap and project information for the two projects and paging statistics since rcapd
was started. The at and pg columns are a number greater than
zero for user1 and zero for user2, which indicates that at some time
in the daemon's history, user1 exceeded its cap but user2 did not.
The subsequent reports show no significant activity.
Monitoring the RSS of a Project
The following example uses project user1, which has an RSS in excess of
its RSS cap.
The following command produces five reports at 5-second sampling intervals.
user1machine% rcapstat 5 5
id project nproc vm rss cap at avgat pg avgpg
376565 user1 3 6249M 6144M 6144M 690M 220M 5528K 2764K
376565 user1 3 6249M 6144M 6144M 0M 131M 4912K 1637K
376565 user1 3 6249M 6171M 6144M 27M 147M 6048K 2016K
376565 user1 3 6249M 6146M 6144M 4872M 174M 4368K 1456K
376565 user1 3 6249M 6156M 6144M 12M 161M 3376K 1125K
The user1 project has three processes that are actively using physical memory. The
positive values in the pg column indicate that rcapd is consistently paging out
memory as it attempts to meet the cap by lowering the physical memory
utilization of the project's processes. However, rcapd does not succeed in keeping the RSS
below the cap value. This is indicated by the varying rss values
that do not show a corresponding decrease. As soon as memory is paged
out, the workload uses it again and the RSS count goes back up.
This means that all of the project's resident memory is being actively used
and the working set size (WSS) is greater than the cap. Thus, rcapd
is forced to page out some of the working set to meet the
cap. Under this condition, the system will continue to experience high page fault
rates, and associated I/O, until one of the following occurs:
In this situation, shortening the sample interval might reduce the discrepancy between the
RSS value and the cap value by causing rcapd to sample the workload
and enforce caps more frequently.
Note - A page fault occurs when either a new page must be created
or the system must copy in a page from a swap device.
Determining the Working Set Size of a Project
The following example is a continuation of the previous example, and it uses
the same project.
The previous example shows that the user1 project is using more physical memory
than its cap allows. This example shows how much memory the project workload
requires.
user1machine% rcapstat 5 5
id project nproc vm rss cap at avgat pg avgpg
376565 user1 3 6249M 6144M 6144M 690M 0K 689M 0K
376565 user1 3 6249M 6144M 6144M 0K 0K 0K 0K
376565 user1 3 6249M 6171M 6144M 27M 0K 27M 0K
376565 user1 3 6249M 6146M 6144M 4872K 0K 4816K 0K
376565 user1 3 6249M 6156M 6144M 12M 0K 12M 0K
376565 user1 3 6249M 6150M 6144M 5848K 0K 5816K 0K
376565 user1 3 6249M 6155M 6144M 11M 0K 11M 0K
376565 user1 3 6249M 6150M 10G 32K 0K 32K 0K
376565 user1 3 6249M 6214M 10G 0K 0K 0K 0K
376565 user1 3 6249M 6247M 10G 0K 0K 0K 0K
376565 user1 3 6249M 6247M 10G 0K 0K 0K 0K
376565 user1 3 6249M 6247M 10G 0K 0K 0K 0K
376565 user1 3 6249M 6247M 10G 0K 0K 0K 0K
376565 user1 3 6249M 6247M 10G 0K 0K 0K 0K
376565 user1 3 6249M 6247M 10G 0K 0K 0K 0KHalfway through the cycle, the cap on the user1 project was increased from
6 Gbytes to 10 Gbytes. This increase stops cap enforcement and allows the
resident set size to grow, limited only by other processes and the amount
of memory in the machine. The rss column might stabilize to reflect the
project working set size (WSS), 6247M in this example. This is the minimum
cap value that allows the project's processes to operate without continuously incurring page
faults.
While the cap on user1 is 6 Gbytes, in every 5–second sample interval
the RSS decreases and I/O increases as rcapd pages out some of the
workload's memory. Shortly after a page out completes, the workload, needing those pages,
pages them back in as it continues running. This cycle repeats until the
cap is raised to 10 Gbytes, approximately halfway through the example. The RSS
then stabilizes at 6.1 Gbytes. Since the workload's RSS is now below the
cap, no more paging occurs. The I/O associated with paging stops as well.
Thus, the project required 6.1 Gbytes to perform the work it was doing
at the time it was being observed.
Also see the vmstat(1M) and iostat(1M) man pages.
Reporting Memory Utilization and the Memory Cap Enforcement Threshold
You can use the -g option of rcapstat to report the following:
The -g option causes a memory utilization and cap enforcement line to be
printed at the end of the report for each interval.
# rcapstat -g
id project nproc vm rss cap at avgat pg avgpg
376565 rcap 0 0K 0K 10G 0K 0K 0K 0K
physical memory utilization: 55% cap enforcement threshold: 0%
id project nproc vm rss cap at avgat pg avgpg
376565 rcap 0 0K 0K 10G 0K 0K 0K 0K
physical memory utilization: 55% cap enforcement threshold: 0%
