Out of Memory (OOM) refers to a computing state where all available memory, including swap space, has been allocated. When this situation occurs, it will cause the system to panic and stop functioning as expected. There is a switch that controls OOM behavior in /proc/sys/vm/panic_on_oom
. When set to 1
the kernel will panic on OOM. A setting of 0
instructs the kernel to call a function named oom_killer
on an OOM. Usually, oom_killer
can kill rogue processes and the system will survive.
The easiest way to change this is to echo the new value to /proc/sys/vm/panic_on_oom
.
# cat /proc/sys/vm/panic_on_oom
1
# echo 0 > /proc/sys/vm/panic_on_oom
# cat /proc/sys/vm/panic_on_oom
0
It is also possible to prioritize which processes get killed by adjusting the oom_killer
score. In /proc/PID
/
there are two tools labelled oom_adj
and oom_score
. Valid scores for oom_adj
are in the range -16 to +15. To see the current oom_killer
score, view the oom_score
for the process. oom_killer
will kill processes with the highest scores first.
This example adjusts the oom_score of a process with a PID
of 12465 to make it less likely that oom_killer
will kill it.
# cat /proc/12465/oom_score
79872
# echo -5 > /proc/12465/oom_adj
# cat /proc/12465/oom_score
78
There is also a special value of -17, which disables oom_killer
for that process. In the example below, oom_score
returns a value of 0, indicating that this process would not be killed.
# cat /proc/12465/oom_score
78
# echo -17 > /proc/12465/oom_adj
# cat /proc/12465/oom_score
0
A function called badness()
is used to determine the actual score for each process. This is done by adding up 'points' for each examined process. The process scoring is done in the following way:
-
The basis of each process's score is its memory size.
-
The memory size of any of the process's children (not including a kernel thread) is also added to the score
-
The process's score is increased for 'niced' processes and decreased for long running processes.
-
Processes with the CAP_SYS_ADMIN
and CAP_SYS_RAWIO
capabilities have their scores reduced.
-
The final score is then bitshifted by the value saved in the oom_adj
file.
Thus, a process with the highest oom_score
value will most probably be a non-priviliged, recently started process that, along with its children, uses a large amount of memory, has been 'niced', and handles no raw I/O.