The kernel outputs a wide range of messages to its log file. These messages can be seen by looking at the system log file (usually located in /var/log/messages), or by running the dmesg command.

Sometimes it is useful to see exactly when those messages were created. dmesg , however, does not put any timestamps on the events it shows, and the time resolution of /var/log/messages is only to the nearest second. You can configure the kernel to assign each message a timestamp that is accurate down to the smallest measurable kernel time value (usually in the microsecond range.)

To enable timestamp options on kernel messages:

Kernel hacking
    [*] Show timing information on printks

The SysRq key on the keyboard can be used to control the kernel in a number of different ways while the kernel is running, or after it has crashed.

To enable this option:

Kernel hacking
    [*] Magic SysRq key

For a full description of the different actions that can be triggered by this option, please see the file Documentation/sysrq.txt in the kernel source tree.

A RAM-based filesystem can be used to output a lot of different debugging information. This filesystem is called debugfs and can be enabled by:

Kernel hacking
    [*] Debug filesystem

After you enable this option and boot the rebuilt kernel, it creates the directory /sys/kernel/debug as a location for the user to mount the debugfs filesystem. Do this manually by:

$ 
mount -t debugfs none /sys/kernel/debug

or have the filesystem mounted automatically at boot time by adding the following line to the /etc/fstab file:

debugfs /sys/kernel/debug debugfs 0 0

After you mount debugfs, a large number of different directories and files will turn up in the /sys/kernel/debug/ directory. These are all virtual and dynamically generated by the kernel, like the files in procfs or sysfs. The files can be used to help debug different kernel subsystems, or just perused to see what is happening to the system as it runs.

Here are a range of other good kernel configuration options that you might wish to enable if you want to help kernel developers debug different problems, or just learn more about how the kernel works by looking at the messages that these options print out. Note that if you enable almost any of these options, the kernel will slow down a small amount, so if you notice any decrease in performance, you might wish to disable the options.

Kernel hacking
    [*] Kernel debugging
    [*]   Detect Soft Lockups
    [ ]   Collect scheduler statistics
    [*]   Debug slab memory allocations
    [*]     Memory leak debugging
    [*]   Mutex debugging, deadlock detection
    [*]   Spinlock debugging
    [*]   Sleep-inside-spinlock checking
    [ ]   kobject debugging
    [ ]   Highmem debugging
    [ ]   Compile the kernel with debug info