By Kurt Seifried [email protected]
Introduction
No matter how good your security is you it can be compromised. That is why it
is so important to have various forms of attack detection present, so that when
an incident happens you are alerted to it as soon as possible (and not when you
start getting complaints from other sites).
Baselines
One major oversight made by a lot of people when securing their machines is
that they forget to create a baseline of the system, that is a profile of the
system, its usage of resources, and so on in normal operation. For example
something as simple as a "netstat -a -n > netstat-output" can give you a
reference to latter check against and see if any ports are open that should not
be. Memory usage and disk usage are also good things to keep an eye on. A sudden
surge in memory usage could result in the system being starved of resources.
Likewise for disk usage. It might be a user accident, a malicious user, or a
worm program that has compromised your system and is now scanning other systems.
Various tools exist to measure memory and disk usage: vmstat, free, df, du, all
of which are covered by their respective man pages.
At the very minimum make a full system backup, and regularly backup config
files and log files, this can also help you pinpoint when an intrusion occurred
(user account "rewt" was added before the April 4th backup, but isn't in the
March 20th backup). Once a system is compromised typically a "rootkit" is
installed, these consist of trojaned binaries, and are near impossible to remove
safely, you are better of formatting the disk and starting from scratch. There
is of course a notable exception to this rule, if you were diligent and used
file/directory integrity tools such as L5 you will be able to pinpoint the
affected files easily and deal with them.
There are also a variety of tools that do not quite fit under the headings
here, but are aimed at attack detection. One is the Linux Intrusion Detection
System (LIDS) project.
File system monitoring
So you've secured your machines, and done all the things that needed to be
done. So how do you make sure it's actually doing what it is supposed to do, or
prove to someone that it is as secure as you say it is? Well you conduct an
audit. This can be as simple as reviewing the installed software, configuration
files and other settings, or as complex as putting together or hiring a tiger
team (or ethical hackers, or whatever buzzword(s) you prefer) to actively try
and penetrate your security. If they can't then you did your job well (or they
suck), and if they do get in, you know what needs to be fixed (this is also a
good method to show the CIO that security is not a one shot affair, it is a
constant battle). One thing almost all attackers do is modify system files, once
you detect a break in, how do you know which files are ok and which are not?
Short of a complete reinstall the only way to be sure (and even then it's not
always 100%) is to use software to create signatures of files that cannot be
forged so you can compare them later on.
Tripwire
Tripwire is no longer a open source tool. I have absolutely NO problems with
commercial software. However, when you expect me to rely on a program to provide
security, when I (nor anyone else really) can not view the source (it is
available under some special license agreement, probably an NDA) I must decline.
Tripwire costs approximately $70 for Linux, and is only available as an RPM
package aimed at Red Hat Linux (tripwire is $500 for other operating systems). I
feel this is rather on the high side for a piece of software that can easily be
replaced with alternatives such as L5 or Gog&Magog. Tripwire is available https://www.tripwiresecurity.com/. There is also the "classic"
version available https://www.tripwire.org/.
AIDE
AIDE is a tripwire replacement that attempts to be better then tripwire. It
is GPL licensed which makes it somewhat more desirable then tripwire from a
trust point of view. It supports several hashing algorithms, and you can
download https://www.cs.tut.fi/~rammer/aide.html and https://sourceforge.net/projects/aide.
ViperDB
ViperDB checks setuid/setgid programs and folders and can notify you (via
syslog) of any changes or reset their permissions and ownership to what they
should be. ViperDB creates a series of databases (flat text files actually) in
the directory root, i.e.: /etc/.ViperDB might contain:
/etc/login.defs,1180,-,root,rw-,root,r--,r--,Apr,15,18:03
/etc/minicom.users,1048,-,root,rw-,root,r--,r--,Mar,21,19:11
/etc/CORBA,1024,d,root,rwx,root,r-x,r-x,Jun,14,16:51
/etc/X11,1024,d,root,rwx,root,r-x,r-x,Jun,14,23:05
/etc/cron.d,1024,d,root,rwx,root,r-x,r-x,Apr,14,17:09
Unfortunately ViperDB doesn't seem to handle sub directories, so you will
have to add them to the viperdb.ini file with something like:
find /etc/ -type d >> /usr/local/etc/viperdb.ini
viperdb.pl has 3 options, -init (creates a set of databases), -check (checks
files against databases, sends any messages to syslog, and then recreates the
databases) and checkstrict (checks files against databases, resets permissions
if necessary, sends any messages to syslog, and then recreates the databases).
What this means is if you use check, you will get a warning that say
/etc/passwd is now world writeable, and since it recreates the databases the
next time you run viperdb you will NOT get a warning. I would advise running
viperdb is checkstrict mode only, and make sure you run viperdb with the init
option after manipulating any file / folder permissions in protected
directories. ViperDB is available https://www.resentment.org/projects/viperdb/.
Pikt
Pikt is an extremely interesting tool, it is actually more of a scripting
language aimed at system administration then a simple program. Pikt allows you
to do things such as killing off idle user processes, enforcing mail quotas,
monitor the system for suspicious usage patterns (off hours, etc), and much
more. About the only problem with Pikt will be a steep learning tools, as it
uses it's own scripting language, but ultimately I think mastering this language
will pay off if you have many systems to administer (especially since Pikt runs
on Solaris, Linux and FreeBSD currently). Pikt is available https://pikt.org/.
Backups
Something people forget about, but you can compare the current files to old
backups, many backup formats (Tape, floppy, CDR, etc.) can be made read only, so
a backup of a newly installed system provides a good benchmark to compare things
to. The utility "diff" and "cmp" can be used to compare files against each
other.
Network monitoring / attack detection
If the last section has you worried you should be. There are however many
defenses, active and passive against those types of attacks. The best ways to
combat network scans are keep software up to date, only run what is needed, and
heavily restrict the rest through the use of firewalls and other mechanisms.
Luckily in Linux these tools are free and easily available, again I will only
cover open source tools, since the idea of a proprietary firewall/etc is rather
worrying. The first line of defense should be a robust firewall, followed by
packet filters on all Internet accessible machines, liberal use of TCP-WRAPPERS,
logging and more importantly automated software to examine the logs for you (it
is unfeasible for an administrator to read log files nowadays).
DTK
The Deception ToolKit is a set of programs that emulate well known services
in order to provide a false set of readings to attackers. The hope is to confuse
and slow down attackers by leading them to false conclusions, you can download
DTK from https://all.net/dtk/.
Psionic TriSentry - PortSentry, HostSentry and LogSentry
Psionic TriSentry consists of three components, PortSentry, HostSentry and
LogSentry. PortSentry detects and logs port scans, including stealthy scans
(basically anything nmap can do it should be able to detect). Psionic PortSentry
can be configured to block the offending machine (in my opinion a bad idea as it
could be used for a denial of service attack on legitimate hosts), making
completion of a port scan difficult. As this tool is in beta I would recommend
against using it, however with some age it should mature into a solid and useful
tool. HostSentry spots local anomolous behaviour in user accounts, and reports
situations that fall outside of normal parameters (i.e. Bob from accounting
logging in at 2 AM on a Sunday). The last component is LogSentry, essentially a
log file monitoring applications which will alert you if it spots strange
problems. TriSentry is available https://www.psionic.com/products/.
scanlogd
scanlogd monitors network packets and if a threshold is exceeded it logs the
packets. You can get it at: https://www.openwall.com/scanlogd/.
Firewalls
Most firewalls support logging of data, and ipfwadm/ipchains are no
exception, using the -l switch you get a syslog entry for each packet, using
automated filters (Perl is good for this) you can detect trends/hostile attempts
and so on. Since most firewalls (UNIX based, and Cisco in any case) log via the
syslog facility, you can easily centralize all your firewall packet logging on a
single host (with a lot of harddrive space hopefully).
TCP-WRAPPERS
Wietse's TCP-WRAPPERS allow you to restrict connections to various services
based on IP address and so forth, but even more importantly it allows you to
configure a response, you can have it email you, finger the offending machine,
and so on (use with caution however). TCP_WRAPPERS comes standard with most
distributions and is available ftp://ftp.porcupine.org/pub/security/.
Intrusion Detection Papers
FAQ: Network Intrusion Detection Systems, an excellent FAQ that covers all
the major (and many minor) issues with IDS systems. Available https://www.robertgraham.com/pubs/network-intrusion-detection.html.
Dealing with attacks
So you've done your homework, you installed tripwire, DTK, and so on. Now
what do you do when your pager starts going off at 3am and tells you that
someone just made changes on the primary NIS server? Dealing with an attack
depends on several factors, is the attack in progress? Did you discover your
company plan being sent out by the mail server to a Hotmail address? Did you get
called in to find a cluster of dead servers? What are your priorities? Restoring
service? Ensuring confidential data is safe? Prosecuting the attacker(s)?
Several things to keep in mind:
- Response from the admin will depend heavily on the environment they are
in. The attacker may have compromised the administrative accounts, so sending
email may not work.
- Most sites usually don't want to report attacks (successful or not) due to
the potential embarrassment and related public relations problems.
- Most quick attacks, denial of service attacks and the like are spoofed.
Tracking down the real attacker is very difficult and resource intensive.
- Even if all goes well there is a chance law enforcement will seize your
equipment as evidence, and hold it, not something to be taken lightly.
- Do you know how the attacker got in (i.e. NFR recorded it), if so you
might just want to plug the holes and go on.
- Try not to ignore attacks, but at the same time there are many people
running garbage attacks in an effort to waste administrators time and energy
(and possibly distract them from more subtle attacks).
Also before you deal with an attack, you should consult your company policy.
If you don't have one consult your manager, the legal department, etc. It's also
a good idea to have a game plan to deal with attacks (i.e., the mail server is
first priority, checking fileservers is number two, who do you notify, etc) this
will prevent a lot of problems when it happens (be prepared). The O'Reilly book
https://www.ora.com/catalog/puis/ "Practical Unix and Internet
Security" covers this topic in great detail so I'm not going to rehash it. Go
buy the book. There is also "Incident response" from O'Reilly, you should
probably read it, it's quite good.
Packet sniffers
Packet sniffing is the practice of capturing network data not destined for
your machine, typically for the purpose of viewing confidential/sensitive
traffic such as telnet sessions or people reading their email. Unfortunately
there is no real reliable way to detect a packet sniffer since it is mostly a
passive activity, however by utilizing network switches and fiber optic
backbones (which are very difficult to tap) you can minimize the threat. There
is also a tool called AntiSniff, that probes network devices and sees if their
response indicates an interface in promiscuous mode. These tools are also
invaluable if your network is under attack and you want to see what is going on.
There is an excellent FAQ on sniffing https://www.robertgraham.com/pubs/sniffing-faq.html.
Snort
Snort is the king of packet sniffers now. It can also be used to detect
various attacks. It can watch for activity such as Queso TCP-IP fingerprinting
scans, Nmap scans, and the like. Snort is available https://www.snort.org/.
tcpdump
The granddaddy of packet sniffers for Linux, this tool has existed as long as
I can remember, and is of primary use for debugging network problems. It is not
very configurable and lacks advanced features of newer packet sniffers, but it
can be useful. Most distributions ships with tcpdump.
Ethereal
A nice looking network protocol analyzer (a.k.a., a souped up sniffer) with
an interface very similar to NT's network monitor. It allows easy viewing of
data payloads for most network protocols (tftp, http, Netbios, etc). It is based
on GTK, thus meaning you will probably have to be running gnome to use it. I
haven't tested it yet (but intend to). It is available https://www.ethereal.com/. Ethereal can also be used to open
trace files from other software.
SPY
SPY is an advanced multi protocol sniffer that runs on various platforms. You
can get it https://www.gromeck.de/Spy/.
Other sniffers
There are a variety of packet sniffers for Linux, based on the libpcap
library among others, here is a short list:
Numerous other packet sniffers are available https://freshmeat.net/search/?site=Freshmeat&;q=packet+sniff§ion=projects.
Packet sniffer detection
In theory most operating systems leave tell tale signs when packet sniffing
(that is to say their network interfaces respond in certain, non standard ways
to network traffic). If the attacker is not to savvy, or is using a compromised
machine then chances are you can detect them. On the other hand if they are
using a specially built cable, or induction ring there is no chance of detecting
them unless you trace every physical piece of network cable and check what is
plugged into it.
AntiSniff
As mentioned before AntiSniff i a tool that probes network devices to try and
see if they are running in promiscuous mode, as opposed to normal modes of
operation. It is supposedly effective, and will work against most sniffers. You
can get it https://www.securitysoftwaretech.com/antisniff/. AntiSniff has
not been maintained in over a year.
