File Transport Protocol (FTP) is an old and complex multi-port protocol
that presents a distinct set of challenges to a clustered environment. To
understand the nature of these challenges, you must first understand some
key things about how FTP works.
With most other server client relationships, the client machine opens
up a connection to the server on a particular port and the server then
responds to the client on that port. When an FTP client connects to an
FTP server it opens a connection to the FTP control port 21. Then the
client tells the FTP server
whether to establish an active or
passive connection. The type of connection
chosen by the client determines how the server responds and on what
ports transactions will occur.
The two types of data connections are:
Active Connections
When an active connection is established, the
server opens a data connection to the
client from port 20 to a high range port on the client
machine. All data from the server is then passed over this
connection.
Passive Connections
When a passive connection is established, the
client asks the FTP server to establish a
passive connection port, which can be on any port higher than
10,000. The server then binds to this high-numbered port for this
particular session and relays that port number back to the
client. The client then opens the newly bound port for the data
connection. Each data request the client makes results in a
separate data connection. Most modern FTP clients attempt to
establish a passive connection when requesting data from servers.
The two important things to note about all of this in regards to
clustering is:
The client determines the type of
connection, not the server. This means, to effectively cluster FTP,
you must configure the LVS routers to handle both active and passive
connections.
The FTP client/server relationship can potentially open a
large number of ports that the
Piranha Configuration Tool and IPVS do not know about.
IPVS packet forwarding only allows connections in and out of the
cluster based on it recognizing its port number or its firewall
mark. If a client from outside the cluster attempts to open a port
IPVS is not configured to handle, it drops the connection. Similarly,
if the real server attempts to open a connection back out to the
Internet on a port IPVS does not know about, it drops the
connection. This means all connections from FTP
clients on the Internet must have the same
firewall mark assigned to them and all connections from the FTP server
must be properly forwarded to the Internet using
network packet filtering rules.
Before assigning any iptables rules for FTP
service, review the information in Section 9.3.1 Assigning Firewall Marks
concerning multi-port services and techniques for checking the
existing network packet filtering rules.
Below are rules which assign the same firewall mark, 21, to FTP
traffic. For these rules to work properly, you must also use the
VIRTUAL SERVER subsection of
Piranha Configuration Tool to configure a virtual server
for port 21 with a value of 21 in the
Firewall Mark field. See Section 10.6.1 The VIRTUAL SERVER Subsection for details.
The rules for active connections tell the kernel to accept and
forward connections coming to the internal
floating IP address on port 20 — the FTP data port.
In the above iptables commands,
n.n.n should be replaced with the first
three values for the floating IP for the NAT interface's internal
network interface defined in the GLOBAL SETTINGS
panel of Piranha Configuration Tool. The command allows the
LVS router to accept outgoing connections from the real servers that
IPVS does not know about.
The rules for passive connections assign the appropriate firewall
mark to connections coming in from the Internet to the floating IP
for the service on a wide range of ports — 10,000 to
20,000.
Warning
If you are limiting the port range for passive connections, you
must also configure the VSFTP server to use a matching port
range. This can be accomplished by adding the following
lines to /etc/vsftpd.conf:
pasv_min_port=10000
pasv_max_port=20000
You must also control the address that the server displays to
the client for passive FTP connections. In a NAT routed LVS
system, add the following line to
/etc/vsftpd.conf to override the real server
IP address to the VIP, which is what the client sees upon
connection. For example:
pasv_address=X.X.X.X
Replace X.X.X.X with the VIP
address of the LVS system.
For configuration of other FTP servers, consult the respective
documentation.
This range should be a wide enough for most situations;
however, you can increase this number to include all available
non-secured ports by changing
10000:20000 in the commands below
to 1024:65535.
iptables
/sbin/iptables -t mangle -A PREROUTING -p tcp \
-d n.n.n.n/32 \
--dport 21 -j MARK --set-mark 21/sbin/iptables -t mangle -A PREROUTING -p tcp \
-d n.n.n.n/32 \
--dport 10000:20000 -j MARK --set-mark 21
In the above iptables commands,
n.n.n.n should be replaced with the
floating IP for the FTP virtual server defined in the
VIRTUAL SERVER subsection of
Piranha Configuration Tool. These commands have the net
effect of assigning any traffic addressed to the floating IP on the
appropriate ports a firewall mark of 21, which is in turn recognized
by IPVS and forwarded appropriately.
Warning
The commands above take effect immediately, but do not
persist through a reboot of the system. To ensure network packet
filter settings are restored after a reboot, see
Section 9.5 Saving Network Packet Filter Settings
