Bug 48767

Humm I suspect you havn't got persistance enabled for the virtual server (no
config file to look at so this is a guess). The experimental rpms offer a
facility on the virtual service

Persistance (time in seconds a connection to this machine should contine to go
to that machine for ftp the normal timeout is 900 seconds so this would be a
good value to place here)

Persistance netmask (should normally be 255.255.255.255)

Phil
=--=

Created attachment 23707 [details]
lvs.cf file

Since submitting this bug, I have tried to use iproute2 in order to change the 
source routing for the master.  This has work a little but I still have 
problems with this.  First, I have not found a way to when the backup to the 
master to take over, how to enter the iproute2 command.  Second, if I did find 
a way to put the command in, I would also need a way to take the command out 
when the master server comes back on line.

Another problem I found with this is that the source port for the data 
connection is not port 20, it is a high TCP port, so the remote firewall still 
rejects it after the IP address has been changed.

Another note is that persistance time out has no effect on this problem.  This 
is not a timeout problem at all.  It is a routing and port problem.

This is definitely a routing and port problem, and its due to the limitations of
ipchains, which is what piranha is using to create its LVS routes. The obvious
solution would be to redo piranha to implent netfilter/iptables so that both
passive and active ftp connections could be supported, instead of the current
"one or the other" scenario. However, while this is a good long-term goal, its
unlikely that this change will be made anytime soon.

Workaround:

Network topology. In this example, I have the two Piranha
nodes off the main network.  Their "outer" network is 192.168.26.0/24. 
The ftp servers are on an "inner" network (10.11.12.0/24).  

Piranha configuration (both nodes):
  Configured for LVS and NAT routing.  
  The virtual ftp service is configured to use firewall mark 21 (the
actual number used shouldn't matter), and has a persistence of 120
seconds.

FTP server configuration (same for all):
  The ftp servers must be configured to report the appropriate hostname
for passive ftp connections and to limit the ports used for those
connections.  You can do this by editing /etc/ftpaccess and adding the
following lines:
     passive address 192.168.26.31 192.168.0.0/16
     passive ports 192.168.0.0/16 10000 20000  
This says to the ftp server: "For clients in the netmask
192.168.0.0/16 report my hostname as 192.168.26.31 and use only ports
10000 to 20000 for passive connections."  A range of 10000 ports is
probably overkill, but keep in mind that with persistent connections,
there may be many ports in use at once.
  See also the man page for ftpaccess.

ipchains configuration (on piranha nodes):
  Piranha is looking for marked packets, so we must provide the marks. 
You'll want to mark all incoming packets to ports 21 and 10000 to 20000. 
The ipchains commands I used for this were:
ipchains -F
ipchains -A input -i eth0 -p tcp -d 192.168.26.31/32 21 -m 21
ipchains -A input -i eth0 -p tcp -d 192.168.26.31/32 10000:65535 -m 21
  Note that the mark matches the one piranha is looking for and the port
range matches the one the ftp servers were configured to use.
---Sample lvs.cf:--------------------------------------
serial_no = 334
primary = 192.168.26.11
primary_private = 10.11.12.11
service = lvs
backup_active = 1
backup = 192.168.26.12
backup_private = 10.11.12.12
heartbeat = 1heartbeat_port = 539
keepalive = 6
deadtime = 18
network = nat
nat_router = 10.11.12.13 eth1:0
nat_nmask = 255.255.255.0
reservation_conflict_action = preempt
debug_level = NONE
virtual lvs_ftp {
     active = 1
     address = 192.168.26.31 eth0:1
     vip_nmask = 255.255.255.0
     fwmark = 21
     port = 21
     persistent = 120
     send = "quit\n"
     expect = "220"     load_monitor = rup
     scheduler = rr
     protocol = tcp
     timeout = 6
     reentry = 15
     server [unnamed] {
         address = 10.11.12.1
         active = 1
         weight = 1
     }
     server [unnamed] {
         address = 10.11.12.2
         active = 1
         weight = 1
     }
     server [unnamed] {
         address = 10.11.12.3
         active = 1
         weight = 1
     }
}
---end of sample lvs.cf--------------------------------------

You'll need to --rebuild the attached srpms and upgrade to those in order to use
this workaround.

Created attachment 33781 [details]
ipvsadm-1.18-8

ok. thats not really ipvsadm-1.18-8. its piranha-0.6.0-8.src.rpm. this is what i
get when i try to multitask. :)

Created attachment 33782 [details]
scsi_reserve-0.7-5.src.rpm (for real. :)

Created attachment 33783 [details]
ipvsadm-1.15-1.src.rpm (honestly, it is. use this for kernel 2.2.X)

Actually it's all built out of the errata system and available externally on
ftp://ftp.linux.org.uk/pub/linux/piranha/6.2

6.2 is 2.2 based and the last release of ipvsadm for that kernel was 1.15-1
7.2 when it *really* gets released uses version 1.18 and above but is not
compatible with a 2.2 kernel

Phil
=--=