Bug 235392 - 2.6.21.x kernels: IP MASQUERADE initialization problems
Summary: 2.6.21.x kernels: IP MASQUERADE initialization problems
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: iptables
Version: rawhide
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Thomas Woerner
QA Contact: Ben Levenson
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2007-04-05 15:26 UTC by Joseph Sacco
Modified: 2007-11-30 22:12 UTC (History)
1 user (show)

Fixed In Version: 2.6.21-1.3123.fc7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2007-05-02 06:59:56 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Mac-On-Linux TUN configuration script (3.26 KB, text/plain)
2007-04-11 18:36 UTC, Joseph Sacco
no flags Details
Mac-On-Linux TUN test framework source code (1.29 KB, text/plain)
2007-04-11 18:37 UTC, Joseph Sacco
no flags Details

Description Joseph Sacco 2007-04-05 15:26:50 UTC
The ipt_MASQUERADE driver is no longer automatically loaded when the tun driver
 is loaded:

Module                  Size  Used by
nf_nat                 20660  0
nf_conntrack_ipv4      13448  1
nf_conntrack           73408  2 nf_nat,nf_conntrack_ipv4
nfnetlink               8344  3 nf_nat,nf_conntrack_ipv4,nf_conntrack
ip_tables              14900  0
x_tables               18404  1 ip_tables
tun                    13504  1
mol                    59304  1

[/var/log/messages: Mac-on-linux starting up ]
Apr  5 10:59:36  plantain kernel: MOL 0.9.73-SVN kernel module loaded
Apr  5 10:59:36  plantain kernel: tun: Universal TUN/TAP device driver, 1.6
Apr  5 10:59:36  plantain kernel: tun: (C) 1999-2004 Max Krasnyansky
<maxk>
Apr  5 10:59:37  plantain kernel: ip_tables: (C) 2000-2006 Netfilter Core Team
Apr  5 10:59:37  plantain kernel: Netfilter messages via NETLINK v0.30.
Apr  5 10:59:37  plantain kernel: nf_conntrack version 0.5.0 (8192 buckets,
65536 max)
Apr  5 10:59:37  plantain kernel: nf_nat_init: can't setup rules.
Apr  5 10:59:37  plantain modprobe: FATAL: Error inserting iptable_nat
(/lib/modules/2.6.20-1.3045.fc7smp/kernel/net/ipv4/netfilter/iptable_nat.ko):
Invalid argument 
Apr  5 11:01:12  plantain ntpd[2011]: synchronized to 18.7.21.144, stratum 2
Apr  5 11:01:12  plantain ntpd[2011]: kernel time sync enabled 0001
Apr  5 11:01:51  plantain kernel: nf_nat_init: can't setup rules.
Apr  5 11:01:52  plantain modprobe: FATAL: Error inserting iptable_nat
(/lib/modules/2.6.20-1.3045.fc7smp/kernel/net/ipv4/netfilter/iptable_nat.ko):
Invalid argument 

This problem does *NOT* occur with the 2.6.20.x kernels in FC6.

If the Mac-On-Linux application is started and stopped two or more times, the
ipt_MASQUERADE driver magically gets loaded:

Module                  Size  Used by
xt_tcpudp               3424  2
ipt_MASQUERADE          4096  1
iptable_nat             8452  1
nf_nat                 20660  2 ipt_MASQUERADE,iptable_nat
nf_conntrack_ipv4      13448  2 iptable_nat
nf_conntrack           73408  4 ipt_MASQUERADE,iptable_nat,nf_nat,nf_conntrack_ipv4
nfnetlink               8344  3 nf_nat,nf_conntrack_ipv4,nf_conntrack
ip_tables              14900  1 iptable_nat
x_tables               18404  4 xt_tcpudp,ipt_MASQUERADE,iptable_nat,ip_tables
tun                    13504  1
mol                    59304  1

and all is well.

Starting from bootup... If the ipt_MASQUERADE driver is manually loaded using
modprobe, *before* Mac-On-Linux is first launched all is well.

So... something that used to automatically pull in ipt_MASQUERADE when the tun
driver and friends load has gone missing.


-Joseph

Comment 1 Dave Jones 2007-04-05 19:13:16 UTC
There's no linkage between the two modules, nor has the ever been.
Why it works for you in FC6 I've no idea, but I expect it's some configuration
that MOL has installed, because...

(15:08:00:root@gelk:~)# modprobe tun
(15:08:02:root@gelk:~)# lsmod | grep ipt
(15:08:04:root@gelk:~)# cat /etc/fedora-release 
Fedora Core release 6 (Zod)
(15:09:12:root@gelk:~)# uname -r
2.6.20-1.2925.fc6debug


This line:

FATAL: Error inserting iptable_nat
(/lib/modules/2.6.20-1.3045.fc7smp/kernel/net/ipv4/netfilter/iptable_nat.ko):
Invalid argument 

Is somewhat telling..  It's being passed an argument it doesn't understand.
What does your /etc/modprobe.conf look like?

Comment 2 Joseph Sacco 2007-04-05 22:58:13 UTC
Dave,

You are correct.  The tun module and the ipt_MASQUERADE module are not linked. I
need to crawl through the MOL code to see how the tunnel is set up initially.  I
think the problem may be there.

As an experiment, I created the /dev/net/tun device manually and then used an
'ip' command to set up tun1. Once the tunnel existed, running

   iptables -t nat -L

which loaded the nat-related modules,

followed by 

   iptables -t nat -A POSTROUTING -s 192.168.41.0/24 -d ! 192.168.41.0/24 -j
MASQUERADE

auto-magically loaded the ipt_MASQUERADE module.

The MOL networking code has worked for a very, very long time. If something has
changed with the 2.6.21.x series kernels then MOL will have to adjust.

I am attempting to extract the MOL tunnel setup code/scripts from MOL so I can
create a test framework.  Stay tuned...


-Joseph

Comment 3 Joseph Sacco 2007-04-11 18:36:47 UTC
Created attachment 152310 [details]
Mac-On-Linux TUN configuration script

Comment 4 Joseph Sacco 2007-04-11 18:37:47 UTC
Created attachment 152311 [details]
Mac-On-Linux TUN test framework source code

Comment 5 Joseph Sacco 2007-04-11 19:00:25 UTC
Sorry for the delay. Real work keeps getting in the way of open-source tasks.

I have created a test framework to demonstrate the problem. The framework
consists of two files:

* tunconfig
* tun-setup.c

The first file is a bash script that is part of MOL. This script configures the
tunnel.

The second file contains the the tunnel setup code extracted from MOL and a
driver that sets up a tunnel, forks a sub-process in which to run tunconfig,
spins for 15 seconds, cleans up, and exits.

BUILDING the test framework
---------------------------
* download both files into the same directory
* compile the .c file: gcc -g -o tun-set tun-setup.c

Running the test framework
----------------------------
* reboot to start with a system without any of the NAT modules loaded

* collect a listing of the kernel modules loaded

/sbin/lsmod > lsmod.boot

* run the test framework

./tun-setup 2>&1 | tee run1.log

* collect a listing of the kernel modules loaded

/sbin/lsmod > lsmod.run1

* collect a listing of the NAT tables

/sbin/iptables -L -t nat > nat-tables.run1

DISCUSSION
----------
FC6: iptable-1.3.5-1.2.1, kernel-2.6.20-1.2933.fc6smp

[MOL networking works OK]

What you should see after the testframework has been run

* kernel modules loaded
Module                  Size  Used by
xt_tcpudp               3584  2
ipt_MASQUERADE          4288  1
iptable_nat             8548  1
nf_nat                 20692  2 ipt_MASQUERADE,iptable_nat
nf_conntrack_ipv4      13608  2 iptable_nat
nf_conntrack           73128  4 ipt_MASQUERADE,iptable_nat,nf_nat,nf_conntrack_ipv4
nfnetlink               8504  3 nf_nat,nf_conntrack_ipv4,nf_conntrack
ip_tables              15060  1 iptable_nat
x_tables               18564  4 xt_tcpudp,ipt_MASQUERADE,iptable_nat,ip_tables

* NAT table listing
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
mol-ns-redirect  tcp  --  anywhere             192.168.41.1        tcp dpt:domain
mol-ns-redirect  udp  --  anywhere             192.168.41.1        udp dpt:domain

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  192.168.41.0/24     !192.168.41.0/24

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain mol-ns-redirect (2 references)
target     prot opt source               destination
DNAT       all  --  anywhere             anywhere            to:68.87.71.226


Fedora/rawhide: iptables-1.3.7-1.1, kernel-2.6.20-1.3056.fc7smp
[MOL networking does not work correctly]

What you should see:

RUN #1

* kernel modules
Module                  Size  Used by
nf_nat                 20660  0
nf_conntrack_ipv4      13448  1
nf_conntrack           73408  2 nf_nat,nf_conntrack_ipv4
nfnetlink               8344  3 nf_nat,nf_conntrack_ipv4,nf_conntrack
ip_tables              14900  0
x_tables               18404  1 ip_tables
tun                    13504  0


Absent: xt_tcpudp, ipt_MASQUERADE, iptable_nat

* NAT table listing
empty... 

the 'sanity' check within tunconfig appears to have failed/

RUN #2

* kernel modules
Module                  Size  Used by
ipt_MASQUERADE          4096  1
iptable_nat             8484  1
nf_nat                 20660  2 ipt_MASQUERADE,iptable_nat
nf_conntrack_ipv4      13448  2 iptable_nat
nf_conntrack           73408  4 ipt_MASQUERADE,iptable_nat,nf_nat,nf_conntrack_ipv4
nfnetlink               8344  3 nf_nat,nf_conntrack_ipv4,nf_conntrack
ip_tables              14900  1 iptable_nat
x_tables               18404  3 ipt_MASQUERADE,iptable_nat,ip_tables
tun                    13504  0

Absent: xt_tcpudp

* NAT table listing
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  0    --  192.168.41.0/24     !192.168.41.0/24

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain mol-ns-redirect (0 references)
target     prot opt source               destination

So... table two attempts, most but not all of the kernel modules have loaded.
The NAT table is beginning to "fill out".

RUN # 3->N
* flush the NAT tables

   /sbin/iptables -t nat -F

and repeat.

Eventually, all of the required kernel modules are loaded, and the NAT table
listing is correct.

Observation:
   The kernel modules get loaded because of calls to iptables in tunconfig.

Maybe there is a "wait" problem, where "something" does not wait for a
particular module to load before proceeding.

Thoughts???

-Joseph

Comment 6 Joseph Sacco 2007-04-13 14:15:44 UTC
Dave,

I think the problem is with iptables-1.3.7. 

I have been looking at the source code [novel idea, eh???].  iptables is
supposed to run modprobe to load modules as required.  I think there is a
problem with iptables-1.3.7 that does not exist with iptables-1.3.5 [used in FC6].

Consider the following:

* reboot to start clean

* run 'lsmod' and inspect the output
==> no NAT associated modules are loaded.

* run 'iptables -L -t nat'
 
==> See error message

iptables v1.3.7: can't initialize iptables table `nat': Table does not exist (do
you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

* run 'lsmod' a second time
==> See NAT-related modules were loaded in spite of error message

Module                  Size  Used by
iptable_nat             8484  0 
nf_nat                 20660  1 iptable_nat
nf_conntrack_ipv4      13448  2 iptable_nat
nf_conntrack           73408  3 iptable_nat,nf_nat,nf_conntrack_ipv4
nfnetlink               8344  3 nf_nat,nf_conntrack_ipv4,nf_conntrack
ip_tables              14900  1 iptable_nat
x_tables               18404  2 iptable_nat,ip_tables

* run 'iptables -L -t nat' a second time

Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

I will attempt to revert fedora/rawhide back to iptables-1.3.5 and run these
tests again.


-Joseph        

Comment 7 Joseph Sacco 2007-04-13 15:03:07 UTC
I reverted five packages:

   iptables-1.3.5-1.2.1.ppc.rpm
   iptables-devel-1.3.5-1.2.1.ppc.rpm
   iptables-ipv6-1.3.5-1.2.1.ppc.rpm
   system-config-securitylevel-1.6.27-1.ppc.rpm
   system-config-securitylevel-tui-1.6.27-1.ppc.rpm

and rebooted.

==> The error reported in comment #6 has "gone away".

I ran the test framework provided above.  ipt_MASQUERADE now loads, but all is
not quite well.  The NAT table listings are not correct.  The mol-ns-redirect
targets did not load. I suspect that replacing those five RPMs was necessary but
not sufficient.

After a reboot,  I tried starting Mac-on-Linux. It took two tries before all the
required modules loaded and networking worked.

So... If I did miss some RPM packages maybe the problem is strictly with
iptables.  If not, there may be some kernel module issues.


-Joseph

Comment 8 Dave Jones 2007-04-13 19:00:19 UTC
as I see it, this is either a MoL bug, or possibly an iptables regression.
AFAIR, nothing changed in the kernel nat modules, so it's not obvious to me how
this could be a kernel bug.

Reassigning.

Comment 9 Joseph Sacco 2007-04-13 19:08:54 UTC
Dave,

I agree.  This is most likely an iptables regression issue. The test in comment
#6 is telling.

-Joseph

Comment 10 Joseph Sacco 2007-04-20 14:08:37 UTC
[fedora/rawhide updated on 20Apr07]

Another factoid:
   
   Flushing the NAT tables sometimes produces an error message [see below]. 

 Question: 
  
    Is this indicative of a fork/exec/wait problem?

-Joseph

===================================================
# iptables -F -t nat
iptables: Invalid argument
# iptables -F -t nat
iptables: Invalid argument
# iptables -F -t nat
iptables: Invalid argument
# iptables -F -t nat
# iptables -F -t nat
iptables: Invalid argument
# iptables -F -t nat
iptables: Invalid argument
# iptables -F -t nat
# iptables -F -t nat
iptables: Invalid argument
# iptables -F -t nat
# iptables -F -t nat
# iptables -F -t nat
# iptables -F -t nat
# iptables -F -t nat
iptables: Invalid argument
# iptables -F -t nat
# iptables -F -t nat
# iptables -F -t nat
iptables: Invalid argument
# iptables -F -t nat
# iptables -F -t nat
iptables: Invalid argument


Comment 11 Joseph Sacco 2007-04-27 17:35:09 UTC
The problem persists with the 2.6.21-1.3116.fc7smp kernel.

-Joseph

Comment 12 Joseph Sacco 2007-04-27 18:30:52 UTC
I would appreciate it if someone would at least try to run the test program I
provided.


FWIW:

* If none of the IPV4 netfilter modules are loaded, running

     /sbin/iptables -L -t nat

reports an error the very first time it is run.

* Manually loading the requisite modules before running MOL eliminates all problems.

* Gentoo on PPC with iptables-1.3.7 and a 2.6.21-rc5 kernel does not have any
problem with modules not be loaded dynamically when MOL is launched [according
to the MOL maintainer.]

-Joseph

Comment 13 Thomas Woerner 2007-04-30 11:15:44 UTC
Can you please verify this with kernel 2.6.20-1.3104.fc7+ and iptables-1.3.7-2.
I can not reproduce the problems with these packages.

I also am getting no error with '/sbin/iptables -L -t nat' - even after lots of
calls without any netfilter modules loaded prior to the call.

Comment 14 Joseph Sacco 2007-04-30 13:22:10 UTC
Thomas,

I am running the latest fedora/rawhide code which includes
kernel-2.6.20-1.3116.fc7smp and iptables-1.3.7-2. 

Let's see...

* reboot
* run 'lsmod'
==> no netfilter modules loaded

* run 'iptables -L -t nat'
See error message:

iptables v1.3.7: can't initialize iptables table `NAT': Table does not exist 
(do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

[oops!!!]

* run 'lsmod'
See two ipv4/netfilter modules were loaded

Module                  Size  Used by
ip_tables              14900  0
x_tables               18404  1 ip_tables

* run 'iptables -L -t nat'
Now works OK:

Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination  

* run 'lsmod'
See five additional ipv4/netfilter modules have been added

Module                  Size  Used by
iptable_nat             8484  0 
nf_nat                 20660  1 iptable_nat
nf_conntrack_ipv4      13352  2 iptable_nat
nf_conntrack           73408  3 iptable_nat,nf_nat,nf_conntrack_ipv4
nfnetlink               8344  3 nf_nat,nf_conntrack_ipv4,nf_conntrack
ip_tables              14900  1 iptable_nat
x_tables               18404  2 iptable_nat,ip_tables

* run 'iptables -L -t nat' N more times
==> no error messages

This is new. This is good!!!

* run 'iptables -F -t nat' [N times]
See error message:

   iptables: Invalid argument

Why should an error occur when flushing all chains in the NAT table?


It would be helpful if:
* someone would take the time to run the test framework attached to this bug report.

* someone would check out this problem on a 32-bit PPC
[maybe it really is a kernel configuration issue]


-Joseph

Comment 15 David Woodhouse 2007-04-30 13:44:50 UTC
Hm, I see the error with 'iptables -t nat -F' too. Will investigate.

The scripts above are completely the wrong thing to do for MoL though -- you
need to add 'tun' support to initscripts, so that the system's networking
scripts bring up the tun/tap device, make it persistent and chown it to the
appropriate user.

All MoL should be doing is opening /dev/net/tun and attaching to that
preconfigured device to send/receive packets.


Comment 16 David Woodhouse 2007-04-30 13:56:43 UTC
It's intermittent even on rawhide. differences between strace output...

@@ -50,9 +50,8 @@ open("/selinux/mls", O_RDONLY|O_LARGEFIL
 read(3, "0", 19)                        = 1
 close(3)                                = 0
 socket(PF_INET, SOCK_RAW, IPPROTO_RAW)  = 3
-getsockopt(3, SOL_IP, 0x40 /* IP_??? */,
"nat\0\0\0\0\3\0\0\0\1\0\0\0\0\0\0\0\0\350S\335\10\350S\334\270\0\20\0\0\0\0\0\31\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0010\0\0\0\230\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0010\0\0\0\230\0\0\0\4\0\0\2x",
[84]) = 0
+getsockopt(3, SOL_IP, 0x40 /* IP_??? */,
"nat\0\0\0\0\3\0\0\0\1\0\0\0\0\0\0\0\0\320o\35\10\320o\34\270\0\20\0\0\0\0\0\31\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0010\0\0\0\230\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0010\0\0\0\230\0\0\0\4\0\0\2x",
[84]) = 0
 getsockopt(3, SOL_IP, 0x41 /* IP_??? */,
"nat\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\2x\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0p\0\230\0\0\0\1\0\0\0\0\0\0\0001\0\0\0\0\0\0\23\244\0(\0\0\362\205\324\214\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\377\377\377\376\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0p\0\230\0\0\0\20\0\0\0\0\0\0\0\3\0\0\0\0\0\0\0\324\0(\0\0\362\205\324\214\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\377\377\377\376\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0p\0\230\0\0\0\10\0\0\0\0\0\0\0\3\0\0\0\0\0\0\0\324\0(\0\0\362\205\324\214\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\377\377\377\376\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0p\0\260\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0@ERROR\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0ERROR\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0",
[672]) = 0
-setsockopt(3, SOL_IP, 0x40 /* IP_??? */,
"nat\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\31\0\0\0\4\0\0\2x\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0010\0\0\0\230\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0010\0\0\0\230\0\0\0\4\20\2\24\210\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0p\0\230\0\0\0\0\0\0\0\0\0\0\0001\0\0\0\0\0\0\23\244\0(\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\377\377\377\376\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0p\0\230\0\0\0\0\0\0\0\0\0\0\0\3\0\0\0\0\0\0\0\324\0(\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\377\377\377\376\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0p\0\230\0\0\0\0\0\0\0\0\0\0\0\3\0\0\0\0\0\0\0\324\0(\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\377\377\377\376\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0p\0\260\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0@ERROR\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0ERROR\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0",
728) = 0
-setsockopt(3, SOL_IP, 0x41 /* IP_??? */,
"nat\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\4\0\0\0\0\0\0\0\0\0\0\0001\0\0\0\0\0\0\23\244\0\0\0\0\0\0\0\3\0\0\0\0\0\0\0\324\0\0\0\0\0\0\0\3\0\0\0\0\0\0\0\324\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0",
104) = 0
-close(3)                                = 0
-exit_group(0)                           = ?
+setsockopt(3, SOL_IP, 0x40 /* IP_??? */,
"nat\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\31\0\0\0\4\0\0\2x\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0010\0\0\0\230\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0010\0\0\0\230\0\0\0\4\20\2\24\210\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0p\0\230\0\0\0\0\0\0\0\0\0\0\0001\0\0\0\0\0\0\23\244\0(\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\377\377\377\376\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0p\0\230\0\0\0\0\0\0\0\0\0\0\0\3\0\0\0\0\0\0\0\324\0(\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\377\377\377\376\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0p\0\230\0\0\0\0\0\0\0\0\0\0\0\3\0\0\0\0\0\0\0\324\0(\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\377\377\377\376\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0p\0\260\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0@ERROR\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0ERROR\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0",
728) = -1 EINVAL (Invalid argument)
+write(2, "iptables: Invalid argument\n", 27) = 27
+exit_group(1)                           = ?


Comment 17 Joseph Sacco 2007-04-30 14:28:29 UTC
David,

Thank you for taking the time to look into this issue. 

The tun/tap mechanisms used by MOL have been around for a very long time [2.4.x
kernels]. The mechanisms employed may need some updating. 

The test framework C program I hacked is taken in part from the MOL source.
FWIW, the approach mirrors the one decribed in the kernel documentation

    Documentation/networking/tuntap.txt

The tunconfig script, taken directly from MOL, has also been around for a very
long time [2.4.x kernels].  MOL has always created things on the fly and cleaned
up after itself when exiting. The mol application that is run as the last step
in the 'startmol' script

   exec $BIN_DIR/mol "${MOLARGS[@]}"

is run as root

  -rwsr-xr-x 1 root root 317716 Apr 16 10:29 mol

which, these days might be considered clumsy, dangerous, and unnecessary.

I will chat with the MOL maintainer, Joe Jazak,

   http://sourceforge.net/projects/mac-on-linux/
 
about your suggestions. MOL still supports a variety of older distros with 2.4.x
kernels, so "we" will need to understand what we can and cannot do to establish
and use the tun channel. All advice and counsel will be cheerfully accepted.


-Joseph


Comment 18 David Woodhouse 2007-05-01 08:04:02 UTC
It's SO_SET_REPLACE which is failing, at about line 2108 of libiptc.c. Enabling
the debugging shows no differences in output before that fails. There's a
difference in the 'libiptc-so_set_replace.blob' left in /tmp though...

-0000c0 00 00 00 00 00 00 05 5f 00 00 00 00 00 03 38 10
+0000c0 00 00 00 00 00 00 05 61 00 00 00 00 00 03 39 95


Comment 19 David Woodhouse 2007-05-01 09:13:42 UTC
That isn't relevant. The problem is entirely within the kernel. We have a
structure which must be 8-byte aligned (struct ipt_entry, because it includes
struct xt_counters). We allocate it with kmalloc(), make no attempt to ensure
it's suitably aligned when slab debugging is enabled, then barf because it's not
suitably aligned.

Comment 20 Joseph Sacco 2007-05-01 12:45:03 UTC
Interesting... PPC's are very fussy about alignment issues.

-Joseph

Comment 21 David Woodhouse 2007-05-02 06:59:56 UTC
Actually the PPC should be fine; I believe the ELF ABI aligns 64-bit integers to
64 bits for performance reasons.

For Fedora 7, I've set ARCH_KMALLOC_MINALIGN to 8 to ensure that all allocations
are suitably aligned for storing 64-bit integers. This disables the slab redzone
checks (which check for naughty code scribbling outside the area they were
allocated) but we're going to turn that off for the release anyway, so it's not
the end of the world.

Upstream, I've sent a slightly more complex patch which fixes other issues with
redzone checks on 64-bit machines too. 
http://www.mail-archive.com/linux-arch@vger.kernel.org/msg02646.html


Note You need to log in before you can comment on or make changes to this bug.