Bug 446250

Summary: IPV6DOD: xfrm reverse icmp feature does not seem to work correctly.
Product: Red Hat Enterprise Linux 5 Reporter: IBM Bug Proxy <bugproxy>
Component: kernelAssignee: Herbert Xu <herbert.xu>
Status: CLOSED ERRATA QA Contact: Martin Jenner <mjenner>
Severity: high Docs Contact:
Priority: urgent    
Version: 5.2CC: cward, davem, dzickus, jplans, nhorman, tgraf
Target Milestone: rcKeywords: OtherQA, ZStream
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-01-20 19:37:15 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 253764, 447688, 448732    
Attachments:
Description Flags
tcpdump of machine B upon receiving icmp error message.
none
Fix reverse flow lookup none

Description IBM Bug Proxy 2008-05-13 18:08:38 UTC
=Comment: #0=================================================
Joy M. Latten <latten.com> - 2008-05-12 21:28 EDT
---Problem Description---
Testing the new xfrm reverse feature for icmpv6 and am not able to get it to
work as I believe it should.

I have following setup.

   A ---------- B ---------- C -------- D
  eth0      eth1  eth0     eth0  eth1     eth0 
    
    B and C are gateways, with eth0-eth0 tunnel.
    A is behind B and D is behind C.

I remove D's interface to cause an icmp desination unreachable message.

My ip configuration below sets up a tunnel between B and C, so I expected the
icmp error message to be "reversed" and map to my tunnel SA. But this did not
happen. Instead src and dst was B(eth1) and A(eth0) respectively. 

Following is my ip config. I modified ip to accept a policy flag
to set XFRM_POLICY_ICMP so that outbound knows to do xfrm_reverse for icmp. 

ip config:

./ip xfrm policy flush
./ip xfrm state flush

./ip xfrm state add src fc00:0:0:105::64 dst fc00:0:0:105::35 proto esp spi
0x201 mode tunnel enc "cbc(des3_ede)"
0x303631383332323363323361323165386233366335363662

./ip xfrm state add src fc00:0:0:105::35 dst fc00:0:0:105::64 proto esp spi
0x3433 mode tunnel enc "cbc(des3_ede)"
0x303631383332323363323361323165386233366335363662

./ip xfrm policy add dir out flag icmp src fc00:0:0:141::/64 dst
fc00:0:0:165::/64 tmpl src fc00:0:0:105::64 dst fc00:0:0:105::35 proto esp mode
tunnel

./ip xfrm policy add dir in flag icmp src fc00:0:0:165::/64 dst
fc00:0:0:141::/64 tmpl src fc00:0:0:105::35 dst fc00:0:0:105::64 proto esp mode
tunnel

./ip xfrm policy add dir fwd flag icmp src fc00:0:0:165::/64 dst
fc00:0:0:141::/64 tmpl src fc00:0:0:105::35 dst fc00:0:0:105::64 proto esp mode
tunnel


In ipv6/icmp.c, icmpv6_send() we call xfrm_lookup first to see if an SA or SPD
entry exists. if not, xfrm6_decode_session_reverse(skb, &fl2) to reverse the
flow and then xfrm_nlookup() is called, but with fl instead of fl2. Thus we
continue to use the same flow instead of the reversed one.

After changing this such that xfrm_nlookup() gets called with fl2, the traffic
looks as I expected in that headers from icmp payload are reversed and used to
lookup SA/SPD. It finds the tunnel SA and uses this.

However, my ping6 is still not successful. It seems B receives the ESP packet,
the icmpv6_rcv appears to process everything successfully and forwards, and then
the pkt disappears.

Will continue to look at.

However, shouldn't xfrm_nlookup() be called with fl2?
Or am I misunderstanding how this works?

Contact Information = Joy Latten/latten.com
 
---uname output---
2.6.18-91.el5
 
Machine Type = p520
 
---------------------------------------------

Red Hat,
Please add  lwang and
herbert.xu to the cc list.
-thanks.

Comment 1 IBM Bug Proxy 2008-05-13 21:48:33 UTC
------- Comment From latten.com 2008-05-13 17:45 EDT-------
It looks like upstream kernel has this problem too...

Comment 2 IBM Bug Proxy 2008-05-13 21:48:35 UTC
Created attachment 305300 [details]
tcpdump of machine B upon receiving icmp error message.

This tcpdump is from machine B at eth0. It shows that machine B receives the
ESP packet just fine. It appears to also decrypt the ipsec packet just fine too
since I see ICMP Dest Unreachable pkt with src=C and dst=A being sent to A.
What I don't get is why do I see this pkt on eth0. Shouldn't this have been
forwarded to B's eth1 to send to A?
Perhaps this is why A never receives the packet. 

Yes, this is the problem... because just for the heck of it, I did a ping from
C to A and it worked just fine. The packet was forwarded to B's eth1 to send to
A. 
But the successful ping from C to A did not require ipsec.

So, I think after the icmp error message pkt is decrypted, ipsec is not
forwarding to correct interface or something.

Comment 3 Herbert Xu 2008-05-15 06:55:11 UTC
Yes the flow on the output side is wrong.  Since you've already patched it could
you send a patch to netdev? Thanks!

As to the other problem, please make sure that the ICMP flag is set on the state
on the receive side as otherwise the packet will fail the inbound policy check
and be discarded.

Comment 4 IBM Bug Proxy 2008-05-15 16:32:35 UTC
------- Comment From latten.com 2008-05-15 12:29 EDT-------
Herbert, yes, will create and send patch.
Geez! I forgot about the input flag! Will try it.

Comment 5 Herbert Xu 2008-05-16 03:42:11 UTC
Created attachment 305608 [details]
Fix reverse flow lookup

Here is a back-port to RHEL5.

Comment 8 IBM Bug Proxy 2008-05-16 21:08:48 UTC
------- Comment From latten.com 2008-05-16 17:04 EDT-------
The sending side works great with this fix, but I am still seeing problems on
receiving side. Even with inbound ICMP flag, XFRM_STATE_ICMP set in SA.

Because the tunnel is between 2 security gateways, B & C, the icmp error message
will get ipsec'd and sent on C. B will receive it... but he will only decrypt
the message and then it should get forwarded, right? B's icmp layer won't see
anything since he is not final destination, right?  A is final destination and
A's icmp layer will process. But it will not have a secpath/SA, or anything
since A is just a box behind the SG.

It seems on B, after packet is decrypted, it is put onto eth0, instead
of being forwarded to eth1.
I am not too familiar with this area, but after looking at code,
I am wondering if in ip6_input_finish(), "resubmit" happens and gets
wrong idev or something...

Comment 9 IBM Bug Proxy 2008-05-16 23:16:40 UTC
------- Comment From latten.com 2008-05-16 19:13 EDT-------
ok, forget what I said about "resubmit" since this is a tunnel mode pkt.

Comment 10 IBM Bug Proxy 2008-05-17 00:08:50 UTC
------- Comment From latten.com 2008-05-16 20:05 EDT-------
not sure why the decrypted pkt is forwarded onto eth0 and not eth1.
routes look correct in my routing table... when I ping C to A things are
forwarded correctly then. But not after decrypting an icmp error message when
src is C and dst is A...

Comment 11 Herbert Xu 2008-05-17 01:11:35 UTC
I see, you're trying to use it as an IPsec gateway as opposed to a host.  I
thought we agreed that we were only going to support this as a host, not a
router.  That's why I never implemented the forwarding part.  Do we want to
support a router as well?

Comment 13 IBM Bug Proxy 2008-05-19 20:24:44 UTC
------- Comment From latten.com 2008-05-19 16:19 EDT-------
Herbert, sorry, I forgot! I have been testing in tunnel mode with SGs!

In rfc 4301, Section 6.2, "The major security concern here is that a compromised
host or oruter might emit erroneous ICMP error messages that could degrade
service for other devices "behind" the security gateway, or that could even
result in violations of confidentiality."

I was thinking of this scenario when I started testing and tried to recreate it!

I am wondering... and would like your opinion, (since I am just now really
understanding all this) but would not this particular "protecting icmp error
messages" be more needed in cases where the packet will be forwarded "behind"
the gateway? If so, I guess we will eventually need it.

Comment 14 Herbert Xu 2008-05-20 01:54:02 UTC
Sorry, it's way too late to add any new features.  Besides, if we were going to
support IPsec as a gateway, then there is a lot more to be done than just ICMP.

Of course, this is something that should eventually be added to Linux, just not
straight away.

Comment 16 Don Zickus 2008-05-20 19:21:02 UTC
in kernel-2.6.18-93.el5
You can download this test kernel from http://people.redhat.com/dzickus/el5

Comment 18 IBM Bug Proxy 2008-06-27 04:24:37 UTC
------- Comment From anoop.vijayan.com 2008-06-27 00:18 EDT-------
Thanks! Closing this bug.

Comment 22 Chris Ward 2008-11-14 14:03:43 UTC
~~~ Attention Partners! ~~~

Please test this URGENT / HIGH priority bug at your earliest convenience to ensure it makes it into the upcoming RHEL 5.3 release. The fix should be present in the Partner Snapshot #2 (kernel*-122), available NOW at ftp://partners.redhat.com. As we are approaching the end of the RHEL 5.3 test cycle, it is critical that you report back testing results as soon as possible. 

If you have VERIFIED the fix, please add PartnerVerified to the Bugzilla Keywords field to indicate this. If you find that this issue has not been properly fixed, set the bug status to ASSIGNED with a comment describing the issues you encountered.

All NEW issues encountered (not part of this bug fix) should have a new bug created with the proper keywords and flags set to trigger a review for their inclusion in the upcoming RHEL 5.3 or other future release. Post a link in this bugzilla pointing to the new issue to ensure it is not overlooked.

For any additional questions, speak with your Partner Manager.

Comment 23 Chris Ward 2008-11-18 18:12:21 UTC
~~ Snapshot 3 is now available ~~ 

Snapshot 3 is now available for Partner Testing, which should contain a fix that resolves this bug. ISO's available as usual at ftp://partners.redhat.com. Your testing feedback is vital! Please let us know if you encounter any NEW issues (file a new bug) or if you have VERIFIED the fix is present and functioning as expected (add PartnerVerified Keyword).

Ping your Partner Manager with any additional questions. Thanks!

Comment 24 Chris Ward 2008-11-28 06:44:17 UTC
~~ Attention ~~ Snapshot 4 is now available for testing @ partners.redhat.com ~~

Partners, it is vital that we get your testing feedback on this important bug fix / feature request. If you are unable to test, please clearly indicate this in a comment to this bug or directly with your partner manager. If we do not receive your test feedback, this bug is at risk from being dropped from the release.

If you have VERIFIED the fix, please add PartnerVerified to the Bugzilla Keywords field, along with a description of the test results. 

If you encounter a new bug, CLONE this bug and request from your Partner manager to review. We are no longer excepting new bugs into the release, bar critical regressions.

Comment 25 IBM Bug Proxy 2008-11-28 14:30:38 UTC
Redhat,
Joy has verified this fix present in 5.3 and this is closed on IBM side. But he noticed the ip command in both 5.3 and prior releases need to add ICMP to policy flags and will plan a patch upstream. Please let me know if the PartnerVerified keyword can be added in the RH Issue Tracker bugzilla in the above circumstances.
Thanks!

Comment 26 Chris Ward 2008-11-28 14:46:24 UTC
IBM, Please file new bug(s) for the issues you mention in comment #25. I'll set PartnerVerified keyword for you. Thanks for the feedback!!

Comment 28 errata-xmlrpc 2009-01-20 19:37:15 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2009-0225.html