It was reported that the Linux kernel is being flagged by Nessus as vulnerable to: http://www.kb.cert.org/vuls/id/464113 In response to that VU# a fix was applied to the upstream kernel from 2.4.20+ which added a (th->rst) check but not a (th->fin) check. http://www.uwsg.iu.edu/hypermail/linux/kernel/0210.3/0380.html However I believe OpenLinux implemented a fin check too, because they issued an advisory mentioning they corrected this issue and it led to this plugin being written: http://www.nessus.org/plugins/index.php?view=viewsrc&id=11618
The original report of this issue contains some minor, but basic errors: SYN|data|FIN is a legal frame in T/TCP and arguably in basic TCP as well (although not used for that). Thus you need to write rules that correctly handle SYN|data|FIN packets. The code to handle this in Linux is, as far as our kernel experts can tell, entirely correct. The kernel was changed after the publication of the report to be suspicious of SYN|RST as that isn't a legal packet, but not SYN|FIN as that is a legal packet. We therefore believe that Linux (and hence Red Hat Enterprise Linux) does the correct thing in handling these packets, and that the Nessus test is giving a false positive as this is not a vulnerability.