system_u:system_r:fail2ban_t:s0 root      3156     1  0 10:18 ?        00:00:02 /usr/bin/python2 -s /usr/bin/fail2ban-server -s /var/run/fail2ban/fail2ban.sock -p /var/run/fail2ban/fail2ban.pid -x -b

Please attach the list of SELinux denials that appear in your scenario. My guess is that the list will contain an { setrlimit } AVC.

# ausearch -m avc -m user_avc -m selinux_err -m user_selinux_err -i -ts today

Created attachment 1352713 [details]
ausearch output

This seems really odd to me.  Here are messages with dontaudit disabled.  But I'm not seeing anything obvious.  But I definitely have to be in permissive mode for the file limit to be increased.

Although it is probably the rlimitinh denial that does it - since presumably systemd sets the file limit before execing the daemon process.

Specifically:

type=AVC msg=audit(1510761934.613:358117): avc:  denied  { rlimitinh } for  pid=15602 comm="fail2ban-server" scontext=system_u:system_r:fail2ban_client_t:s0 tcontext=system_u:system_r:fail2ban_t:s0 tclass=process

since the server is started via fail2ban-client:

# grep Exec /usr/lib/systemd/system/fail2ban.service
ExecStart=/usr/bin/fail2ban-client -x start
ExecStop=/usr/bin/fail2ban-client stop
ExecReload=/usr/bin/fail2ban-client reload

This seems to do the trick:

module fail2ban-rlimit 1.0;

require {
        type fail2ban_client_t;
        type fail2ban_t;
        class process { rlimitinh };
}

allow fail2ban_client_t fail2ban_t:process { rlimitinh };

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3111