Testing with the patch below, which calls mlockall after the first fork(), I'm seeing what I would expect:

[root@qe-c01-m01 ~]# grep VmLck /proc/`pidof corosync`/task/*/status
/proc/29170/task/29170/status:VmLck:      209688 kB
/proc/29170/task/29171/status:VmLck:      209688 kB
[root@qe-c01-m01 ~]# grep VmSwap /proc/`pidof corosync`/task/*/status
/proc/29170/task/29170/status:VmSwap:          0 kB
/proc/29170/task/29171/status:VmSwap:          0 kB

Note that the first fork() is done conditionally in corosync_tty_detach() so corosync -f shouldn't have this problem, and -f should not be used in reproducers.


diff --git a/exec/main.c b/exec/main.c
index 0922044..645153c 100644
--- a/exec/main.c
+++ b/exec/main.c
@@ -400,6 +400,8 @@ static void priv_drop (void)
        return; /* TODO: we are still not dropping privs */
 }
 
+static void corosync_mlockall (void);
+
 static void corosync_tty_detach (void)
 {
        int devnull;
@@ -424,7 +426,7 @@ static void corosync_tty_detach (void)
 
        /* Create new session */
        (void)setsid();
-
+       corosync_mlockall();
        /*
         * Map stdin/out/err to /dev/null.
         */

You're right. The mlockall() man page states

Memory locks are not inherited by a child created via fork(2)  and  are
automatically  removed  (unlocked)  during  an  execve(2)  or  when the
process terminates.

I've been running the s390x d_io tests against my quick and dirty patch from comment #1 and I haven't seen a corosync scheduling warning in 15+ hours. It would be good to get this one fixed for 7.4 since the stability gains with memory intensive workloads (like heavy buffered i/o) are pretty clear.

I'm tempted to mark this as a regression, but it hasn't worked since RHEL-5!

I don't know if there's any particular reason why mlockall is called so early in the startup, (it was later in RHEL-5 aisexec) but this patch works for me (including master branch with knet)

diff --git a/exec/main.c b/exec/main.c
index 0922044..3ea901d 100644
--- a/exec/main.c
+++ b/exec/main.c
@@ -1171,8 +1171,6 @@ int main (int argc, char **argv, char **envp)
                corosync_setscheduler ();
        }
 
-       corosync_mlockall ();
-
        /*
         * Other signals are registered later via qb_loop_signal_add
         */
@@ -1304,6 +1302,7 @@ int main (int argc, char **argv, char **envp)
        if (background) {
                corosync_tty_detach ();
        }
+       corosync_mlockall ();
 
        corosync_poll_handle = qb_loop_create ();

That looks more sensible to me.

The reason why mlockall was moved is ed7d054e552b4cb2a0cb502b65f84310ce6da844 (Remove priority inversion in logsys.). And I can fully agree with that patch - logsys should have had same priority as main corosync process. But mlockall shouldn't had been moved and it was for sure some accident.

So good catch, nice patch, ACK and merged as https://github.com/corosync/corosync/commit/86012ebb45c85df277eb8fe5dc5223e2fdebf25c .

I believe it shouldn't be problem to get this into 7.4.

Created attachment 1273899 [details]
Main: Call mlockall after fork

Main: Call mlockall after fork

Man page of mlockall is clear:
Memory locks are not inherited by a child created via fork(2) and are
automatically removed (unlocked) during an execve(2) or when the
process terminates.

So calling mlockall before corosync_tty_detach is noop when corosync is
executed as a daemon (corosync -f was not affected).

This regression is caused by ed7d054e552b4cb2a0cb502b65f84310ce6da844
(setprio for logsys/qblog was correct, mlockall was not).

Solution is to move corosync_mlockall call on correct place.

Signed-off-by: Andrew Price <anprice>
Reviewed-by: Christine Caulfield <ccaulfie>
Reviewed-by: Jan Friesse <jfriesse>
(cherry picked from commit 86012ebb45c85df277eb8fe5dc5223e2fdebf25c)

For QE: This bug is not really "reproducible" so test is in Comment 1.

*** Bug 1347782 has been marked as a duplicate of this bug. ***

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:1999