Bug 2033397

Summary: criu: Implement rseq support
Product: [Fedora] Fedora Reporter: Radostin Stoyanov <rstoyano>
Component: criuAssignee: Adrian Reber <adrian>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 36CC: adrian, aoliva, areber, arjun.is, avagin, codonell, dj, fweimer, law, mcermak, mfabian, pfrankli, rstoyanov1, rth, sipoyare
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: criu-3.16.1-12.fc36 criu-3.17-1.fc36 criu-3.17-1.fc35 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-05-07 04:23:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2033446    

Description Radostin Stoyanov 2021-12-16 16:51:00 UTC
Description of problem:

Most CRIU tests fail with Segmentation fault when running in a Rawhide container with glibc-2.34.9000-30. Everything seems to work with glibc-2.34.9000-29.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Run a test container:

    sudo podman run -it --rm \
    --privileged \
    --mount type=tmpfs,tmpfs-size=512M,destination=/tmp \
    --mount type=tmpfs,tmpfs-size=512M,destination=/run \
    --network=host \
    --cgroupns=host \
    -v /lib/modules:/lib/modules \
    registry.fedoraproject.org/fedora:rawhide bash

The following steps are executed inside the running container.

2. Install build dependencies:

    dnf install -y diffutils findutils gcc git gnutls-devel gzip iproute iptables nftables nftables-devel libaio-devel libasan libcap-devel libnet-devel libnl3-devel make procps-ng protobuf-c-devel protobuf-devel python3-flake8 python3-PyYAML python3-future python3-protobuf python3-junit_xml python-unversioned-command redhat-rpm-config sudo tar which e2fsprogs rubygem-asciidoctor kmod


3. Clone CRIU repository:

    git clone https://github.com/checkpoint-restore/criu

4. Compile CRIU:

    cd criu && make -j4

5. Run test:

    python3 ./test/zdtm.py run -t zdtm/static/vdso01


Actual results:

========================= Run zdtm/static/vdso01 in h ==========================
Start test
./vdso01 --pidfile=vdso01.pid --outfile=vdso01.out
Run criu dump
Run criu restore
=[log]=> dump/zdtm/static/vdso01/56/1/restore.log
------------------------ grep Error ------------------------
b'(00.003072) pie: 56:    `- skip pagemap'
b'(00.003075) pie: 56:    `- skip pagemap'
b'(00.003077) pie: 56:    `- skip pagemap'
b'(00.003080) pie: 56:    `- skip pagemap'
b'(00.003300) Error (criu/cr-restore.c:1480): 56 killed by signal 11: Segmentation fault'
b'(00.003318) Error (criu/cr-restore.c:2470): Restoring FAILED.'
------------------------ ERROR OVER ------------------------
################# Test zdtm/static/vdso01 FAIL at CRIU restore #################

Expected results:

========================= Run zdtm/static/vdso01 in h ==========================
Start test
./vdso01 --pidfile=vdso01.pid --outfile=vdso01.out
Run criu dump
Run criu restore
Send the 15 signal to  179
Wait for zdtm/static/vdso01(179) to die for 0.100000
Removing dump/zdtm/static/vdso01/179
========================= Test zdtm/static/vdso01 PASS =========================


Additional info:

It looks like CRIU fails in the restore phase. In this phase CRIU is using the so-called "parasite code" (https://criu.org/Parasite_code).

Comment 1 Florian Weimer 2021-12-16 20:09:34 UTC
It's rseq-related. Apparently CRIU has not implemented any rseq support yet. Turning off rseq registration in glibc makes the test pass for me (despite the logged error messages).

I do not see any rseq references in the CRIU upstream repository, despite the syscall being added to Linux 4.18 (so quite a while back).

# GLIBC_TUNABLES=glibc.pthread.rseq=0 python3 ./test/zdtm.py  run -t zdtm/static/vdso01
userns is supported
=== Run 1/1 ================ zdtm/static/vdso01
======================== Run zdtm/static/vdso01 in uns =========================
Start test
./vdso01 --pidfile=vdso01.pid --outfile=vdso01.out
Run criu dump
=[log]=> dump/zdtm/static/vdso01/62/1/dump.log
------------------------ grep Error ------------------------
b'(00.022041) \tRunning ip -6 route save'
b'(00.023376) \tRunning ip rule save'
b'(00.026419) iptables has nft backend: iptables-save v1.8.7 (nf_tables)'
b''
b'Error (criu/util.c:621): execvp("iptables-legacy-save", ...) failed: No such file or directory'
b'(00.027019) Error (criu/util.c:636): exited, status=1'
b'(00.027032) Error (criu/util.c:1502): iptables-legacy-save -V failed'
b'(00.028462) iptables has nft backend: ip6tables-save v1.8.7 (nf_tables)'
b''
b'Error (criu/util.c:621): execvp("ip6tables-legacy-save", ...) failed: No such file or directory'
b'(00.029056) Error (criu/util.c:636): exited, status=1'
b'(00.029069) Error (criu/util.c:1502): ip6tables-legacy-save -V failed'
------------------------ ERROR OVER ------------------------
Run criu restore
Send the 15 signal to  86
Wait for zdtm/static/vdso01(86) to die for 0.100000
Removing dump/zdtm/static/vdso01/62
========================= Test zdtm/static/vdso01 PASS =========================
========================= Run zdtm/static/vdso01 in h ==========================
Start test
./vdso01 --pidfile=vdso01.pid --outfile=vdso01.out
Run criu dump
Run criu restore
Send the 15 signal to  157
Wait for zdtm/static/vdso01(157) to die for 0.100000
Removing dump/zdtm/static/vdso01/157
========================= Test zdtm/static/vdso01 PASS =========================
========================= Run zdtm/static/vdso01 in ns =========================
Start test
./vdso01 --pidfile=vdso01.pid --outfile=vdso01.out
Run criu dump
=[log]=> dump/zdtm/static/vdso01/219/1/dump.log
------------------------ grep Error ------------------------
b'(00.022675) \tRunning ip -6 route save'
b'(00.024267) \tRunning ip rule save'
b'(00.027671) iptables has nft backend: iptables-save v1.8.7 (nf_tables)'
b''
b'Error (criu/util.c:621): execvp("iptables-legacy-save", ...) failed: No such file or directory'
b'(00.028444) Error (criu/util.c:636): exited, status=1'
b'(00.028462) Error (criu/util.c:1502): iptables-legacy-save -V failed'
b'(00.030365) iptables has nft backend: ip6tables-save v1.8.7 (nf_tables)'
b''
b'Error (criu/util.c:621): execvp("ip6tables-legacy-save", ...) failed: No such file or directory'
b'(00.031165) Error (criu/util.c:636): exited, status=1'
b'(00.031182) Error (criu/util.c:1502): ip6tables-legacy-save -V failed'
------------------------ ERROR OVER ------------------------
Run criu restore
Send the 15 signal to  241
Wait for zdtm/static/vdso01(241) to die for 0.100000
Removing dump/zdtm/static/vdso01/219
========================= Test zdtm/static/vdso01 PASS =========================

Comment 2 Radostin Stoyanov 2022-02-02 13:47:17 UTC
We have updated the Fedora Rawhide package for CRIU with patches that implement initial support for rseq.
There is an open pull request upstream [1] and pending release [2].

[1] https://github.com/checkpoint-restore/criu/pull/1706
[2] https://github.com/checkpoint-restore/criu/pull/1727

Comment 3 Florian Weimer 2022-02-02 14:01:51 UTC
(In reply to Radostin Stoyanov from comment #2)
> We have updated the Fedora Rawhide package for CRIU with patches that
> implement initial support for rseq.
> There is an open pull request upstream [1] and pending release [2].
> 
> [1] https://github.com/checkpoint-restore/criu/pull/1706
> [2] https://github.com/checkpoint-restore/criu/pull/1727

This is great, thanks!

Comment 4 Ben Cotton 2022-02-08 20:20:15 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 36 development cycle.
Changing version to 36.

Comment 5 Fedora Update System 2022-04-13 07:38:31 UTC
FEDORA-2022-aebdfb72f9 has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-aebdfb72f9

Comment 6 Fedora Update System 2022-04-13 19:49:04 UTC
FEDORA-2022-aebdfb72f9 has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-aebdfb72f9`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-aebdfb72f9

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Fedora Update System 2022-05-06 22:51:00 UTC
FEDORA-2022-f7a8201e8c has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-f7a8201e8c

Comment 8 Fedora Update System 2022-05-06 22:52:06 UTC
FEDORA-2022-d3a5b47584 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2022-d3a5b47584

Comment 9 Fedora Update System 2022-05-07 04:23:38 UTC
FEDORA-2022-aebdfb72f9 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 10 Fedora Update System 2022-05-08 03:06:51 UTC
FEDORA-2022-f7a8201e8c has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-f7a8201e8c`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-f7a8201e8c

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 11 Fedora Update System 2022-05-08 03:27:23 UTC
FEDORA-2022-d3a5b47584 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2022-d3a5b47584`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2022-d3a5b47584

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 12 Fedora Update System 2022-05-16 01:08:26 UTC
FEDORA-2022-f7a8201e8c has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 13 Fedora Update System 2022-05-16 02:05:04 UTC
FEDORA-2022-d3a5b47584 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.