Bug 2081664

Summary: Active toolbox process blocks system shutdown
Product: [Fedora] Fedora Reporter: Stephane Travostino <steph>
Component: podmanAssignee: Matthew Heon <mheon>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 36CC: acui, bbaude, container-sig, debarshir, dwalsh, go-sig, harrymichal, jnovy, lsm5, mheon, ogutierr, patrick, pehunt, rh.container.bot, santiago
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: podman-4.4.1-3.fc36 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-02-25 04:00:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Stephane Travostino 2022-05-04 10:22:58 UTC
Description of problem:

If the system is reboot or shutdown while there's an active toolbox process, the systemd shutdown procedure will hang for 1min 30s until it's finally killed.

How reproducible:

Always.

Steps to Reproduce:
1. Open a toolbox shell with `toolbox enter`
2. Run `sleep infinity` in the toolbox
3. Reboot/shutdown

Expected results:

toolbox processes to be terminated immediately like any other host-level process.

Comment 1 Ondřej Míchal 2022-05-04 13:06:09 UTC
Yes, I can confirm this is a thing but I'm curious whether this is exclusive to Toolbx. Is it possible to find a reproducer with Podman itself? Something along the lines of:

1. Create a container
2. Start a shell session in the container with `podman exec -it ...`
3. Reboot/shutdown

Comment 2 Stephane Travostino 2022-05-04 13:18:53 UTC
It might be possible to reproduce on Podman (haven't tried yet), but I opened the bug here because while this behaviour might be desirable or not that much of an issue with podman, it is definitely an issue with the intended use case of toolbox.

As additional details, journalctl suggests that the hung shutdown is caused by /usr/bin/conmon not responding to signals. It only seems to be stuck when running an interactive process.

Example:

If I run `toolbox run sleep infinity`, the toolbox pod can be stopped immediately with `podman stop` or sending SIGINT to the conmon process.

If I run `toolbox run /bin/sh`, the toolbox pod CANNOT be stopped by `podman stop` (fails with: "container  has active exec sessions, refusing to clean up: container state improper"), and conmon doesn't respond to SIGINT.

The use case of toolbox is creating an interactive shell, so any long running process in the toolbox always cause the second, undesirable behaviour.

Comment 3 Debarshi Ray 2023-01-26 11:04:08 UTC
This was recently fixed in Podman:
https://github.com/containers/podman/pull/17025

Comment 4 Fedora Update System 2023-02-14 15:58:57 UTC
FEDORA-2023-431945fc20 has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2023-431945fc20

Comment 5 Fedora Update System 2023-02-15 02:26:43 UTC
FEDORA-2023-431945fc20 has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-431945fc20`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-431945fc20

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 6 Fedora Update System 2023-02-18 02:21:26 UTC
FEDORA-2023-998dbd3b79 has been pushed to the Fedora 36 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2023-998dbd3b79`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2023-998dbd3b79

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 7 Fedora Update System 2023-02-25 04:00:52 UTC
FEDORA-2023-998dbd3b79 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.