Bug 2008651 - posix_spawn failing when in podman container
Summary: posix_spawn failing when in podman container
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: glibc
Version: CentOS Stream
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: ---
Assignee: glibc team
QA Contact: qe-baseos-tools-bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-09-28 19:22 UTC by Dave Gimpl
Modified: 2023-07-18 14:29 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-10-07 13:31:51 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-98396 0 None None None 2021-09-28 19:23:04 UTC

Description Dave Gimpl 2021-09-28 19:22:30 UTC
Description of problem:
call to posix_spawn fails when running in stream 9 container

Version-Release number of selected component (if applicable):
CentOS-Stream-9-20210922.0-x86_64-dvd1.iso

How reproducible:
easily

Steps to Reproduce:
1. create centos stream 9 container, run it
2. run cmake (which calls posix_spawn)
3. fails

Actual results:
posix_spawn reutrns 1

Expected results:
posix_spawn reutrns 0

Additional info:

Comment 1 Florian Weimer 2021-09-28 19:35:08 UTC
What is the container host, and how do you launch the container? Thanks.

Comment 2 Dave Gimpl 2021-09-28 20:04:40 UTC
podman run -ti cent9stream:9 /bin/sh

is how I call the container.

I did a few more variations of test.  This works when the host is running centos stream 9, but fails when host is centos 8.

Comment 3 Florian Weimer 2021-09-28 20:15:16 UTC
(In reply to Dave Gimpl from comment #2)
> podman run -ti cent9stream:9 /bin/sh
> 
> is how I call the container.
> 
> I did a few more variations of test.  This works when the host is running
> centos stream 9, but fails when host is centos 8.

Thanks. Would you please test if it works on a CentOS 8 Stream host? I suspect this is a duplicate of bug 1985499.

Comment 4 Dave Gimpl 2021-09-28 20:36:21 UTC
When i run on a centos 8 stream host (4.18.0-305.10.2.el8_4.x86_64), the posix_spawn() fails within the centos 9 stream container.
1985499 seems similar in that its indicating EPERM is the return code, yet doesn't explicitly name posix_spawn but have to assume a thread is likely spawned via posix_spawn.  That one also showed verified as of Aug 3rd, which makes me wonder if the fix made it into the stream yet or is still a work in progress. (perhaps verified means someone was able to verify the bug exists, not necessarily verify the fix had been verified).

Comment 5 Florian Weimer 2021-09-28 21:10:57 UTC
(In reply to Dave Gimpl from comment #4)
> When i run on a centos 8 stream host (4.18.0-305.10.2.el8_4.x86_64), the
> posix_spawn() fails within the centos 9 stream container.

Sorry, that's not a CentOS 8 Stream kernel. I believe the most recent compose has kernel-4.18.0-338.el8. I don't believe there are any CentOS 8 Stream packages with an .el8_4 dist tag, due to the way CentOS 8 Stream is built.

Comment 6 Carlos O'Donell 2021-10-01 13:34:03 UTC
(In reply to Florian Weimer from comment #5)
> (In reply to Dave Gimpl from comment #4)
> > When i run on a centos 8 stream host (4.18.0-305.10.2.el8_4.x86_64), the
> > posix_spawn() fails within the centos 9 stream container.
> 
> Sorry, that's not a CentOS 8 Stream kernel. I believe the most recent
> compose has kernel-4.18.0-338.el8. I don't believe there are any CentOS 8
> Stream packages with an .el8_4 dist tag, due to the way CentOS 8 Stream is
> built.

Dave,

Have you had a chance to check your host configuration?

If you are running CentOS 8 Stream then you would have a fixed configuration that supports the newer clone3 syscall used in RHEL9 userspace.

Comment 8 Dave Gimpl 2021-10-04 19:07:05 UTC
One host I have is at
cat /etc/centos-release
CentOS Linux release 8.4.2105
which I believe is the regular CentOS 8.4.  This is the one with the libc issue.

Another has centos stream 9
cat /etc/centos-release
CentOS Stream release 9
which is the one that works.

I used a windows laptop with Docker Desktop (yes, I know, not podman) and via WSL2 it has the ability to properly run the centos stream 9 container image.

Another server has
cat /etc/centos-release
CentOS Stream release 8
which also works.

A colleague who uses a Mac indicates docker there also runs the centos stream 9 container image fine.

So, it seems its just CenOS 8.4 (perhaps too old, downlevel) that caused me to originally to create this tracker.

Comment 9 Carlos O'Donell 2021-10-04 21:04:40 UTC
(In reply to Dave Gimpl from comment #8)
> One host I have is at
> cat /etc/centos-release
> CentOS Linux release 8.4.2105
> which I believe is the regular CentOS 8.4.  This is the one with the libc
> issue.

Thanks, this clarifies things.

You are running a CentOS Stream 9 container on a CentOS 8 host.

This works, but often requires updates to the host for container security hardening policy.

> Another has centos stream 9
> cat /etc/centos-release
> CentOS Stream release 9
> which is the one that works.

This is expected to work and has all the fixes required for it to work.

> I used a windows laptop with Docker Desktop (yes, I know, not podman) and
> via WSL2 it has the ability to properly run the centos stream 9 container
> image.

Yes, I expect that WSL2 has likely updated their container security hardening policy defaults to fix this issue.

> Another server has
> cat /etc/centos-release
> CentOS Stream release 8
> which also works.

Yes, this is CentOS Stream 8 and it has all the fixes required to run a CentOS Stream 9 container.
 
> A colleague who uses a Mac indicates docker there also runs the centos
> stream 9 container image fine.

This is expected because we worked with upstream to ensure vendors were aware of the issue and updated their container security hardening policies.
 
> So, it seems its just CenOS 8.4 (perhaps too old, downlevel) that caused me
> to originally to create this tracker.

CentOS 8.4 should be able to run the CentOS Stream 9 container for specific workloads without any issue, *but* the container security hardening policy needs an update.

My suggestion is to switch to CentOS Stream 8, since that is where you'll get the best experience, and CentOS 8 goes EOL in 90 days.

Does that answer all of your questions?

Comment 10 Dave Gimpl 2021-10-07 12:49:04 UTC
I took recommendation and moved up to stream (given future of CentOS 8); operational now.
Yes, all questions answered.  Thanks.

Comment 11 Carlos O'Donell 2021-10-07 13:31:51 UTC
(In reply to Dave Gimpl from comment #10)
> I took recommendation and moved up to stream (given future of CentOS 8);
> operational now.
> Yes, all questions answered.  Thanks.

Dave,

I'm glad to hear things are working. We are always happy to provide guidance in these areas. If we get enough questions we can blog about it and expand our reach.

I'm marking this as CLOSED/NOTABUG.


Note You need to log in before you can comment on or make changes to this bug.