Bug 2028408
Summary: | Podman healthcheck fails if the command contains unicode characters. | ||
---|---|---|---|
Product: | Red Hat Enterprise Linux 8 | Reporter: | Ashish Reddy <asreddy> |
Component: | podman | Assignee: | Jindrich Novy <jnovy> |
Status: | CLOSED ERRATA | QA Contact: | Alex Jia <ajia> |
Severity: | medium | Docs Contact: | |
Priority: | unspecified | ||
Version: | 8.5 | CC: | bbaude, dwalsh, jligon, jnovy, lsm5, mheon, pthomas, snangare, tsweeney, umohnani, ypu |
Target Milestone: | rc | ||
Target Release: | --- | ||
Hardware: | All | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | podman-4.0.2-1.el8 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2022-11-08 09:14:55 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Ashish Reddy
2021-12-02 10:42:52 UTC
Giuseppe could you take a look at this? the healthcheck command seems to be executed correctly: $ podman run --name foo -d --health-interval 5m --health-timeout 3s --health-cmd "echo hi 2>&1" ubi8/ubi-minimal sleep 100000 $ (sleep 10; podman healthcheck run foo) & # sudo /usr/share/bcc/tools/execsnoop | grep echo sh 342878 342874 0 /bin/sh -c echo hi 2>&1 So is the issue you are seeing just in the "podman inspect" output? Even in this case, I am not sure it is an issue, since the command is correctly encoded: $ echo -e "mysqladmin --port 6446 --protocol TCP ping 2\u003c\u00261 | grep Access || exit 1" mysqladmin --port 6446 --protocol TCP ping 2<&1 | grep Access || exit 1 What is the '2<&1' for? Did you mean '2>&1'? How have you checked that the checkpoint command is launched incorrectly? The exact command customer using for healthcheck is as below: ~~~ "mysqladmin --port 6446 --protocol TCP ping 2>&1 | grep Access || exit 1" ~~~ With docker, the container is getting the whole command as seen from the inspect output of the container. But with podman it gives Unicode values for ">" and "&" characters. So it is failing the healthchecks for them. They have verified this by the inspect output from the container. I have also tested this from the inspect output. that doesn't answer my question. The "inspect" output is just a formatting difference, that doesn't mean the Podman will use exactly that string to run the healthcheck. The test I suggest to do is to create the container, then from another terminal run "/usr/share/bcc/tools/execsnoop" to see what processes are launched. e.g.: $ podman run -dit --rm --health-interval 5m --health-timeout 3s --health-cmd "echo Access 2>&1 | grep Access || exit 1" ubi8/ubi-minimal && sleep 20 && podman healthcheck run fa2880186f2bd596832aaf5705349e0d10d0b74155519ea904a7c6109d743609 (from another terminal) # /usr/share/bcc/tools/execsnoop | grep echo sh 702697 702693 0 /bin/sh -c echo Access 2>&1 | grep Access || exit 1 What command do you see launched? Hello, Yes, I can see the exact command in the execsnoop outputs. But the cu came back with the below query ~~~ When I run podman healthcheck run on my image built with Docker built or the original mysql/enterprise-router:8.0.26 I get: podman healthcheck run my_container Error: healthcheck command exceeded timeout of 0s The clue is 'exceeded timeout of 0s'. After a little further investigation a believe that the problem isn't the encoding, as you stated, but instead it's about default values for the healthcheck parameters. If no timeout is set in the image Podman defaults it to 0s and as a consequence the healthcheck command fails to complete within that interval (of course) and thus fails. To compare, Docker has a default 30s which is plenty of time for this simple healthcheck. Interval has a similar problem. Docker has a default of 30s but Podman defaults to nil. At least it appears so because the healthcheck is never run unless specified explicitly. I haven't checked the retries parameter but I wouldn't be surprised if it defaults to infinity if not set. Dockers default i s 3 ~~~ I could reproduce this with below dockerfile ~~~ [test@vm251-225 ~]$ cat Dockerfile FROM ubi8/ubi-minimal HEALTHCHECK CMD mysqladmin --port 6446 --protocol TCP ping 2>&1 | grep Access || exit 1 [test@vm251-225 ~]$ ~~~ Now when I run helathcheck on a container launched with this image, I get the below error. ~~~ [test@vm251-225 ~]$ podman healthcheck run 6916529718cf Error: healthcheck command exceeded timeout of 0s [test@vm251-225 ~]$ ~~~ From podman docs, I see that the default value is correct and is "30s" - https://docs.podman.io/en/latest/markdown/podman-run.1.html#health-interval-interval Also, is there anyway we could improve the output on the "podman inspect", so that it would be less confusing the issue seems to be that when there is no timeout specified in the image, we default to 0s. I am not very familiar with health checks and what we are supposed to do in this case. Does timeout=0s mean infinite? If it is set to 0 (we don't seem to make a difference between 0 and not set), should it be 30s? Brent, do you have any hints? upstream fixed proposed -> https://github.com/containers/podman/pull/12614 the fix was merged and it is part of podman 4 This bug has been verified on podman-4.0.2-1.module+el8.7.0+14421+e3b24aca. [root@sweetpig-18 ~]# cat /etc/redhat-release Red Hat Enterprise Linux release 8.7 Beta (Ootpa) [root@sweetpig-18 ~]# rpm -q podman runc systemd kernel podman-4.0.2-1.module+el8.7.0+14421+e3b24aca.x86_64 runc-1.0.3-3.module+el8.7.0+14440+ed5f482d.x86_64 systemd-239-58.el8.x86_64 kernel-4.18.0-372.2.1.el8.x86_64 [root@sweetpig-18 ~]# podman build --format docker -t foobar . STEP 1/2: FROM quay.io/libpod/alpine Trying to pull quay.io/libpod/alpine:latest... Getting image source signatures Copying blob 9d16cba9fb96 done Copying config 9617696764 done Writing manifest to image destination Storing signatures STEP 2/2: HEALTHCHECK CMD ls -l / 2>&1 COMMIT foobar --> f55ad067db6 Successfully tagged localhost/foobar:latest f55ad067db6d4aae7aa483dcd53425a58e1f80c2b36159fd88ad5c3ec2ea5dca [root@sweetpig-18 ~]# podman run -td --name hctest foobar ls 1eccb36e11f9e51be9a718f462c90b8429ba101498039d8cd4f0c6586575b018 [root@sweetpig-18 ~]# podman ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1eccb36e11f9 localhost/foobar:latest ls 7 seconds ago Exited (0) 5 seconds ago (starting) hctest [root@sweetpig-18 ~]# podman inspect hctest | grep -A2 CMD-SHELL "CMD-SHELL", "ls -l / 2>&1" ], Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: container-tools:rhel8 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:7457 |