Bug 1915377 - Unable to install iputils package
Summary: Unable to install iputils package
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: sandboxed-containers
Version: 4.8
Hardware: All
OS: Linux
high
high
Target Milestone: ---
: ---
Assignee: Christophe de Dinechin
QA Contact: Cameron Meadors
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-01-12 14:30 UTC by Qian Cai
Modified: 2022-09-21 08:22 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-09-21 08:22:19 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Qian Cai 2021-01-12 14:30:53 UTC
Description of problem:
# podman run --rm --runtime=kata -it ubi8 bash
[root@24821b30f377 ~]# dnf -y install iputils
Error unpacking rpm package iputils-20180629-2.el8.x86_64
  Running scriptlet: iputils-20180629-2.el8.x86_64                                                                                                                                                             3/3 
error: unpacking of archive failed on file /usr/bin/ping;5ffda71f: cpio: cap_set_file
error: iputils-20180629-2.el8.x86_64: install failed

Even adding --cap-add=ALL did not help.

Vanilla containers works fine.

Version-Release number of selected component (if applicable):
podman-2.0.5-5.module+el8.3.0+8221+97165c3f.x86_64
kata-agent-1.11.3-1.el8.x86_64
kata-shim-1.11.3-1.el8.x86_64
kata-osbuilder-1.11.3-1.el8.x86_64
kata-runtime-1.11.3-1.el8.x86_64

Comment 1 Qian Cai 2021-01-14 18:22:33 UTC
Same thing with crio.

# cat container-pod.json 
{
  "metadata": {
      "name": "ubi8"
  },
  "image":{
      "image": "registry.access.redhat.com/ubi8"
  },
  "command": [
      "sleep", "3600"
  ],
  "log_path":"ubi8.0.log",
  "linux": {
  }
}


cri-tools-1.20.0-1.el8.x86_64
cri-o-1.20.0-0.rhaos4.7.git845747f.el8.40.x86_64

# crictl exec -it 6d2a85e08c914062682e3d1e468b943ec03ddb363787057115bfc27707bfe668 bash

# dnf -y install iputils
Updating Subscription Management repositories.
Unable to read consumer identity
Subscription Manager is operating in container mode.

This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.

Last metadata expiration check: 0:00:27 ago on Thu Jan 14 18:17:00 2021.
Dependencies resolved.
===================================================================================================================================================================================================================
 Package                                         Architecture                                   Version                                                 Repository                                            Size
===================================================================================================================================================================================================================
Installing:
 iputils                                         x86_64                                         20180629-2.el8                                          ubi-8-baseos                                         149 k

Transaction Summary
===================================================================================================================================================================================================================
Install  1 Package

Total download size: 149 k
Installed size: 361 k
Downloading Packages:
iputils-20180629-2.el8.x86_64.rpm                                                                                                                                                   16 kB/s | 149 kB     00:09    
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                               16 kB/s | 149 kB     00:09     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                           1/1 
  Installing       : iputils-20180629-2.el8.x86_64                                                                                                                                                             1/1 
Error unpacking rpm package iputils-20180629-2.el8.x86_64
  Verifying        : iputils-20180629-2.el8.x86_64                                                                                                                                                             1/1 
Installed products updated.

Failed:
  iputils-20180629-2.el8.x86_64

Comment 2 Qian Cai 2021-01-14 19:03:26 UTC
# cat /var/log/dnf.log

2021-01-14T18:55:44Z DEBUG DNF version: 4.2.23
2021-01-14T18:55:44Z DDEBUG Command: dnf -y install iputils 
2021-01-14T18:55:44Z DDEBUG Installroot: /
2021-01-14T18:55:44Z DDEBUG Releasever: 8
2021-01-14T18:55:44Z DEBUG cachedir: /var/cache/dnf
2021-01-14T18:55:44Z DDEBUG Base command: install
2021-01-14T18:55:44Z DDEBUG Extra commands: ['-y', 'install', 'iputils']
2021-01-14T18:55:44Z DEBUG User-Agent: constructed: 'libdnf (Red Hat Enterprise Linux 8.3; generic; Linux.x86_64)'
2021-01-14T18:55:44Z DEBUG repo: downloading from remote: ubi-8-baseos
2021-01-14T18:55:57Z DEBUG ubi-8-baseos: using metadata from Tue Jan  5 07:27:32 2021.
2021-01-14T18:55:57Z DEBUG repo: downloading from remote: ubi-8-appstream
2021-01-14T18:56:10Z DEBUG ubi-8-appstream: using metadata from Thu Jan 14 04:51:30 2021.
2021-01-14T18:56:10Z DEBUG repo: downloading from remote: ubi-8-codeready-builder
2021-01-14T18:56:20Z DEBUG ubi-8-codeready-builder: using metadata from Tue Dec 15 07:24:46 2020.
2021-01-14T18:56:20Z INFO Last metadata expiration check: -1 day, 23:59:59 ago on Thu Jan 14 18:56:21 2021.
2021-01-14T18:56:21Z DDEBUG timer: sack setup: 36983 ms
2021-01-14T18:56:21Z DEBUG Completion plugin: Generating completion cache...
2021-01-14T18:56:21Z DEBUG --> Starting dependency resolution
2021-01-14T18:56:21Z DEBUG ---> Package iputils.x86_64 20180629-2.el8 will be installed
2021-01-14T18:56:21Z DEBUG --> Finished dependency resolution
2021-01-14T18:56:21Z DDEBUG timer: depsolve: 187 ms
2021-01-14T18:56:21Z INFO Dependencies resolved.
2021-01-14T18:56:21Z INFO ===================================================================================================================================================================================================================
 Package                                         Architecture                                   Version                                                 Repository                                            Size
===================================================================================================================================================================================================================
Installing:
 iputils                                         x86_64                                         20180629-2.el8                                          ubi-8-baseos                                         149 k

Transaction Summary
===================================================================================================================================================================================================================
Install  1 Package

2021-01-14T18:56:21Z INFO Total download size: 149 k
2021-01-14T18:56:21Z INFO Installed size: 361 k
2021-01-14T18:56:21Z INFO Downloading Packages:
2021-01-14T18:56:27Z INFO -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2021-01-14T18:56:27Z INFO Total                                                                                                                                                                               26 kB/s | 149 kB     00:05     
2021-01-14T18:56:27Z INFO Running transaction check
2021-01-14T18:56:27Z INFO Transaction check succeeded.
2021-01-14T18:56:27Z INFO Running transaction test
2021-01-14T18:56:27Z INFO Transaction test succeeded.
2021-01-14T18:56:27Z DDEBUG timer: transaction test: 118 ms
2021-01-14T18:56:27Z INFO Running transaction
2021-01-14T18:56:27Z DEBUG RPMDB altered outside of DNF.
2021-01-14T18:56:27Z DDEBUG RPM transaction start.
2021-01-14T18:56:27Z DDEBUG RPM transaction over.
2021-01-14T18:56:27Z DEBUG Errors occurred during transaction.
2021-01-14T18:56:27Z DDEBUG timer: verify transaction: 72 ms
2021-01-14T18:56:27Z DDEBUG timer: transaction: 363 ms
2021-01-14T18:56:27Z DEBUG Completion plugin: Generating completion cache...
2021-01-14T18:56:28Z INFO Installed products updated.
2021-01-14T18:56:28Z INFO 
Failed:
  iputils-20180629-2.el8.x86_64                                                                                                                                                                                    

2021-01-14T18:56:28Z DDEBUG Cleaning up.
2021-01-14T18:56:28Z DDEBUG /var/cache/dnf/ubi-8-baseos-53c30a88cff3796c/packages/iputils-20180629-2.el8.x86_64.rpm removed
2021-01-14T18:56:28Z SUBDEBUG 
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 67, in main
    return _main(base, args, cli_class, option_parser_class)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 106, in _main
    return cli_run(cli, base)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 130, in cli_run
    ret = resolving(cli, base)
  File "/usr/lib/python3.6/site-packages/dnf/cli/main.py", line 176, in resolving
    base.do_transaction(display=displays)
  File "/usr/lib/python3.6/site-packages/dnf/cli/cli.py", line 257, in do_transaction
    raise dnf.exceptions.Error(_('Transaction failed'))
dnf.exceptions.Error: Transaction failed
2021-01-14T18:56:28Z CRITICAL Error: Transaction failed

Comment 3 Qian Cai 2021-02-05 13:44:01 UTC
This can be also reproduced on OCP 4.7 even using fedora image.

Client Version: 4.7.0-0.nightly-2021-02-04-031352
Server Version: 4.7.0-0.nightly-2021-02-04-132953
Kubernetes Version: v1.20.0+cac2421

# dnf -y install iputils
Last metadata expiration check: 0:05:09 ago on Fri Feb  5 13:35:21 2021.
Dependencies resolved.
===================================================================================================================================================================================================================
 Package                                          Architecture                                    Version                                                    Repository                                       Size
===================================================================================================================================================================================================================
Installing:
 iputils                                          x86_64                                          20180629-4.fc30                                            fedora                                          123 k

Transaction Summary
===================================================================================================================================================================================================================
Install  1 Package

Total download size: 123 k
Installed size: 351 k
Downloading Packages:
iputils-20180629-4.fc30.x86_64.rpm                                                                                                                                                 269 kB/s | 123 kB     00:00    
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                              110 kB/s | 123 kB     00:01     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                           1/1 
  Installing       : iputils-20180629-4.fc30.x86_64                                                                                                                                                            1/1 
Error unpacking rpm package iputils-20180629-4.fc30.x86_64
  Verifying        : iputils-20180629-4.fc30.x86_64                                                                                                                                                            1/1 

Failed:
  iputils-20180629-4.fc30.x86_64                                                                                                                                                                                   

Error: Transaction failed

Comment 4 Qian Cai 2021-02-05 15:16:05 UTC
Today's kata 2.0 has the same issue.

Comment 5 Qian Cai 2021-02-05 17:03:15 UTC
I was told this could be virtiofs related even though I tried -o xattr (for virtiofsd) without luck.

Comment 6 Vivek Goyal 2021-02-05 17:57:53 UTC
Is there anything in selinux in guest. We have had reported issues of package installation failing because of SELinux in guest. There is a separate bug to fix that and  patches are making its way through upstream. Disable SELinux in guest and see if that works.

Comment 7 Qian Cai 2021-02-05 19:23:00 UTC
There is no selinux running inside the guest. Set the host's selinux to permissive did not help either.

Comment 8 Dr. David Alan Gilbert 2021-02-08 10:23:50 UTC
This is probably it not being able to set the security.capability xattr - that would explain it for RHEL which has:
]# getcap /usr/bin/ping
/usr/bin/ping = cap_net_admin,cap_net_raw+p

but fedora doesn't seem to.

Can you please give the complete set of virttiofs options?

Comment 9 Qian Cai 2021-02-08 12:50:51 UTC
/usr/libexec/virtiofsd --fd=3 -o source=/run/kata-containers/shared/sandboxes/ab0f9795e9f2823e75c70d01977e655a62d5f780896f43e4d6ea430ce3fb76d3/shared -o cache=always --syslog -o no_posix_lock -f

Also tried "-o xattr" without luck, i.e.,

/usr/libexec/virtiofsd --fd=3 -o source=/run/kata-containers/shared/sandboxes/ab0f9795e9f2823e75c70d01977e655a62d5f780896f43e4d6ea430ce3fb76d3/shared -o cache=always --syslog -o no_posix_lock -f -o xattr

Comment 10 Vivek Goyal 2021-02-08 13:50:08 UTC
We probably need to give virtiofsd CAP_SYS_ADMIN so that it can set security.capability xattr?

From "man xattr"

********
When no security module is loaded,
       all processes have read access to  extended  security  attributes,  and
       write  access is limited to processes that have the CAP_SYS_ADMIN capa‐
       bility.
*******

Or fallback to remapping xattr mechanism in virtiofsd.

Comment 11 Vivek Goyal 2021-02-08 13:53:35 UTC
Cai Qian, can you give cap_sys_admin to virtiofsd and try this again. Pass option "-o modcaps=+sys_admin" to virtiofsd to grant it cap_sys_admin.

Comment 12 Qian Cai 2021-02-08 14:16:30 UTC
It needs both "-o modcaps=+sys_admin -o xattr" to work correctly.

Comment 13 Vivek Goyal 2021-02-08 14:26:04 UTC
(In reply to Qian Cai from comment #12)
> It needs both "-o modcaps=+sys_admin -o xattr" to work correctly.

Yes, "-o xattr" is implied. If virtiofsd does not support xattr, then guest can not set any xattr on files in virtiofs.

Comment 16 Joseph Tingiris 2021-11-18 18:37:55 UTC
I'm seeing the same errors today, trying to install ubi8/ubi:8.5-200 in ocp 4.8.11 ... is there a work-around?

Comment 17 Joseph Tingiris 2022-01-10 18:54:15 UTC
any update on a resolution or work-around?

Comment 19 Daniel 2022-08-18 12:53:28 UTC
Hi @Joseph, I trust you are well.
I was investigating a solution for the same problem you had, and In my case, this did work.
https://access.redhat.com/solutions/6607631

securityContext:
  capabilities:
    add: 
    - "SETFCAP"

I hope this helps you.

Thanks

Comment 20 Jens Freimann 2022-09-21 08:22:19 UTC
It's not a kata bug and a workaround exists. Closing this.


Note You need to log in before you can comment on or make changes to this bug.