+++ This bug was initially created as a clone of Bug #1905430 +++ Description of problem: Noticed that MCO 4.6 e2e-gcp-op ci failing because of extensions test failure. After analyzing logs and running it locally, it seems usbguard doesn't have required dependency available. # rpm-ostree install usbguard Checking out tree d9bad68... done Enabled rpm-md repositories: coreos-extensions rpm-md repo 'coreos-extensions' (cached); generated: 2020-12-02T02:33:44Z Importing rpm-md... done Resolving dependencies... done error: Could not depsolve transaction; 1 problem detected: Problem: conflicting requests - nothing provides libprotobuf.so.15()(64bit) needed by usbguard-0.7.4-4.el8.x86_64 Although, I have noticed that dependencies shipped in m-o-c extensions/dependencies/ dir has protobuf-3.6.1-4.el8ost.x86_64.rpm but this package provides incompatible lib libprotobuf.so.17 We would need to tag in correct dpes or update usbguard package. Version-Release number of selected component (if applicable): OCP 4.6 How reproducible: Always in recent 4.6 MCO gcp-op tes. recnt job - https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_machine-config-operator/2289/pull-ci-openshift-machine-config-operator-release-4.6-e2e-gcp-op/1335894493852864512 --- Additional comment from Sinny Kumari on 2020-12-08 10:51:28 UTC --- Setting priority to high as it impacts MCO ci test and will block any PR merge in 4.6 --- Additional comment from Micah Abbott on 2020-12-08 14:50:39 UTC --- The RHAOS 4.6 repo has a newer version of `protobuf` tagged into it, which is being selected when we download the extensions and dependencies. We recently split the download of extension and dependencies, so we are getting different behavior. If I do `dnf install usbguard` on a RHEL8 system with the same repos enabled that we used to build RHCOS, I get the correct version of `protobuf` ``` $ sudo dnf install usbguard Updating Subscription Management repositories. Unable to read consumer identity This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. Repository 'art-rhaos-4.6' is missing name in configuration, using id. Repository 'rhel-8-baseos' is missing name in configuration, using id. Repository 'rhel-8-appstream' is missing name in configuration, using id. Repository 'rhel-8-nfv' is missing name in configuration, using id. art-rhaos-4.6 4.6 MB/s | 2.7 MB 00:00 Last metadata expiration check: 0:00:01 ago on Tue 08 Dec 2020 09:44:23 AM EST. Dependencies resolved. ============================================================================================================================================================================================================================================================================================================================== Package Architecture Version Repository Size ============================================================================================================================================================================================================================================================================================================================== Installing: usbguard x86_64 0.7.4-4.el8 rhel-8-appstream 477 k Installing dependencies: libqb x86_64 1.0.3-10.el8 rhel-8-baseos 113 k protobuf x86_64 3.5.0-7.el8 rhel-8-appstream 895 k Transaction Summary ============================================================================================================================================================================================================================================================================================================================== Install 3 Packages ``` If I just do `dnf install protobuf`, I get the newer version that is incompatible. ``` $ sudo dnf install protobuf Updating Subscription Management repositories. Unable to read consumer identity This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register. Repository 'art-rhaos-4.6' is missing name in configuration, using id. Repository 'rhel-8-baseos' is missing name in configuration, using id. Repository 'rhel-8-appstream' is missing name in configuration, using id. Repository 'rhel-8-nfv' is missing name in configuration, using id. Last metadata expiration check: 0:00:23 ago on Tue 08 Dec 2020 09:44:23 AM EST. Dependencies resolved. ============================================================================================================================================================================================================================================================================================================================== Package Architecture Version Repository Size ============================================================================================================================================================================================================================================================================================================================== Installing: protobuf x86_64 3.6.1-4.el8ost art-rhaos-4.6 915 k Installing dependencies: emacs-filesystem noarch 1:26.1-5.el8 rhel-8-baseos 69 k Transaction Summary ============================================================================================================================================================================================================================================================================================================================== Install 2 Packages ``` We should be able to exclude `protobuf` from the RHAOS 4.6 repo definition, which should address this.
https://gitlab.cee.redhat.com/coreos/redhat-coreos/-/merge_requests/1193 This MR has already merged, we should see RHCOS 4.6 builds using it soon.
Thank you Micah for fixing the issue with priority. MCO 4.6 e2e-gcp-op test is green now!
Moving this to MODIFIED so that when the next ART nightly is built it should go ON_QA without further action.
$ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.6.0-0.nightly-2020-12-13-230909 True False 2m38s Cluster version is 4.6.0-0.nightly-2020-12-13-230909 $ oc get mc NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE 00-master eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 25m 00-worker eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 25m 01-master-container-runtime eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 25m 01-master-kubelet eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 25m 01-worker-container-runtime eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 25m 01-worker-kubelet eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 25m 99-master-generated-registries eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 25m 99-master-ssh 3.1.0 35m 99-worker-generated-registries eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 25m 99-worker-ssh 3.1.0 35m rendered-master-abf49d28f54aca1f839f586d03317e8b eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 25m rendered-worker-f11107cabe3fba649a22418182377786 eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 25m $ cp ../extensions-usbguard.yaml . $ cat extensions-usbguard.yaml apiVersion: machineconfiguration.openshift.io/v1 kind: MachineConfig metadata: labels: machineconfiguration.openshift.io/role: worker name: worker-extensions-usbguard spec: config: ignition: version: 3.1.0 extensions: - usbguard $ oc get nodes NAME STATUS ROLES AGE VERSION ip-10-0-146-248.us-west-2.compute.internal Ready worker 19m v1.19.0+7070803 ip-10-0-151-204.us-west-2.compute.internal Ready master 27m v1.19.0+7070803 ip-10-0-171-205.us-west-2.compute.internal Ready master 28m v1.19.0+7070803 ip-10-0-177-134.us-west-2.compute.internal Ready worker 19m v1.19.0+7070803 ip-10-0-207-32.us-west-2.compute.internal Ready worker 19m v1.19.0+7070803 ip-10-0-213-46.us-west-2.compute.internal Ready master 27m v1.19.0+7070803 $ oc debug node/ip-10-0-146-248.us-west-2.compute.internal -- chroot /host rpm -q usbguard Starting pod/ip-10-0-146-248us-west-2computeinternal-debug ... To use host binaries, run `chroot /host` package usbguard is not installed Removing debug pod ... $ oc create -f extensions-usbguard.yaml machineconfig.machineconfiguration.openshift.io/worker-extensions-usbguard created $ oc get mc NAME GENERATEDBYCONTROLLER IGNITIONVERSION AGE 00-master eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 26m 00-worker eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 26m 01-master-container-runtime eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 26m 01-master-kubelet eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 26m 01-worker-container-runtime eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 26m 01-worker-kubelet eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 26m 99-master-generated-registries eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 26m 99-master-ssh 3.1.0 36m 99-worker-generated-registries eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 26m 99-worker-ssh 3.1.0 36m rendered-master-abf49d28f54aca1f839f586d03317e8b eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 26m rendered-worker-4bf1bdb7cc3b8c5cb38d8ae5235eb47c eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 0s rendered-worker-f11107cabe3fba649a22418182377786 eb9778355a9020673e8ce9aee092cb98d80cde5e 3.1.0 26m worker-extensions-usbguard 3.1.0 5s $ oc get mcp/worker NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE worker rendered-worker-f11107cabe3fba649a22418182377786 True False False 3 3 3 0 27m $ oc get mcp/worker NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE worker rendered-worker-f11107cabe3fba649a22418182377786 False True False 3 0 0 0 28m $ watch oc get mcp/worker $ oc debug node/ip-10-0-146-248.us-west-2.compute.internal -- chroot /host rpm -q usbguard Starting pod/ip-10-0-146-248us-west-2computeinternal-debug ... To use host binaries, run `chroot /host` usbguard-0.7.4-4.el8.x86_64 Removing debug pod ...
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: OpenShift Container Platform 4.6.9 security and bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5614