Bug 1890361

Summary:	s390x: Generate new ostree rpm with fix for rootfs immutability
Product:	OpenShift Container Platform	Reporter:	Prashanth Sundararaman <psundara>
Component:	RHCOS	Assignee:	Colin Walters <walters>
Status:	CLOSED ERRATA	QA Contact:	Michael Nguyen <mnguyen>
Severity:	medium	Docs Contact:
Priority:	high
Version:	4.5	CC:	bbreard, imcleod, jligon, miabbott, nstielau
Target Milestone:	---
Target Release:	4.7.0
Hardware:	s390x
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:
Clones:	1890667 (view as bug list)		Environment:
Last Closed:	2021-02-24 15:27:41 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1890667, 1890668

Description Prashanth Sundararaman 2020-10-22 01:28:44 UTC

Recently the s390x 4.5 RHCOS pipeline started failing because the ostree version of fedora32 which is used in the cosa container had the fix for 

https://bugzilla.redhat.com/show_bug.cgi?id=1867601

Because of this - the root file system is immutable - which is good, but the ostree version in RHCOS 4.5 is an older version which does not have the fix for the above BZ. This causes rollbacks and upgrades to fail.

For now, the coreos-assembler is patched to remove the immutable bit, but it would be good if ostree is bumped with the fix for RHCOS 4.5.

Comment 1 Prashanth Sundararaman 2020-10-22 01:30:18 UTC

This is the said fix that would need to be backported:

https://github.com/ostreedev/ostree/pull/2179

Comment 2 Micah Abbott 2020-10-22 16:54:49 UTC

Targeting 4.7 (though the fix should already be present there and in 4.6); we will probably need to clone this for 4.5.z

Comment 3 Micah Abbott 2020-10-25 18:36:32 UTC

This was fixed in ostree-2020.5-4.el8 which is already included in RHCOS 4.7.

Comment 5 Michael Nguyen 2020-10-27 00:59:03 UTC

Verified on 4.7.0-0.nightly-2020-10-26-124513.  ostree version is ostree-2020.5-4

$ oc get nodes
NAME                                         STATUS   ROLES    AGE     VERSION
ip-10-0-136-111.us-west-2.compute.internal   Ready    worker   5h12m   v1.19.0+e67f5dc
ip-10-0-138-139.us-west-2.compute.internal   Ready    master   5h17m   v1.19.0+e67f5dc
ip-10-0-166-15.us-west-2.compute.internal    Ready    worker   5h12m   v1.19.0+e67f5dc
ip-10-0-187-209.us-west-2.compute.internal   Ready    master   5h17m   v1.19.0+e67f5dc
ip-10-0-218-114.us-west-2.compute.internal   Ready    worker   5h7m    v1.19.0+e67f5dc
ip-10-0-219-131.us-west-2.compute.internal   Ready    master   5h18m   v1.19.0+e67f5dc

$ oc get pods
No resources found.
$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.7.0-0.nightly-2020-10-26-124513   True        False         5h4m    Cluster version is 4.7.0-0.nightly-2020-10-26-124513
$ oc debug node/ip-10-0-136-111.us-west-2.compute.internal -- chroot /host rpm -q ostree
Starting pod/ip-10-0-136-111us-west-2computeinternal-debug ...
To use host binaries, run `chroot /host`
ostree-2020.5-4.el8.x86_64

Removing debug pod ...
$ oc debug node/ip-10-0-136-111.us-west-2.compute.internal -- chroot /host rpm-ostree status
Starting pod/ip-10-0-136-111us-west-2computeinternal-debug ...
To use host binaries, run `chroot /host`
State: idle
Deployments:
* pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3dc9cdce1901a5f5ca6cbab5e435c48d924eb813a01f5c48e49dd91c1d2e340c
              CustomOrigin: Managed by machine-config-operator
                   Version: 47.82.202010241242-0 (2020-10-24T12:46:24Z)

  ostree://6dff002defa66e3f9f53b57da7dc3eef37db39762655c00c13b977d72516649c
                   Version: 47.82.202010211043-0 (2020-10-21T10:47:13Z)

Removing debug pod ...

Comment 8 errata-xmlrpc 2021-02-24 15:27:41 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633