Bug 1862568

Summary:	sysroot not remounted rw
Product:	[Fedora] Fedora	Reporter:	Paul Whalen <pwhalen>
Component:	ostree	Assignee:	Colin Walters <walters>
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	rawhide	CC:	dustymabe, jonathan, miabbott, rfairley, walters
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ostree-2020.4-4.fc33 ostree-2020.4-4.fc32	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-08-06 04:02:46 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1269538

Description Paul Whalen 2020-07-31 18:05:37 UTC

Description of problem:
Booting the latest IoT compose (Fedora-IoT-33-20200731.0), /sysroot is not getting remounted rw, as a result many services fail. 


Jul 31 12:44:15 localhost.localdomain systemd[1]: Starting OSTree Remount OS/ Bind Mounts...
Jul 31 12:44:15 localhost.localdomain ostree-remount[632]: Remounted rw: /var
Jul 31 12:44:15 localhost.localdomain systemd[1]: Finished OSTree Remount OS/ Bind Mounts.

mount | grep sysroot
/dev/mapper/fedora--iot_fedora-root on /sysroot type ext4 (ro,relatime,seclabel)

Remounting rw and restarting the various services works. 


Version-Release number of selected component (if applicable):
ostree-2020.4-2.fc33.x86_64

How reproducible:
Every time

Comment 1 Colin Walters 2020-07-31 18:25:58 UTC

Hmm, this must be fallout from https://github.com/ostreedev/ostree/pull/2113 - but are you enabling the sysroot=readonly flag or not?

See also https://github.com/coreos/fedora-coreos-tracker/issues/589 but I don't think you're running from read-only storage, or are you?

Comment 2 Paul Whalen 2020-07-31 19:23:31 UTC

(In reply to Colin Walters from comment #1)
> Hmm, this must be fallout from https://github.com/ostreedev/ostree/pull/2113
> - but are you enabling the sysroot=readonly flag or not?

Where would that be enabled? Looking for related messages

Jul 31 13:33:34 fedora ostree-prepare-root[486]: sysroot configured read-only: 0, currently writable: 0


> See also https://github.com/coreos/fedora-coreos-tracker/issues/589 but I
> don't think you're running from read-only storage, or are you?

No, this is not read-only storage.

Comment 3 Colin Walters 2020-08-01 17:25:42 UTC

OK one thing I had to triple check is that Silverblue works, and it certainly appears to.

Unfortunately we're not testing non-CoreOS systems much in OSTree CI right now (need to fix that).

I'd done manual testing for this patch for Silverblue because I knew there was a risk, so something
else is going on with IoT.

Oh but wait... /sysroot *is* readonly now even without it being configured on, which is clearly broken in that patch.
It's strange this doesn't break SB but does break IoT, nevertheless working on a fix now.

Comment 4 Fedora Update System 2020-08-02 17:15:07 UTC

FEDORA-2020-e4b4702285 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-e4b4702285

Comment 5 Fedora Update System 2020-08-04 01:06:36 UTC

FEDORA-2020-e4b4702285 has been pushed to the Fedora 32 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-e4b4702285`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-e4b4702285

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 6 Fedora Update System 2020-08-06 04:02:46 UTC

FEDORA-2020-e4b4702285 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.