Bug 1822753

Summary: ostree installs fail with 'Commit metadata for runtime/org.fedoraproject.Platform/x86_64/f32 not matching expected metadata'
Product: [Fedora] Fedora Reporter: Adam Williamson <awilliam>
Component: anacondaAssignee: Kalev Lember <klember>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: rawhideCC: anaconda-maint-list, dustymabe, jkonecny, jonathan, kellin, klember, tpopela, vanmeeuwen+fedora, vponcova, walters, wwoods
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard: openqa
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-17 05:07:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Adam Williamson 2020-04-09 18:55:49 UTC
Since the crash from https://bugzilla.redhat.com/show_bug.cgi?id=1820418 was fixed, Rawhide ostree installs are failing with a different error. The UI shows "Failed to install flatpaks: flatpak-error-quark: Aborted due to failure (4)". anaconda.log shows:

20:17:59,680 INF payload.rpmostreepayload: executing ostreesetup=<pykickstart.commands.ostreesetup.F21_OSTreeSetup object at 0x7ffaa5198ca0>
20:28:01,076 INF payload.rpmostreepayload: ostree pull: 11773 metadata, 111414 content objects imported
20:28:02,496 INF payload.rpmostreepayload: ostree admin deploy starting
20:28:30,411 INF payload.rpmostreepayload: ostree admin deploy complete
20:28:31,260 INF payload.rpmostreepayload: Copying bootloader data: grub2
20:28:32,774 DBG payload.flatpak: Cleaning up flatpak repository
20:28:32,812 DBG payload.flatpak: Removing flatpak repository /var/tmp/anaconda-flatpak-temp
20:28:32,821 DBG payload.flatpak: Configure flatpak for path /mnt/sysroot/var/lib/flatpak
20:28:32,949 DBG payload.flatpak: Flatpak operation: install of ref runtime/org.fedoraproject.Platform/x86_64/f32 state started
20:29:09,632 WRN misc: /usr/lib64/python3.8/site-packages/pyanaconda/payload/flatpak.py:177: Warning: g_value_get_flags: assertion 'G_VALUE_HOLDS_FLAGS (value)' failed
  self._transaction.run()

20:29:09,649 DBG payload.flatpak: Flatpak operation: install of ref runtime/org.fedoraproject.Platform/x86_64/f32 state failed
20:29:09,650 ERR payload.flatpak: Flatpak installation failed with message: 'Commit metadata for runtime/org.fedoraproject.Platform/x86_64/f32 not matching expected metadata' -- error is fatal False
20:29:09,656 ERR payload.rpmostreepayload: Failed to install flatpaks: flatpak-error-quark: Aborted due to failure (4)

Comment 1 Kalev Lember 2020-04-21 08:53:25 UTC
I looked into it in the morning and https://pagure.io/flatpak-module-tools/pull-request/9 should hopefully fix this

Comment 2 Adam Williamson 2020-05-08 14:40:05 UTC
so the pull request was merged two weeks ago, but this is still broken:

https://openqa.fedoraproject.org/tests/594284#step/_do_install_and_reboot/47

Log messages look just the same:

13:41:08,529 DBG payload.flatpak: Flatpak operation: install of ref runtime/org.fedoraproject.Platform/x86_64/f32 state started
13:42:01,613 WRN misc: /usr/lib64/python3.8/site-packages/pyanaconda/payload/flatpak.py:177: Warning: g_value_get_flags: assertion 'G_VALUE_HOLDS_FLAGS (value)' failed
  self._transaction.run()

13:42:01,633 DBG payload.flatpak: Flatpak operation: install of ref runtime/org.fedoraproject.Platform/x86_64/f32 state failed
13:42:01,633 ERR payload.flatpak: Flatpak installation failed with message: 'Commit metadata for runtime/org.fedoraproject.Platform/x86_64/f32 not matching expected metadata' -- error is fatal False
13:42:01,634 ERR payload.rpmostreepayload: Failed to install flatpaks: flatpak-error-quark: Aborted due to failure (4)

What's going on? Fix didn't work? Fix isn't in effect yet?

Comment 3 Kalev Lember 2020-05-10 14:51:35 UTC
We need the fix deployed on the builders as well. I was first waiting for the infra freeze to end and then https://pagure.io/flatpak-module-tools/pull-request/10 to be merged so that we can deploy both fixes at the same time.

I'll make sure this gets some traction next week.

Comment 4 Fedora Update System 2020-05-11 22:31:04 UTC
FEDORA-2020-d7afeb1476 has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-d7afeb1476

Comment 5 Fedora Update System 2020-05-11 22:31:06 UTC
FEDORA-2020-5ff86c2609 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-5ff86c2609

Comment 6 Fedora Update System 2020-05-11 22:31:07 UTC
FEDORA-EPEL-2020-eb770e14c1 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-eb770e14c1

Comment 7 Fedora Update System 2020-05-11 22:31:08 UTC
FEDORA-2020-6edcc178fe has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-6edcc178fe

Comment 8 Kalev Lember 2020-05-11 22:35:36 UTC
OK, Fedora builds done and infra request is in to get the OSBS buildroot updated: https://pagure.io/fedora-infrastructure/issue/8907

Comment 9 Fedora Update System 2020-05-12 05:25:00 UTC
FEDORA-2020-6edcc178fe has been pushed to the Fedora 30 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-6edcc178fe`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-6edcc178fe

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 10 Fedora Update System 2020-05-12 06:44:09 UTC
FEDORA-2020-5ff86c2609 has been pushed to the Fedora 31 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-5ff86c2609`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-5ff86c2609

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 11 Fedora Update System 2020-05-12 07:19:17 UTC
FEDORA-2020-d7afeb1476 has been pushed to the Fedora 32 testing repository.
In short time you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2020-d7afeb1476`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-d7afeb1476

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 12 Fedora Update System 2020-05-12 07:59:33 UTC
FEDORA-EPEL-2020-eb770e14c1 has been pushed to the Fedora EPEL 7 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-eb770e14c1

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 13 Fedora Update System 2020-05-14 02:28:51 UTC
FEDORA-2020-5ff86c2609 has been pushed to the Fedora 31 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 14 Fedora Update System 2020-05-14 02:36:02 UTC
FEDORA-2020-d7afeb1476 has been pushed to the Fedora 32 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 15 Fedora Update System 2020-05-14 09:33:11 UTC
FEDORA-EPEL-2020-eb770e14c1 has been pushed to the Fedora EPEL 7 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 16 Fedora Update System 2020-05-14 09:52:24 UTC
FEDORA-2020-6edcc178fe has been pushed to the Fedora 30 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 17 Kalev Lember 2020-05-14 11:47:24 UTC
Still working on getting the flatpak-module-tools deployed in infra. See https://pagure.io/fedora-infrastructure/issue/8907#comment-652257 for the latest status on this.

After that is done, we should be able to rebuild the runtime.

Comment 18 Kalev Lember 2020-05-14 13:54:31 UTC
OK, the fix is now deployed in infra and I'm working on rebuilding the runtimes. I'll link all of the rebuilds here (but only mark the F32 one as closing this bug as this is the one that Anaconda tries to install).

Comment 19 Fedora Update System 2020-05-14 13:55:56 UTC
FEDORA-FLATPAK-2020-350bbafc22 has been submitted as an update to Fedora 30 Flatpaks. https://bodhi.fedoraproject.org/updates/FEDORA-FLATPAK-2020-350bbafc22

Comment 20 Fedora Update System 2020-05-14 14:50:05 UTC
FEDORA-FLATPAK-2020-35790ab903 has been submitted as an update to Fedora 31 Flatpaks. https://bodhi.fedoraproject.org/updates/FEDORA-FLATPAK-2020-35790ab903

Comment 21 Fedora Update System 2020-05-14 15:49:46 UTC
FEDORA-FLATPAK-2020-f273904664 has been submitted as an update to Fedora 32 Flatpaks. https://bodhi.fedoraproject.org/updates/FEDORA-FLATPAK-2020-f273904664

Comment 22 Fedora Update System 2020-05-14 15:49:47 UTC
FEDORA-FLATPAK-2020-f273904664 has been submitted as an update to Fedora 32 Flatpaks. https://bodhi.fedoraproject.org/updates/FEDORA-FLATPAK-2020-f273904664

Comment 23 Fedora Update System 2020-05-15 05:50:20 UTC
FEDORA-FLATPAK-2020-f273904664 has been pushed to the Fedora 32 Flatpaks testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-FLATPAK-2020-f273904664

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 24 Fedora Update System 2020-05-15 05:52:01 UTC
FEDORA-FLATPAK-2020-350bbafc22 has been pushed to the Fedora 30 Flatpaks testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-FLATPAK-2020-350bbafc22

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 25 Fedora Update System 2020-05-15 05:53:48 UTC
FEDORA-FLATPAK-2020-35790ab903 has been pushed to the Fedora 31 Flatpaks testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-FLATPAK-2020-35790ab903

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 26 Fedora Update System 2020-05-17 05:07:38 UTC
FEDORA-FLATPAK-2020-f273904664 has been pushed to the Fedora 32 Flatpaks stable repository.
If problem still persists, please make note of it in this bug report.

Comment 27 Fedora Update System 2020-05-17 05:08:50 UTC
FEDORA-FLATPAK-2020-350bbafc22 has been pushed to the Fedora 30 Flatpaks stable repository.
If problem still persists, please make note of it in this bug report.

Comment 28 Fedora Update System 2020-05-17 05:10:17 UTC
FEDORA-FLATPAK-2020-35790ab903 has been pushed to the Fedora 31 Flatpaks stable repository.
If problem still persists, please make note of it in this bug report.

Comment 29 Jiri Konecny 2020-05-18 16:55:26 UTC
I've tested it and seems to be working fine. Even though I wasn't able to check the installed system because the installation failed on bootloader later in the process.

Comment 30 Kalev Lember 2020-05-18 17:01:04 UTC
OK, great! Thanks for testing it.