Fedora Account System
Red Hat Associate
Red Hat Customer
Also hit this issue in Fedora 33. The fix version is 2021.5. We may need a rebase in Fedora 33. +++ This bug was initially created as a clone of Bug #1959907 +++ Description of problem: When run rpm-ostree upgrade on RHEL 8.5 ostree system, rpm-ostreed crashed with core dump. Do not have this issue on RHEL 8.5 ostree system. [admin@vm-bios ~]$ journalctl -u rpm-ostreed -- Logs begin at Wed 2021-05-12 14:38:25 UTC, end at Wed 2021-05-12 15:08:45 UTC. -- May 12 14:54:26 vm-bios systemd[1]: Starting rpm-ostree System Management Daemon... May 12 14:54:26 vm-bios rpm-ostree[1074]: Reading config file '/etc/rpm-ostreed.conf' May 12 14:54:26 vm-bios systemd[1]: Started rpm-ostree System Management Daemon. May 12 14:54:26 vm-bios rpm-ostree[1074]: In idle state; will auto-exit in 60 seconds May 12 14:54:26 vm-bios rpm-ostree[1074]: client(id:cli dbus:1.23 unit:session-3.scope uid:0) added; new total=1 May 12 14:54:26 vm-bios rpm-ostree[1074]: Locked sysroot May 12 14:54:26 vm-bios rpm-ostree[1074]: Initiated txn Upgrade for client(id:cli dbus:1.23 unit:session-3.scope uid:0): /org/projectatomic/rpmostree1/edge_commit May 12 14:54:30 vm-bios rpm-ostree[1074]: libostree pull from 'edge-commit' for rhel/8/x86_64/edge complete security: GPG: disabled security: SIGN: disabled http: TLS non-delta: meta: 98 content: 66 transfer: secs: 4 size: 13.7 MB May 12 14:54:32 vm-bios rpm-ostree[1074]: Created new deployment /ostree/deploy/edge-commit/deploy/4107a6f83a76d40fb045cfcfbcfae6c4128840cd4d7c5018f8643ced399f9c88.0 May 12 14:54:32 vm-bios rpm-ostree[1074]: sanitycheck(/usr/bin/true) successful May 12 14:54:33 vm-bios rpm-ostree[1074]: ** May 12 14:54:33 vm-bios rpm-ostree[1074]: ERROR:src/lib/rpmostree-package.c:295:_rpm_ostree_package_list_for_commit: assertion failed (pkglist->len > 0): (0 > 0) May 12 14:54:33 vm-bios systemd[1]: rpm-ostreed.service: Main process exited, code=killed, status=6/ABRT May 12 14:54:33 vm-bios systemd[1]: rpm-ostreed.service: Failed with result 'signal'. [admin@vm-bios ~]$ coredumpctl info 1074 PID: 1074 (rpm-ostree) UID: 0 (root) GID: 0 (root) Signal: 6 (ABRT) Timestamp: Wed 2021-05-12 14:54:33 UTC (3min 3s ago) Command Line: /usr/bin/rpm-ostree start-daemon Executable: /usr/bin/rpm-ostree Control Group: / Slice: -.slice Boot ID: bf95678384534218b03d95e06417d478 Machine ID: 4e906e2d7c8b4cc78590f0196ce7fd85 Hostname: vm-bios Storage: /var/lib/systemd/coredump/core.rpm-ostree.0.bf95678384534218b03d95e06417d478.1074.1620831273000000.lz4 (inaccessible) Message: Process 1074 (rpm-ostree) of user 0 dumped core. Stack trace of thread 1077: #0 0x00007f27ce52337f raise (libc.so.6) #1 0x00007f27ce50ddb5 abort (libc.so.6) #2 0x00007f27cf9130c3 g_assertion_message.cold.15 (libglib-2.0.so.0) #3 0x00007f27cf96c4ba g_assertion_message_cmpnum (libglib-2.0.so.0) #4 0x00007f27d1ca56cd _rpm_ostree_package_list_for_commit (librpmostree-1.so.1) #5 0x00007f27d1ca4de2 rpm_ostree_db_diff_ext (librpmostree-1.so.1) #6 0x00005561c976b4a0 rpm_diff_add_db_diff.constprop.20 (rpm-ostree) #7 0x00005561c976d0a4 rpmostreed_update_generate_variant (rpm-ostree) #8 0x00005561c9778c8a generate_update_variant (rpm-ostree) #9 0x00005561c977d1e3 deploy_transaction_execute (rpm-ostree) #10 0x00005561c976eacd transaction_execute_thread (rpm-ostree) #11 0x00007f27cfef7367 g_task_thread_pool_thread (libgio-2.0.so.0) #12 0x00007f27cf96d943 g_thread_pool_thread_proxy (libglib-2.0.so.0) #13 0x00007f27cf96cf3a g_thread_proxy (libglib-2.0.so.0) #14 0x00007f27ce8b914a start_thread (libpthread.so.0) #15 0x00007f27ce5e8dc3 __clone (libc.so.6) Stack trace of thread 1074: #0 0x00007f27ce5dda41 __poll (libc.so.6) #1 0x00007f27cf944bf6 g_main_context_iterate.isra.21 (libglib-2.0.so.0) #2 0x00007f27cf944d20 g_main_context_iteration (libglib-2.0.so.0) #3 0x00005561c976745c rpmostreed_daemon_run_until_idle_exit (rpm-ostree) #4 0x00005561c972d8d3 rpmostree_builtin_start_daemon (rpm-ostree) #5 0x00005561c9716e0a main (rpm-ostree) #6 0x00007f27ce50f493 __libc_start_main (libc.so.6) #7 0x00005561c971717e _start (rpm-ostree) Stack trace of thread 1075: #0 0x00007f27ce5dda41 __poll (libc.so.6) #1 0x00007f27cf944bf6 g_main_context_iterate.isra.21 (libglib-2.0.so.0) #2 0x00007f27cf944d20 g_main_context_iteration (libglib-2.0.so.0) #3 0x00007f27cf944d71 glib_worker_main (libglib-2.0.so.0) #4 0x00007f27cf96cf3a g_thread_proxy (libglib-2.0.so.0) #5 0x00007f27ce8b914a start_thread (libpthread.so.0) #6 0x00007f27ce5e8dc3 __clone (libc.so.6) Stack trace of thread 1076: #0 0x00007f27ce5dda41 __poll (libc.so.6) #1 0x00007f27cf944bf6 g_main_context_iterate.isra.21 (libglib-2.0.so.0) #2 0x00007f27cf944fb2 g_main_loop_run (libglib-2.0.so.0) #3 0x00007f27cff3a5aa gdbus_shared_thread_func (libgio-2.0.so.0) #4 0x00007f27cf96cf3a g_thread_proxy (libglib-2.0.so.0) #5 0x00007f27ce8b914a start_thread (libpthread.so.0) #6 0x00007f27ce5e8dc3 __clone (libc.so.6) Stack trace of thread 1088: #0 0x00007f27ce5e352d syscall (libc.so.6) #1 0x00007f27cf98bdae g_cond_wait_until (libglib-2.0.so.0) #2 0x00007f27cf917131 g_async_queue_pop_intern_unlocked (libglib-2.0.so.0) #3 0x00007f27cf917720 g_async_queue_timeout_pop (libglib-2.0.so.0) #4 0x00007f27cf96d9e1 g_thread_pool_thread_proxy (libglib-2.0.so.0) #5 0x00007f27cf96cf3a g_thread_proxy (libglib-2.0.so.0) #6 0x00007f27ce8b914a start_thread (libpthread.so.0) #7 0x00007f27ce5e8dc3 __clone (libc.so.6) Version-Release number of selected component (if applicable): rpm-ostree-libs-2020.7-4.el8.x86_64 ostree-2020.7-4.el8.x86_64 ostree-libs-2020.7-4.el8.x86_64 rpm-ostree-2020.7-4.el8.x86_64 libcurl-7.61.1-20.el8.x86_64 curl-7.61.1-20.el8.x86_64 How reproducible: Steps to Reproduce: 1. Build an RHEL for Edge image (edge-commit) with image builder on RHEL 8.5 2. Setup a web servera as ostree repo 3. Extract edge-commit(tar) into /var/www/html/ 4. Install ostree commit with kickstart on VM 5. Build an RHEL for Edge upgrade commit 6. Extract upgrade commit, pull and summary ostree repo 7. Run rpm-ostree upgrade on VM Actual results: ⠠ Scanning metadata: 2310 Scanning metadata: 2310... done Staging deployment... done error: Bus owner changed, aborting. This likely means the daemon crashed; check logs with `journalctl -xe`. Expected results: upgrade successful Additional info: --- Additional comment from Jonathan Lebon on 2021-05-12 15:50:02 UTC --- Since it's the same root issue, let's dedupe these two.
Given https://github.com/osbuild/osbuild-composer/pull/1498, it sounds like we can close this, right?
I think we are good on this now yes.
Note that the bug is still there, so building Fedora IoT images with Image Builder is still broken. We had to disable our test for it as fixing it is out of our control and our development was therefore blocked. Would love to see it fixed so we can reenable the tests, and most importantly restore the functionality.
Hm, OK. Any particular reason to still care about Fedora 33 instead of 34? Anyways I'm building the new rpm-ostree for 33 too.
Thanks Colin! We try to make sure Image Builder works in Fedora until it is EOL, which is why we still test Fedora 33 in our CI (and release updates into, which is debatable, but makes our lives simpler). Ideally these tests would be part of Fedora gating, so we would not only catch regressions in other projects in our upstream CI, but we are not quite there yet.
FEDORA-2021-35a30ae214 has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2021-35a30ae214
FEDORA-2021-35a30ae214 has been pushed to the Fedora 33 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-35a30ae214` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-35a30ae214 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Verifed on rpm-ostree-2021.6-2.fc33.x86_64 and rpm-ostree-libs-2021.6-2.fc33.x86_64.
FEDORA-2021-35a30ae214 has been pushed to the Fedora 33 stable repository. If problem still persists, please make note of it in this bug report.