Bug 1648113

Description of problem:
`sudo rpm-ostree install <package>`, where <package> can be many things, but good examples are `python3-pip`, `kernel-modules-extra` and `buildah`, usually fails to complete. See here: `http://bit.ly/2Da2eYT` for the bug report in rpm-ostee, which didn't seem to get to the bottom of the issue.  I don't believe that my setup is particularly special and I've seen failures on a number of Raspberry Pi 3s.


Version-Release number of selected component (if applicable):
 rpm-ostree status
State: idle
AutomaticUpdates: disabled
Deployments:
● ostree://fedora-iot:fedora/29/aarch64/iot
                   Version: 29.20181106.0 (2018-11-06T11:14:51Z)

rpm-ostree-2018.9-3.fc29.aarch64

How reproducible:
> 50%

Steps to Reproduce:
1. start with clean fedora iot on a raspberry pi
2. `sudo rpm-ostree install kernel-modules-extra`
3. often fails to complete

Actual results:

Here's a success and a fail:
`
Nov 05 12:04:09 localhost.localdomain rpm-ostree[1309]: client(id:cli dbus:1.38 unit:session-5.scope uid:0) vanished; remaining=0
Nov 05 12:04:09 localhost.localdomain rpm-ostree[1309]: In idle state; will auto-exit in 64 seconds
Nov 05 12:05:13 localhost.localdomain rpm-ostree[1309]: In idle state; will auto-exit in 63 seconds
Nov 05 12:05:32 localhost.localdomain systemd[1]: Starting RPM-OSTree System Man Daemon...
Nov 05 12:05:32 localhost.localdomain rpm-ostree[1383]: Reading config file '/etc/rpm-ostreed.conf'
Nov 05 12:05:32 localhost.localdomain rpm-ostree[1383]: In idle state; will auto-exit in 64 seconds
Nov 05 12:05:32 localhost.localdomain systemd[1]: Started RPM-OSTree System Management Daemon.
Nov 05 12:05:32 localhost.localdomain rpm-ostree[1383]: client(id:cli dbus:1.46 unit:session-5.scope uid:0) added; new total=1
Nov 05 12:05:32 localhost.localdomain rpm-ostree[1383]: Initiated txn PkgChange for client(id:cli dbus:1.46 unit:session-5.scope uid:0): /org/projectatomic/rpmostree1/fedora_iot
Nov 05 12:10:49 localhost.localdomain rpm-ostree[1383]: Preparing pkg txn; enabled repos: ['fedora-modular', 'updates-modular', 'updates', 'fedora'] solvables: 60909
Nov 05 12:11:42 localhost.localdomain rpm-ostree[1383]: Imported 1 pkg
Nov 05 12:11:42 localhost.localdomain rpm-ostree[1383]: No files matched %transfiletriggerin(lib) for glibc-common
Nov 05 12:11:42 localhost.localdomain rpm-ostree[1383]: No files matched %transfiletriggerin(lib64) for glibc-common
Nov 05 12:11:44 localhost.localdomain rpm-ostree[1383]: Executed %transfiletriggerin(glibc-common) for lib, lib64, usr/lib, usr/lib64 in 1576ms; 20746 matched files
Nov 05 12:11:44 localhost.localdomain rpm-ostree[1383]: No files matched %transfiletriggerin(usr/lib64/gio/modules) for glib2
Nov 05 12:11:44 localhost.localdomain rpm-ostree[1383]: No files matched %transfiletriggerin(usr/share/glib-2.0/schemas) for glib2
Nov 05 12:11:44 localhost.localdomain rpm-ostree[1383]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/hwdb.d in 367ms; 18 matched files
Nov 05 12:11:45 localhost.localdomain rpm-ostree[1383]: Executed %transfiletriggerin(systemd-udev) for usr/lib/udev/rules.d in 379ms; 40 matched files
Nov 05 12:11:45 localhost.localdomain rpm-ostree[1383]: Executed %transfiletriggerin(info) for usr/share/info in 706ms; 2 matched files
Nov 05 12:11:46 localhost.localdomain rpm-ostree[1383]: sanitycheck(/usr/bin/true) successful
Nov 05 12:12:44 localhost.localdomain rpm-ostree[1383]: Wrote commit: b172c2003fe5109d56d2ad39f84b137416fe97863acf09a90e0528e64f735b7f; New objects: meta:14 content:10 totaling 15.8 MB)
Nov 05 12:14:19 localhost.localdomain rpm-ostree[1383]: Txn PkgChange on /org/projectatomic/rpmostree1/fedora_iot successful
Nov 05 12:14:20 localhost.localdomain rpm-ostree[1383]: client(id:cli dbus:1.46 unit:session-5.scope uid:0) vanished; remaining=0
Nov 05 12:14:20 localhost.localdomain rpm-ostree[1383]: In idle state; will auto-exit in 63 seconds
Nov 05 12:14:37 localhost.localdomain rpm-ostree[1383]: client(id:cli dbus:1.50 unit:session-5.scope uid:0) added; new total=1
Nov 05 12:14:38 localhost.localdomain rpm-ostree[1383]: Initiated txn PkgChange for client(id:cli dbus:1.50 unit:session-5.scope uid:0): /org/projectatomic/rpmostree1/fedora_iot
Nov 05 12:14:58 localhost.localdomain rpm-ostree[1383]: Preparing pkg txn; enabled repos: ['fedora-modular', 'updates-modular', 'updates', 'fedora'] solvables: 60909
Nov 05 12:15:36 localhost.localdomain rpm-ostree[1383]: Imported 1 pkg
Nov 05 12:15:38 localhost.localdomain rpm-ostree(kernel-modules-extra.post)[1383]: depmod: ERROR: openat(3, extra, O_RDONLY): Transport endpoint is not connected
Nov 05 12:15:38 localhost.localdomain rpm-ostree(kernel-modules-extra.post)[1383]: depmod: ERROR: openat(3, vdso, O_RDONLY): Transport endpoint is not connected
Nov 05 12:15:38 localhost.localdomain rpm-ostree(kernel-modules-extra.post)[1383]: depmod: ERROR: openat(3, kernel, O_RDONLY): Transport endpoint is not connected
Nov 05 12:15:38 localhost.localdomain rpm-ostree(kernel-modules-extra.post)[1383]: depmod: WARNING: could not open /lib/modules/4.18.16-300.fc29.aarch64/modules.order: Transport endpoint is not connected
Nov 05 12:15:38 localhost.localdomain rpm-ostree(kernel-modules-extra.post)[1383]: depmod: FATAL: could not open directory /lib/modules/4.18.16-300.fc29.aarch64: Transport endpoint is not connected
Nov 05 12:15:51 localhost.localdomain rpm-ostree[1383]: Txn PkgChange on /org/projectatomic/rpmostree1/fedora_iot failed: Running %post for kernel-modules-extra: Executing bwrap(/bin/sh): Child process killed by signal 7; run `journalctl -t 'rpm-ostree(kernel-modules-extra.post)'` for more information
Nov 05 12:15:51 localhost.localdomain rpm-ostree[1383]: client(id:cli dbus:1.50 unit:session-5.scope uid:0) vanished; remaining=0
Nov 05 12:15:51 localhost.localdomain rpm-ostree[1383]: In idle state; will auto-exit in 60 seconds
Nov 05 12:16:51 localhost.localdomain rpm-ostree[1383]: In idle state; will auto-exit in 63 seconds
Nov 05 13:58:01 localhost.localdomain systemd[1]: Starting RPM-OSTree System Management Daemon...
Nov 05 13:58:01 localhost.localdomain rpm-ostree[1884]: Reading config file '/etc/rpm-ostreed.conf'
Nov 05 13:58:02 localhost.localdomain rpm-ostree[1884]: In idle state; will auto-exit in 60 seconds
Nov 05 13:58:02 localhost.localdomain systemd[1]: Started RPM-OSTree System Management Daemon.
Nov 05 13:58:02 localhost.localdomain rpm-ostree[1884]: client(id:cli dbus:1.55 unit:session-5.scope uid:0) added; new total=1
Nov 05 13:58:02 localhost.localdomain rpm-ostree[1884]: Initiated txn Upgrade for client(id:cli dbus:1.55 unit:session-5.scope uid:0): /org/projectatomic/rpmostree1/fedora_iot
Nov 05 14:08:25 localhost.localdomain rpm-ostree[1884]: libostree pull from 'fedora-iot' for fedora/29/aarch64/iot complete
                                                        security: GPG: commit http: TLS
                                                        non-delta: meta: 517 content: 2097
                                                        transfer: secs: 623 size: 106.0 MB
Nov 05 14:09:17 localhost.localdomain rpm-ostree[1884]: Preparing pkg txn; enabled repos: ['fedora-modular', 'updates-modular', 'updates', 'fedora'] solvables: 60909
Nov 05 14:09:33 localhost.localdomain rpm-ostree[1884]: Relabeled 0/1 pkgs
Nov 05 14:09:54 localhost.localdomain rpm-ostree[1884]: Txn Upgrade on /org/projectatomic/rpmostree1/fedora_iot failed: Executing %transfiletriggerin for info: Executing bwrap(/bin/sh): Child process killed by signal 7; run `journalctl -t 'rpm-ostree(info.transfiletriggerin)'` for more information
Nov 05 14:09:54 localhost.localdomain rpm-ostree[1884]: client(id:cli dbus:1.55 unit:session-5.scope uid:0) vanished; remaining=0
Nov 05 14:09:54 localhost.localdomain rpm-ostree[1884]: In idle state; will auto-exit in 63 seconds
Nov 05 14:10:57 localhost.localdomain rpm-ostree[1884]: In idle state; will auto-exit in 61 seconds

MacBook-Pro:fedora tim$ pwd
/Users/tim/Projects/iotaa/fedora
MacBook-Pro:fedora tim$ mv rpm-ostreed rpm-ostreed.txt
MacBook-Pro:fedora tim$ ll *txt
-rw-r--r--  1 tim  staff  19620  5 Nov 14:18 journactl-full.txt
-rw-r--r--  1 tim  staff   7620  5 Nov 14:22 rpm-ostreed.txt
MacBook-Pro:fedora tim$ rm *txt
MacBook-Pro:fedora tim$ ll -tr
total 16877560
-rw-r--r--   1 tim  staff         271  5 Aug 11:13 Fedora-IoT-28-20180610.0.x86.raw.xz
-rw-r--r--   1 tim  staff        6056 23 Oct 10:58 fedora-iot-29-x86_64.json
drwxr-xr-x  31 tim  staff         992 23 Oct 11:02 bento
-rwxr-xr-x   1 tim  staff          57 23 Oct 14:22 clean
-rw-r--r--   1 tim  staff  3342877184 24 Oct 12:11 Fedora-IoT-28-20180610.0.x86_64.vdi
-rw-r--r--   1 tim  staff   440250012 24 Oct 13:41 Fedora-IoT-29-20181021.0.x86_64.raw.xz
-rw-r--r--   1 tim  staff  4294967296 24 Oct 13:41 Fedora-IoT-29-20181021.0.x86_64.raw
-rw-r--r--   1 tim  staff  1000341504 24 Oct 13:58 Fedora-IoT-ostree-x86_64-29-20181021.0.iso
-rw-r--r--   1 tim  staff         518 24 Oct 14:26 go
-rwx------   1 tim  staff        5986 24 Oct 15:36 iot-start.ovf
-rw-r--r--   1 tim  staff   730257408 24 Oct 15:37 iot-start-disk001.vmdk
-rw-r--r--   1 tim  staff  1653621248 24 Oct 15:58 Fedora-IoT-29-20181021.0.x86_64.vdi
-rw-r--r--   1 tim  staff        1390 25 Oct 11:01 notes
-rw-r--r--   1 tim  staff        2045 25 Oct 11:46 uperr
-rw-r--r--   1 tim  staff      397676 25 Oct 17:01 badvagrant
-rw-r--r--   1 tim  staff         306 25 Oct 18:56 Vagrantfile
-rw-r--r--   1 tim  staff        1009 28 Oct 11:27 Rakefile
drwxr-xr-x  18 tim  staff         576  5 Nov 12:22 aarch64
MacBook-Pro:fedora tim$ ll aarch64/
total 35744
-rw-r--r--  1 tim  staff  15185107 29 Oct 15:28 adminapp-1.41.1-1.noarch.rpm
-rw-r--r--  1 tim  staff      1518 29 Oct 15:28 boto3-needs
-rw-r--r--  1 tim  staff      1216 29 Oct 15:28 boto3-needs-again
-rw-r--r--  1 tim  staff      1185 29 Oct 15:28 botocore-needs
-rw-r--r--  1 tim  staff   2972937 29 Oct 15:28 hub-connect-1.38.rc1-1.noarch.rpm
-rw-r--r--  1 tim  staff     15542 29 Oct 15:28 hubaccess-0.68-1.noarch.rpm
-rw-r--r--  1 tim  staff       629 29 Oct 15:28 hubaccess-installed
-rw-r--r--  1 tim  staff      1275 29 Oct 15:28 hubaccess-needs
-rw-r--r--  1 tim  staff      5610  5 Nov 12:20 journalctl.txt
-rw-r--r--  1 tim  staff       370 29 Oct 15:28 needsdiff
-rw-r--r--  1 tim  staff      1712 29 Oct 15:31 notes
-rw-r--r--  1 tim  staff       973 29 Oct 15:28 packages-for-pip
-rw-r--r--  1 tim  staff      9585 29 Oct 15:28 pushkeys-0.68-1.noarch.rpm
-rw-r--r--  1 tim  staff      1558 29 Oct 15:28 pushkeys-needs
-rw-r--r--  1 tim  staff     52445 29 Oct 15:28 sensei-core-0.44-1.noarch.rpm
-rw-r--r--  1 tim  staff      9686 29 Oct 15:28 sensei-master-0.44-1.noarch.rpm
MacBook-Pro:fedora tim$ ll -tr aarch64/
total 35744
-rw-r--r--  1 tim  staff  15185107 29 Oct 15:28 adminapp-1.41.1-1.noarch.rpm
-rw-r--r--  1 tim  staff      1518 29 Oct 15:28 boto3-needs
-rw-r--r--  1 tim  staff      1216 29 Oct 15:28 boto3-needs-again
-rw-r--r--  1 tim  staff      1185 29 Oct 15:28 botocore-needs
-rw-r--r--  1 tim  staff     15542 29 Oct 15:28 hubaccess-0.68-1.noarch.rpm
-rw-r--r--  1 tim  staff       629 29 Oct 15:28 hubaccess-installed
-rw-r--r--  1 tim  staff      1275 29 Oct 15:28 hubaccess-needs
-rw-r--r--  1 tim  staff   2972937 29 Oct 15:28 hub-connect-1.38.rc1-1.noarch.rpm
-rw-r--r--  1 tim  staff       370 29 Oct 15:28 needsdiff
-rw-r--r--  1 tim  staff       973 29 Oct 15:28 packages-for-pip
-rw-r--r--  1 tim  staff      9585 29 Oct 15:28 pushkeys-0.68-1.noarch.rpm
-rw-r--r--  1 tim  staff      1558 29 Oct 15:28 pushkeys-needs
-rw-r--r--  1 tim  staff     52445 29 Oct 15:28 sensei-core-0.44-1.noarch.rpm
-rw-r--r--  1 tim  staff      9686 29 Oct 15:28 sensei-master-0.44-1.noarch.rpm
-rw-r--r--  1 tim  staff      1712 29 Oct 15:31 notes
-rw-r--r--  1 tim  staff      5610  5 Nov 12:20 journalctl.txt
MacBook-Pro:fedora tim$ rm aarch64/journalctl.txt 
MacBook-Pro:fedora tim$ cd ../open-zwave/
MacBook-Pro:open-zwave tim$ ll
total 200
-rw-r--r--   1 tim  staff  39380  7 Nov 12:03 ChangeLog
-rw-r--r--   1 tim  staff   2605  7 Nov 12:03 INSTALL
-rw-r--r--   1 tim  staff   1848  7 Nov 12:03 Makefile
-rw-r--r--   1 tim  staff   5744  7 Nov 12:03 README.md
drwxr-xr-x  92 tim  staff   2944  7 Nov 12:03 config
drwxr-xr-x   7 tim  staff    224  7 Nov 12:03 cpp
drwxr-xr-x  23 tim  staff    736  7 Nov 12:03 debian
drwxr-xr-x   4 tim  staff    128  7 Nov 12:03 dist
-rw-r--r--   1 tim  staff   4468  7 Nov 12:03 dist.mk
-rw-r--r--   1 tim  staff  32711  7 Nov 12:03 distfiles.mk
drwxr-xr-x   6 tim  staff    192  7 Nov 12:03 docs
drwxr-xr-x   5 tim  staff    160  7 Nov 12:03 dotnet
drwxr-xr-x   6 tim  staff    192  7 Nov 12:03 license
-rwxr-xr-x   1 tim  staff    954  7 Nov 12:03 makedist
MacBook-Pro:open-zwave tim$ less Makefile 
MacBook-Pro:open-zwave tim$ cat makedist 
#!/usr/bin/perl

use strict;
use XML::Simple;
use Data::Dumper;
use File::Basename;
use List::Util 1.33 'any';

my $input = ".distfiles";
my $output = "distfiles.mk";

my @excludedir = (".github");
my @excludefile = ();

open( my $fh => $input) || die "Cannot open $input: $!";
open( my $oh, ">", $output) || die "Cannot open $output: $!";

print $oh "# This File is automatically generated by make dist-update\n";
print $oh "# Any Edits on this file will be lost next time dist-update is run\n";
print $oh "\n";
print $oh "DISTFILES =\t";

while(my $line = <$fh>) {
    chomp($line);
    my $dir = dirname($line);
    if (any {/^$dir$/} @excludedir) 
    {
        print "Excluded File $line - (Directory Excluded)\n";
        next;
    }
    if (any {/^$line$/} @excludefile)
    {
        print "Excluded File $line - (File Excluded)\n";
        next;
    }
    print $oh $line." \\\n\t";
}
print $oh "cpp/src/vers.cpp\n";
close($oh);
close($fh);
    MacBook-Pro:open-zwave tim$ 
MacBook-Pro:open-zwave tim$ 
MacBook-Pro:open-zwave tim$ 
MacBook-Pro:open-zwave tim$ 
MacBook-Pro:open-zwave tim$ less Makefile 
MacBook-Pro:open-zwave tim$ less Makefile 
MacBook-Pro:open-zwave tim$ make all
LDFLAGS="" CPPFLAGS="" /Applications/Xcode.app/Contents/Developer/usr/bin/make -C /Users/tim/Projects/iotaa/open-zwave/cpp/build/ -
Building OpenZWave Version 1.4-3251-g3b96eca3-dirty
Building tinystr.o
Building tinyxml.o
Building tinyxmlerror.o
Building tinyxmlparser.o
Building hid.o
Building aes_modes.o
Building aescrypt.o
Building aeskey.o
Building aestab.o
Building Alarm.o
Building ApplicationStatus.o
Building Association.o
Building AssociationCommandConfiguration.o
Building Basic.o
Building BasicWindowCovering.o
Building Battery.o
Building CRC16Encap.o
Building CentralScene.o
Building ClimateControlSchedule.o
Building Clock.o
Building Color.o
Building CommandClass.o
Building CommandClasses.o
Building Configuration.o
Building ControllerReplication.o
Building DeviceResetLocally.o
Building DoorLock.o
Building DoorLockLogging.o
Building EnergyProduction.o
Building Hail.o
Building Indicator.o
Building Language.o
Building Lock.o
Building ManufacturerSpecific.o
Building Meter.o
Building MeterPulse.o
Building MultiChannelAssociation.o
Building MultiCmd.o
Building MultiInstance.o
Building NoOperation.o
Building NodeNaming.o
Building Powerlevel.o
Building Proprietary.o
Building Protection.o
Building SceneActivation.o
Building Security.o
Building SensorAlarm.o
Building SensorBinary.o
Building SensorMultilevel.o
Building SwitchAll.o
Building SwitchBinary.o
Building SwitchMultilevel.o
Building SwitchToggleBinary.o
Building SwitchToggleMultilevel.o
Building ThermostatFanMode.o
Building ThermostatFanState.o
Building ThermostatMode.o
Building ThermostatOperatingState.o
Building ThermostatSetpoint.o
Building TimeParameters.o
Building UserCode.o
Building Version.o
Building WakeUp.o
Building ZWavePlusInfo.o
Building Value.o
Building ValueBool.o
Building ValueButton.o
Building ValueByte.o
Building ValueDecimal.o
Building ValueInt.o
Building ValueList.o
Building ValueRaw.o
Building ValueSchedule.o
Building ValueShort.o
Building ValueStore.o
Building ValueString.o
Building Controller.o
Building Event.o
Building FileOps.o
Building HidController.o
Building Log.o
Building Mutex.o
Building SerialController.o
Building Stream.o
Building Thread.o
Building TimeStamp.o
Building Wait.o
Building EventImpl.o
Building FileOpsImpl.o
Building LogImpl.o
Building MutexImpl.o
Building SerialControllerImpl.o
Building ThreadImpl.o
Building TimeStampImpl.o
Building WaitImpl.o
Building Driver.o
Building Group.o
Building Manager.o
Building Msg.o
Building Node.o
Building Notification.o
Building Options.o
Building Scene.o
Building Utils.o
Building ZWSecurity.o
Creating vers.cpp
Building vers.o
Linking Static Library
Linking Shared Library
Making ozw_config file
LDFLAGS="" CPPFLAGS="" /Applications/Xcode.app/Contents/Developer/usr/bin/make -C /Users/tim/Projects/iotaa/open-zwave/cpp/examples/MinOZW/ -
Building Main.o
Main.cpp:139:14: warning: explicitly assigning value of variable of type 'NodeInfo *' to itself [-Wself-assign]
                                nodeInfo = nodeInfo;            // placeholder for real action
                                ~~~~~~~~ ^ ~~~~~~~~
Main.cpp:149:14: warning: explicitly assigning value of variable of type 'NodeInfo *' to itself [-Wself-assign]
                                nodeInfo = nodeInfo;            // placeholder for real action
                                ~~~~~~~~ ^ ~~~~~~~~
Main.cpp:192:14: warning: explicitly assigning value of variable of type 'NodeInfo *' to itself [-Wself-assign]
                                nodeInfo = nodeInfo;            // placeholder for real action
                                ~~~~~~~~ ^ ~~~~~~~~
3 warnings generated.
Main.cpp:139:14: warning: explicitly assigning value of variable of type 'NodeInfo *' to itself [-Wself-assign]
                                nodeInfo = nodeInfo;            // placeholder for real action
                                ~~~~~~~~ ^ ~~~~~~~~
Main.cpp:149:14: warning: explicitly assigning value of variable of type 'NodeInfo *' to itself [-Wself-assign]
                                nodeInfo = nodeInfo;            // placeholder for real action
                                ~~~~~~~~ ^ ~~~~~~~~
Main.cpp:192:14: warning: explicitly assigning value of variable of type 'NodeInfo *' to itself [-Wself-assign]
                                nodeInfo = nodeInfo;            // placeholder for real action
                                ~~~~~~~~ ^ ~~~~~~~~
3 warnings generated.
Linking /Users/tim/Projects/iotaa/open-zwave/.lib/MinOZW
g++   -arch i386 -arch x86_64 -o /Users/tim/Projects/iotaa/open-zwave/.lib/MinOZW /Users/tim/Projects/iotaa/open-zwave/.lib/Main.o /Users/tim/Projects/iotaa/open-zwave/libopenzwave-1.4.dylib /Users/tim/Projects/iotaa/open-zwave/libopenzwave.dylib -pthread
clang: warning: argument unused during compilation: '-pthread' [-Wunused-command-line-argument]
Creating Temporary Shell Launch Script
MacBook-Pro:open-zwave tim$ ssh 192.168.1.138
Last login: Thu Nov  8 11:08:01 2018 from 10.0.34.218
[tim@localhost ~]$ rpm -q rpm-ostree
rpm-ostree-2018.9-3.fc29.aarch64
[tim@localhost ~]$ rpm -qi rpm-ostree
Name        : rpm-ostree
Version     : 2018.9
Release     : 3.fc29
Architecture: aarch64
Install Date: Tue 06 Nov 2018 11:06:09 UTC
Group       : Unspecified
Size        : 1209753
License     : LGPLv2+
Signature   : RSA/SHA256, Fri 02 Nov 2018 16:51:50 UTC, Key ID a20aa56b429476b4
Source RPM  : rpm-ostree-2018.9-3.fc29.src.rpm
Build Date  : Fri 02 Nov 2018 16:33:52 UTC
Build Host  : buildhw-aarch64-04.arm.fedoraproject.org
Relocations : (not relocatable)
Packager    : Fedora Project
Vendor      : Fedora Project
URL         : https://github.com/projectatomic/rpm-ostree
Bug URL     : https://bugz.fedoraproject.org/rpm-ostree
Summary     : Hybrid image/package system
Description :
rpm-ostree is a hybrid image/package system.  It supports
"composing" packages on a build server into an OSTree repository,
which can then be replicated by client systems with atomic upgrades.
Additionally, unlike many "pure" image systems, with rpm-ostree
each client system can layer on additional packages, providing
a "best of both worlds" approach.
[tim@localhost ~]$ man rofiles-fuse
bash: man: command not found
[tim@localhost ~]$ rofiles-fuse --help
usage: rofiles-fuse basepath mountpoint [options]

   Makes basepath visible at mountpoint such that files are read-only, directories are writable

general options:
   -o opt,[opt...]     mount options
   -h  --help          print help

[tim@localhost ~]$ sudo gdb /usr/bin/rofiles-fuse /var/lib/systemd/coredump/core.rofiles-fuse.0.4036b166301042369614a959f7a83cff.1328.1541604333000000
GNU gdb (GDB) Fedora 8.2-3.fc29
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/rofiles-fuse...(no debugging symbols found)...done.
[New LWP 1345]
[New LWP 1328]
[New LWP 1331]
[New LWP 1340]
[New LWP 1346]
[New LWP 1344]
[New LWP 1330]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `rofiles-fuse --copyup usr /tmp/rpmostree-rofiles-fuse.xhO0Bo'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000ffff97494008 in do_releasedir () from /lib64/libfuse.so.2
[Current thread is 1 (Thread 0xffff877fdfd0 (LWP 1345))]
Missing separate debuginfos, use: dnf debuginfo-install ostree-2018.9-1.fc29.aarch64
(gdb) ?
Undefined command: "".  Try "help".
(gdb) help
List of classes of commands:

aliases -- Aliases of other commands
breakpoints -- Making program stop at certain points
data -- Examining data
files -- Specifying and examining files
internals -- Maintenance commands
obscure -- Obscure features
running -- Running the program
stack -- Examining the stack
status -- Status inquiries
support -- Support facilities
tracepoints -- Tracing of program execution without stopping the program
user-defined -- User-defined commands

Type "help" followed by a class name for a list of commands in that class.
Type "help all" for the list of all commands.
Type "help" followed by command name for full documentation.
Type "apropos word" to search for commands related to "word".
Command name abbreviations are allowed if unambiguous.
(gdb) help stack
Examining the stack.
The stack is made up of stack frames.  Gdb assigns numbers to stack frames
counting from zero for the innermost (currently executing) frame.

At any time gdb identifies one frame as the "selected" frame.
Variable lookups are done with respect to the selected frame.
When the program being debugged stops, gdb selects the innermost frame.
The commands below can be used to select other frames by number or address.

List of commands:

backtrace -- Print backtrace of all stack frames
bt -- Print backtrace of all stack frames
down -- Select and print stack frame called by this one
frame -- Select and print a stack frame
return -- Make selected stack frame return to its caller
select-frame -- Select a stack frame without printing anything
up -- Select and print stack frame that called this one

Type "help" followed by command name for full documentation.
Type "apropos word" to search for commands related to "word".
Command name abbreviations are allowed if unambiguous.
(gdb) stack bt
Undefined command: "stack".  Try "help".
(gdb) bt
#0  0x0000ffff97494008 in do_releasedir () from /lib64/libfuse.so.2
#1  0x0000ffff9749507c in fuse_ll_process_buf () from /lib64/libfuse.so.2
#2  0x0000ffff97492190 in fuse_do_work () from /lib64/libfuse.so.2
#3  0x0000ffff969737f8 in start_thread () from /lib64/libpthread.so.0
#4  0x0000ffff968c744c in thread_start () from /lib64/libc.so.6
(gdb) help
List of classes of commands:

aliases -- Aliases of other commands
breakpoints -- Making program stop at certain points
data -- Examining data
files -- Specifying and examining files
internals -- Maintenance commands
obscure -- Obscure features
running -- Running the program
stack -- Examining the stack
status -- Status inquiries
support -- Support facilities
tracepoints -- Tracing of program execution without stopping the program
user-defined -- User-defined commands

Type "help" followed by a class name for a list of commands in that class.
Type "help all" for the list of all commands.
Type "help" followed by command name for full documentation.
Type "apropos word" to search for commands related to "word".
Command name abbreviations are allowed if unambiguous.
(gdb) help status
Status inquiries.

List of commands:

info -- Generic command for showing things about the program being debugged
info address -- Describe where symbol SYM is stored
info all-registers -- List of all registers and their contents
info args -- Argument variables of current stack frame
info auto-load -- Print current status of auto-loaded files
info auto-load gdb-scripts -- Print the list of automatically loaded sequences of commands
info auto-load guile-scripts -- Print the list of automatically loaded Guile scripts
info auto-load libthread-db -- Print the list of loaded inferior specific libthread_db
info auto-load local-gdbinit -- Print whether current directory .gdbinit file has been loaded
info auto-load python-scripts -- Print the list of automatically loaded Python scripts
info auxv -- Display the inferior's auxiliary vector
info bookmarks -- Status of user-settable bookmarks
info breakpoints -- Status of specified breakpoints (all user-settable breakpoints if no argument)
info checkpoints -- IDs of currently known checkpoints
info classes -- All Objective-C classes
info common -- Print out the values contained in a Fortran COMMON block
info copying -- Conditions for redistributing copies of GDB
info dcache -- Print information on the dcache performance
info display -- Expressions to display when program stops
info exceptions -- List all Ada exception names
info extensions -- All filename extensions associated with a source language
info files -- Names of targets and files being debugged
info float -- Print the status of the floating point unit
info frame -- All about selected stack frame
info frame-filter -- List all registered Python frame-filters
info functions -- All function names
info guile -- Prefix command for Guile info displays
info handle -- What debugger does when program gets various signals
--Type <RET> for more, q to quit, c to continue without paging--
info inferiors -- Print a list of inferiors being managed
info line -- Core addresses of the code for a source line
info locals -- Local variables of current stack frame
info macro -- Show the definition of MACRO
info macros -- Show the definitions of all macros at LINESPEC
info mem -- Memory region attributes
info os -- Show OS data ARG
info pretty-printer -- GDB command to list all registered pretty-printers
info probes -- Show available static probes
info probes all -- Show information about all type of probes
info probes dtrace -- Show information about DTrace static probes
info probes stap -- Show information about SystemTap static probes
info proc -- Show /proc process information about any running process
info proc all -- List all available /proc info
info proc cmdline -- List command line arguments of the process
info proc cwd -- List current working directory of the process
info proc exe -- List absolute filename for executable of the process
info proc mappings -- List of mapped memory regions
info proc stat -- List process info from /proc/PID/stat
info proc status -- List process info from /proc/PID/status
info program -- Execution status of the program
info record -- Info record options
info registers -- List of integer registers and their contents
info scope -- List the variables local to a scope
info selectors -- All Objective-C selectors
info set -- Show all GDB settings
info sharedlibrary -- Status of loaded shared object libraries
info signals -- What debugger does when program gets various signals
info skip -- Display the status of skips
info source -- Information about the current source file
info sources -- Source files in the program
info stack -- Backtrace of the stack
--Type <RET> for more, q to quit, c to continue without paging--
info static-tracepoint-markers -- List target static tracepoints markers
info symbol -- Describe what symbol is at location ADDR
info target -- Names of targets and files being debugged
info tasks -- Provide information about all known Ada tasks
info terminal -- Print inferior's saved terminal status
info threads -- Display currently known threads
info tracepoints -- Status of specified tracepoints (all tracepoints if no argument)
info tvariables -- Status of trace state variables and their values
info type-printers -- GDB command to list all registered type-printers
info types -- All type names
info unwinder -- GDB command to list unwinders
info variables -- All global and static variable names
info vector -- Print the status of the vector unit
info vtbl -- Show the virtual function table for a C++ object
info warranty -- Various kinds of warranty you do not have
info watchpoints -- Status of specified watchpoints (all watchpoints if no argument)
info win -- List of all displayed windows
info xmethod -- GDB command to list registered xmethod matchers
macro -- Prefix for commands dealing with C preprocessor macros
macro define -- Define a new C/C++ preprocessor macro
macro expand -- Fully expand any C/C++ preprocessor macro invocations in EXPRESSION
macro expand-once -- Expand C/C++ preprocessor macro invocations appearing directly in EXPRESSION
macro list -- List all the macros defined using the `macro define' command
macro undef -- Remove the definition of the C/C++ preprocessor macro with the given name
show -- Generic command for showing things about the debugger
show ada -- Generic command for showing Ada-specific settings
show ada print-signatures -- Show whether the output of formal and return types for functions in the overloads selection menu is activated
show ada trust-PAD-over-XVS -- Show whether an optimization trusting PAD types over XVS types is activated
show agent -- Show debugger's willingness to use agent as a helper
show annotate -- Show annotation_level
show architecture -- Show architecture of target
show args -- Show argument list to give program being debugged when it is started
--Type <RET> for more, q to quit, c to continue without paging--
show arm -- Various ARM-specific commands
show arm abi -- Show the ABI
show arm apcs32 -- Show usage of ARM 32-bit mode
show arm disassembler -- Show the disassembly style
show arm fallback-mode -- Show the mode assumed when symbols are unavailable
show arm force-mode -- Show the mode assumed even when symbols are available
show arm fpu -- Show the floating point type
show auto-connect-native-target -- Show whether GDB may automatically connect to the native target
show auto-load -- Show auto-loading specific settings
show auto-load gdb-scripts -- Show whether auto-loading of canned sequences of commands scripts is enabled
show auto-load guile-scripts -- Show the debugger's behaviour regarding auto-loaded Guile scripts
show auto-load libthread-db -- Show whether auto-loading inferior specific libthread_db is enabled
show auto-load local-gdbinit -- Show whether auto-loading .gdbinit script in current directory is enabled
show auto-load python-scripts -- Show the debugger's behaviour regarding auto-loaded Python scripts
show auto-load safe-path -- Show the list of files and directories that are safe for auto-loading
show auto-load scripts-directory -- Show the list of directories from which to load auto-loaded scripts
show auto-solib-add -- Show autoloading of shared library symbols
show backtrace -- Show backtrace specific variables
show backtrace limit -- Show the upper bound on the number of backtrace levels
show backtrace past-entry -- Show whether backtraces should continue past the entry point of a program
show backtrace past-main -- Show whether backtraces should continue past "main"
show basenames-may-differ -- Show whether a source file may have multiple base names
show breakpoint -- Breakpoint specific settings
show breakpoint always-inserted -- Show mode for inserting breakpoints
show breakpoint auto-hw -- Show automatic usage of hardware breakpoints
show breakpoint condition-evaluation -- Show mode of breakpoint condition evaluation
show breakpoint pending -- Show debugger's behavior regarding pending breakpoints
show build-id-core-loads -- Show whether CORE-FILE loads the build-id associated files automatically
show build-id-verbose -- Show debugging level of the build-id locator
show can-use-hw-watchpoints -- Show debugger's willingness to use watchpoint hardware
show case-sensitive -- Show case sensitivity in name search
show charset -- Show the host and target character sets
--Type <RET> for more, q to quit, c to continue without paging--
show check -- Show the status of the type/range checker
show check range -- Show range checking
show check type -- Show strict type checking
show circular-trace-buffer -- Show target's use of circular trace buffer
show code-cache -- Show cache use for code segment access
show coerce-float-to-double -- Show coercion of floats to doubles when calling functions
show commands -- Show the history of commands you typed
show compile-args -- Show compile command GCC command-line arguments
show compile-gcc -- Show compile command GCC driver filename
show complaints -- Show max number of complaints about incorrect symbols
show configuration -- Show how GDB was configured at build time
show confirm -- Show whether to confirm potentially dangerous operations
show convenience -- Debugger convenience ("$foo") variables and functions
show copying -- Conditions for redistributing copies of GDB
show cp-abi -- Show the ABI used for inspecting C++ objects
show cwd -- Show the current working directory that is used when the inferior is started
show data-directory -- Show GDB's data directory
show dcache -- Show dcachesettings
show dcache line-size -- Show dcache line size
show dcache size -- Show number of dcache lines
show debug -- Generic command for showing gdb debugging flags
show debug aarch64 -- Show AArch64 debugging
show debug arch -- Show architecture debugging
show debug arm -- Show ARM debugging
show debug auto-load -- Show auto-load verifications debugging
show debug bfd-cache -- Show bfd cache debugging
show debug check-physname -- Show cross-checking of "physname" code against demangler
show debug coff-pe-read -- Show coff PE read debugging
show debug compile -- Show compile command debugging
show debug displaced -- Show displaced stepping debugging
show debug dwarf-die -- Show debugging of the DWARF DIE reader
show debug dwarf-line -- Show debugging of the dwarf line reader
--Type <RET> for more, q to quit, c to continue without paging--
show debug dwarf-read -- Show debugging of the DWARF reader
show debug entry-values -- Show entry values and tail call frames debugging
show debug expression -- Show expression debugging
show debug frame -- Show frame debugging
show debug infrun -- Show inferior debugging
show debug jit -- Show JIT debugging
show debug libthread-db -- Show libthread-db debugging
show debug lin-lwp -- Show debugging of GNU/Linux lwp module
show debug linux-namespaces -- Show debugging of GNU/Linux namespaces module
show debug notification -- Show debugging of async remote notification
show debug observer -- Show observer debugging
show debug overload -- Show debugging of C++ overloading
show debug parser -- Show parser debugging
show debug py-unwind -- Show Python unwinder debugging
show debug record -- Show debugging of record/replay feature
show debug remote -- Show debugging of remote protocol
show debug separate-debug-file -- Show printing of separate debug info file search debug
show debug serial -- Show serial debugging
show debug stap-expression -- Show SystemTap expression debugging
show debug symbol-lookup -- Show debugging of symbol lookup
show debug symfile -- Show debugging of the symfile functions
show debug symtab-create -- Show debugging of symbol table creation
show debug target -- Show target debugging
show debug timestamp -- Show timestamping of debugging messages
show debug varobj -- Show varobj debugging
show debug xml -- Show XML parser debugging
show debug-file-directory -- Show the directories where separate debug symbols are searched for
show default-collect -- Show the list of expressions to collect by default
show demangle-style -- Show the current C++ demangling style
show detach-on-fork -- Show whether gdb will detach the child of a fork
show directories -- Show the search path for finding source files
show disable-randomization -- Show disabling of debuggee's virtual address space randomization
--Type <RET> for more, q to quit, c to continue without paging--
show disassemble-next-line -- Show whether to disassemble next source line or insn when execution stops
show disassembler-options -- Show the disassembler options
show disconnected-dprintf -- Show whether dprintf continues after GDB disconnects
show disconnected-tracing -- Show whether tracing continues after GDB disconnects
show displaced-stepping -- Show debugger's willingness to use displaced stepping
show dprintf-channel -- Show the channel to use for dynamic printf
show dprintf-function -- Show the function to use for dynamic printf
show dprintf-style -- Show the style of usage for dynamic printf
show dump-excluded-mappings -- Show whether gcore should dump mappings marked with the VM_DONTDUMP flag
show editing -- Show editing of command lines as they are typed
show endian -- Show endianness of target
show environment -- The environment to give the program
show exec-direction -- Show direction of execution (forward/reverse)
show exec-done-display -- Show notification of completion for asynchronous execution commands
show exec-wrapper -- Show the wrapper for running programs
show extended-prompt -- Show the extended prompt
show extension-language -- Show mapping between filename extension and source language
show filename-display -- Show how to display filenames
show follow-exec-mode -- Show debugger response to a program call of exec
show follow-fork-mode -- Show debugger response to a program call of fork or vfork
show frame-filter -- Prefix command for 'show' frame-filter related operations
show frame-filter priority -- GDB command to show the priority of the specified frame-filter
show gnutarget -- Show the current BFD target
show guile -- Prefix command for Guile preference settings
show guile print-stack -- Show the mode of Guile exception printing on error
show height -- Show number of lines in a page for GDB output pagination
show history -- Generic command for showing command history parameters
show history expansion -- Show history expansion on command input
show history filename -- Show the filename in which to record the command history
show history remove-duplicates -- Show how far back in history to look for and remove duplicate entries
show history save -- Show saving of the history record on exit
show history size -- Show the size of the command history
--Type <RET> for more, q to quit, c to continue without paging--
show host-charset -- Show the host character set
show inferior-tty -- Show terminal for future runs of program being debugged
show input-radix -- Show default input radix for entering numbers
show interactive-mode -- Show whether GDB's standard input is a terminal
show language -- Show the current source language
show libthread-db-search-path -- Show the current search path or libthread_db
show listsize -- Show number of source lines gdb will list by default
show logging -- Show logging options
show logging file -- Show the current logfile
show logging overwrite -- Show whether logging overwrites or appends to the log file
show logging redirect -- Show the logging output mode
show max-completions -- Show maximum number of completion candidates
show max-user-call-depth -- Show the max call depth for non-python/scheme user-defined commands
show max-value-size -- Show maximum sized value gdb will load from the inferior
show may-insert-breakpoints -- Show permission to insert breakpoints in the target
show may-insert-fast-tracepoints -- Show permission to insert fast tracepoints in the target
show may-insert-tracepoints -- Show permission to insert tracepoints in the target
show may-interrupt -- Show permission to interrupt or signal the target
show may-write-memory -- Show permission to write into target memory
show may-write-registers -- Show permission to write into registers
show mem -- Memory regions settings
show mem  inaccessible-by-default -- Show handling of unknown memory regions
show mi-async -- Show whether MI asynchronous mode is enabled
show multiple-symbols -- Show how the debugger handles ambiguities in expressions
show non-stop -- Show whether gdb controls the inferior in non-stop mode
show observer -- Show whether gdb controls the inferior in observer mode
show opaque-type-resolution -- Show resolution of opaque struct/class/union types (if set before loading symbols)
show osabi -- Show OS ABI of target
show output-radix -- Show default output radix for printing of values
show overload-resolution -- Show overload resolution in evaluating C++ functions
show pagination -- Show state of GDB output pagination
show paths -- Current search path for finding object files
--Type <RET> for more, q to quit, c to continue without paging--
show powerpc -- Various PowerPC-specific commands
show powerpc exact-watchpoints -- Show whether to use just one debug register for watchpoints on scalars
show powerpc soft-float -- Show whether to use a soft-float ABI
show powerpc vector-abi -- Show the vector ABI
show print -- Generic command for showing print settings
show print address -- Show printing of addresses
show print array -- Show pretty formatting of arrays
show print array-indexes -- Show printing of array indexes
show print asm-demangle -- Show demangling of C++/ObjC names in disassembly listings
show print demangle -- Show demangling of encoded C++/ObjC names when displaying symbols
show print elements -- Show limit on string chars or array elements to print
show print entry-values -- Show printing of function arguments at function entry
show print frame-arguments -- Show printing of non-scalar frame arguments
show print inferior-events -- Show printing of inferior events (e.g.
show print max-symbolic-offset -- Show the largest offset that will be printed in <symbol+1234> form
show print null-stop -- Show printing of char arrays to stop at first null char
show print object -- Show printing of object's derived type based on vtable info
show print pascal_static-members -- Show printing of pascal static members
show print pretty -- Show pretty formatting of structures
show print raw -- Generic command for showing "print raw" settings
show print raw frame-arguments -- Show whether to print frame arguments in raw form
show print repeats -- Show threshold for repeated print elements
show print sevenbit-strings -- Show printing of 8-bit characters in strings as \nnn
show print static-members -- Show printing of C++ static members
show print symbol -- Show printing of symbol names when printing pointers
show print symbol-filename -- Show printing of source filename and line number with <symbol>
show print symbol-loading -- Show printing of symbol loading messages
show print thread-events -- Show printing of thread events (such as thread start and exit)
show print type -- Generic command for showing type-printing settings
show print type methods -- Show printing of methods defined in classes
show print type nested-type-limit -- Show the number of recursive nested type definitions to print
show print type typedefs -- Show printing of typedefs defined in classes
--Type <RET> for more, q to quit, c to continue without paging--
show print union -- Show printing of unions interior to structures
show print vtbl -- Show printing of C++ virtual function tables
show prompt -- Show gdb's prompt
show python -- Prefix command for python preference settings
show python print-stack -- Show the mode of Python stack printing on error
show radix -- Show the default input and output number radices
show range-stepping -- Show whether target-assisted range stepping is enabled
show ravenscar -- Prefix command for showing Ravenscar-specific settings
show ravenscar task-switching -- Show whether support for GNAT Ravenscar tasks is enabled
show record -- Show record options
show record btrace -- Show record options
show record btrace bts -- Show record btrace bts options
show record btrace bts buffer-size -- Show the record/replay bts buffer size
show record btrace cpu -- Show the cpu to be used for trace decode
show record btrace pt -- Show record btrace pt options
show record btrace pt buffer-size -- Show the record/replay pt buffer size
show record btrace replay-memory-access -- Show what memory accesses are allowed during replay
show record full -- Show record options
show record full insn-number-max -- Show record/replay buffer limit
show record full memory-query -- Show whether query if PREC cannot record memory change of next instruction
show record full stop-at-limit -- Show whether record/replay stops when record/replay buffer becomes full
show record function-call-history-size -- Show number of functions to print in "record function-call-history"
show record instruction-history-size -- Show number of instructions to print in "record instruction-history"
show remote -- Remote protocol specific variables
show remote P-packet -- Show current use of remote protocol `P' (set-register) packet
show remote TracepointSource-packet -- Show current use of remote protocol `TracepointSource' (TracepointSource) packet
show remote X-packet -- Show current use of remote protocol `X' (binary-download) packet
show remote Z-packet -- Show use of remote protocol `Z' packets 
show remote access-watchpoint-packet -- Show current use of remote protocol `Z4' (access-watchpoint) packet
show remote agent-packet -- Show current use of remote protocol `QAgent' (agent) packet
show remote allow-packet -- Show current use of remote protocol `QAllow' (allow) packet
show remote attach-packet -- Show current use of remote protocol `vAttach' (attach) packet
--Type <RET> for more, q to quit, c to continue without paging--q
Quit
(gdb) info stack
#0  0x0000ffff97494008 in do_releasedir () from /lib64/libfuse.so.2
#1  0x0000ffff9749507c in fuse_ll_process_buf () from /lib64/libfuse.so.2
#2  0x0000ffff97492190 in fuse_do_work () from /lib64/libfuse.so.2
#3  0x0000ffff969737f8 in start_thread () from /lib64/libpthread.so.0
#4  0x0000ffff968c744c in thread_start () from /lib64/libc.so.6
(gdb) info locals
No symbol table info available.
(gdb) info registers
x0             0xffff78001240      281472695013952
x1             0x1c                28
x2             0xffff9424b038      281473167175736
x3             0xffff877fd594      281472955045268
x4             0xffff9748d3e8      281473219875816
x5             0x1d0               464
x6             0x600000000         25769803776
x7             0xffff780008d0      281472695011536
x8             0x7                 7
x9             0xffff877fe6d0      281472955049680
x10            0xb0                176
x11            0x0                 0
x12            0x2                 2
x13            0x0                 0
x14            0x0                 0
x15            0x1                 1
x16            0xffff974cef88      281473220145032
x17            0xffff96977570      281473208251760
x18            0x7f                127
x19            0xffff974ce000      281473220141056
x20            0xffff877fd790      281472955045776
x21            0xffff78001240      281472695013952
x22            0xaaab00826340      187651424674624
x23            0xffff974ce000      281473220141056
x24            0xffff974a0000      281473219952640
x25            0x0                 0
x26            0xffff9424b038      281473167175736
x27            0xffff974be5c0      281473220076992
x28            0xaaab00826558      187651424675160
x29            0xffff877fd570      281472955045232
x30            0xffff9749507c      281473219907708
sp             0xffff877fd570      0xffff877fd570
--Type <RET> for more, q to quit, c to continue without paging--
pc             0xffff97494008      0xffff97494008 <do_releasedir+40>
cpsr           0x80000000          [ EL=0 N ]
fpsr           0x0                 0
fpcr           0x0                 0
(gdb) help asm
Undefined command: "asm".  Try "help".
(gdb) help info
Generic command for showing things about the program being debugged.

List of info subcommands:

info address -- Describe where symbol SYM is stored
info all-registers -- List of all registers and their contents
info args -- Argument variables of current stack frame
info auto-load -- Print current status of auto-loaded files
info auxv -- Display the inferior's auxiliary vector
info bookmarks -- Status of user-settable bookmarks
info breakpoints -- Status of specified breakpoints (all user-settable breakpoints if no argument)
info checkpoints -- IDs of currently known checkpoints
info classes -- All Objective-C classes
info common -- Print out the values contained in a Fortran COMMON block
info copying -- Conditions for redistributing copies of GDB
info dcache -- Print information on the dcache performance
info display -- Expressions to display when program stops
info exceptions -- List all Ada exception names
info extensions -- All filename extensions associated with a source language
info files -- Names of targets and files being debugged
info float -- Print the status of the floating point unit
info frame -- All about selected stack frame
info frame-filter -- List all registered Python frame-filters
info functions -- All function names
info guile -- Prefix command for Guile info displays
info handle -- What debugger does when program gets various signals
info inferiors -- Print a list of inferiors being managed
info line -- Core addresses of the code for a source line
info locals -- Local variables of current stack frame
info macro -- Show the definition of MACRO
info macros -- Show the definitions of all macros at LINESPEC
info mem -- Memory region attributes
--Type <RET> for more, q to quit, c to continue without paging--q
Quit
(gdb) info args
No symbol table info available.
(gdb) help info
Generic command for showing things about the program being debugged.

List of info subcommands:

info address -- Describe where symbol SYM is stored
info all-registers -- List of all registers and their contents
info args -- Argument variables of current stack frame
info auto-load -- Print current status of auto-loaded files
info auxv -- Display the inferior's auxiliary vector
info bookmarks -- Status of user-settable bookmarks
info breakpoints -- Status of specified breakpoints (all user-settable breakpoints if no argument)
info checkpoints -- IDs of currently known checkpoints
info classes -- All Objective-C classes
info common -- Print out the values contained in a Fortran COMMON block
info copying -- Conditions for redistributing copies of GDB
info dcache -- Print information on the dcache performance
info display -- Expressions to display when program stops
info exceptions -- List all Ada exception names
info extensions -- All filename extensions associated with a source language
info files -- Names of targets and files being debugged
info float -- Print the status of the floating point unit
info frame -- All about selected stack frame
info frame-filter -- List all registered Python frame-filters
info functions -- All function names
info guile -- Prefix command for Guile info displays
info handle -- What debugger does when program gets various signals
info inferiors -- Print a list of inferiors being managed
info line -- Core addresses of the code for a source line
info locals -- Local variables of current stack frame
info macro -- Show the definition of MACRO
info macros -- Show the definitions of all macros at LINESPEC
info mem -- Memory region attributes
--Type <RET> for more, q to quit, c to continue without paging--
info os -- Show OS data ARG
info pretty-printer -- GDB command to list all registered pretty-printers
info probes -- Show available static probes
info proc -- Show /proc process information about any running process
info program -- Execution status of the program
info record -- Info record options
info registers -- List of integer registers and their contents
info scope -- List the variables local to a scope
info selectors -- All Objective-C selectors
info set -- Show all GDB settings
info sharedlibrary -- Status of loaded shared object libraries
info signals -- What debugger does when program gets various signals
info skip -- Display the status of skips
info source -- Information about the current source file
info sources -- Source files in the program
info stack -- Backtrace of the stack
info static-tracepoint-markers -- List target static tracepoints markers
info symbol -- Describe what symbol is at location ADDR
info target -- Names of targets and files being debugged
info tasks -- Provide information about all known Ada tasks
info terminal -- Print inferior's saved terminal status
info threads -- Display currently known threads
info tracepoints -- Status of specified tracepoints (all tracepoints if no argument)
info tvariables -- Status of trace state variables and their values
info type-printers -- GDB command to list all registered type-printers
info types -- All type names
info unwinder -- GDB command to list unwinders
info variables -- All global and static variable names
info vector -- Print the status of the vector unit
info vtbl -- Show the virtual function table for a C++ object
info warranty -- Various kinds of warranty you do not have
info watchpoints -- Status of specified watchpoints (all watchpoints if no argument)
--Type <RET> for more, q to quit, c to continue without paging--
info win -- List of all displayed windows
info xmethod -- GDB command to list registered xmethod matchers

Type "help info" followed by info subcommand name for full documentation.
Type "apropos word" to search for commands related to "word".
Command name abbreviations are allowed if unambiguous.
(gdb) info threads
  Id   Target Id                        Frame 
* 1    Thread 0xffff877fdfd0 (LWP 1345) 0x0000ffff97494008 in do_releasedir () from /lib64/libfuse.so.2
  2    Thread 0xffff95af8010 (LWP 1328) 0x0000ffff9697c160 in do_futex_wait.constprop () from /lib64/libpthread.so.0
  3    Thread 0xffff952b1fd0 (LWP 1331) 0x0000ffff9697d1cc in read () from /lib64/libpthread.so.0
  4    Thread 0xffff94a8efd0 (LWP 1340) 0x0000ffff9697dc74 in pread64 () from /lib64/libpthread.so.0
  5    Thread 0xffff86ffcfd0 (LWP 1346) 0x0000ffff9697d1c8 in read () from /lib64/libpthread.so.0
  6    Thread 0xffff87ffefd0 (LWP 1344) 0x0000ffff9697d1c8 in read () from /lib64/libpthread.so.0
  7    Thread 0xffff95ad4fd0 (LWP 1330) 0x0000ffff9697d1cc in read () from /lib64/libpthread.so.0
(gdb) info stack
#0  0x0000ffff97494008 in do_releasedir () from /lib64/libfuse.so.2
#1  0x0000ffff9749507c in fuse_ll_process_buf () from /lib64/libfuse.so.2
#2  0x0000ffff97492190 in fuse_do_work () from /lib64/libfuse.so.2
#3  0x0000ffff969737f8 in start_thread () from /lib64/libpthread.so.0
#4  0x0000ffff968c744c in thread_start () from /lib64/libc.so.6
(gdb) quit
[tim@localhost ~]$ rpm-ostree status
State: idle
AutomaticUpdates: disabled
Deployments:
● ostree://fedora-iot:fedora/29/aarch64/iot
                   Version: 29.20181106.0 (2018-11-06T11:14:51Z)
                BaseCommit: 283d542c1dbecba57c9bc15f0dcd9b2aac42b4f94ce10c0aae04c8401827b480
              GPGSignature: Valid signature by 5A03B4DD8254ECA02FDA1637A20AA56B429476B4
           LayeredPackages: buildah gdb

  ostree://fedora-iot:fedora/29/aarch64/iot
                   Version: 29.20181106.0 (2018-11-06T11:14:51Z)
                BaseCommit: 283d542c1dbecba57c9bc15f0dcd9b2aac42b4f94ce10c0aae04c8401827b480
              GPGSignature: Valid signature by 5A03B4DD8254ECA02FDA1637A20AA56B429476B4
           LayeredPackages: gdb
[tim@localhost ~]$ rpm -q rpm-ostree
rpm-ostree-2018.9-3.fc29.aarch64
[tim@localhost ~]$ sudo rpm-ostree install python3-pip
Checking out tree 283d542... done
Enabled rpm-md repositories: updates fedora updates-modular fedora-modular
rpm-md repo 'updates' (cached); generated: 2018-11-08T03:01:32Z
rpm-md repo 'fedora' (cached); generated: 2018-10-28T11:00:32Z
rpm-md repo 'updates-modular' (cached); generated: 2018-10-31T14:34:38Z
rpm-md repo 'fedora-modular' (cached); generated: 2018-10-28T11:01:36Z
Importing metadata [=============] 100%
Resolving dependencies... done
Will download: 1 package (1.9 MB)
  Downloading from fedora: [=============] 100%
Importing (1/1) [=============] 100%
Checking out packages (25/25) [=============] 100%
Running pre scripts... 0 done
Running post scripts... 8 done
Writing rpmdb... done
Writing OSTree commit... done
Staging deployment... done
Freed: 83.2 MB (pkgcache branches: 49)
Added:
  python3-pip-18.0-4.fc29.noarch
Run "systemctl reboot" to start a reboot
[tim@localhost ~]$ sudo rpm-ostree install kernel-modules-extra
Checking out tree 283d542... done
Enabled rpm-md repositories: updates fedora updates-modular fedora-modular
rpm-md repo 'updates' (cached); generated: 2018-11-08T03:01:32Z
rpm-md repo 'fedora' (cached); generated: 2018-10-28T11:00:32Z
rpm-md repo 'updates-modular' (cached); generated: 2018-10-31T14:34:38Z
rpm-md repo 'fedora-modular' (cached); generated: 2018-10-28T11:01:36Z
Importing metadata [=============] 100%
Resolving dependencies... done
Will download: 1 package (1.2 MB)
  Downloading from fedora: [=============] 100%
Importing (1/1) [=============] 100%
Checking out packages (26/26) [=============] 100%
Running pre scripts... 0 done
Running post scripts... error: Running %post for kernel-modules-extra: Executing bwrap(/bin/sh): Child process killed by signal 7; run `journalctl -t 'rpm-ostree(kernel-modules-extra.post)'` for more information
[tim@localhost ~]$ 

`

Expected results:
Package should install

Additional info:
jounalctl indicates that the first program to fail is rofiles-fuse. gdb of the dumped core shows indicates a point of failure (which I've no idea how of the reliability of):

```
Reading symbols from /usr/bin/rofiles-fuse...(no debugging symbols found)...done.
[New LWP 1423]
[New LWP 1414]
[New LWP 1411]
[New LWP 1413]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `rofiles-fuse --copyup usr /tmp/rpmostree-rofiles-fuse.2S7WdQ'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x0000ffffabfbb008 in do_releasedir () from /lib64/libfuse.so.2
[Current thread is 1 (Thread 0xffffa95b5fd0 (LWP 1423))]
Missing separate debuginfos, use: dnf debuginfo-install ostree-2018.9-1.fc29.aarch64
```