Bug 2042150 - dnf list result in writes to rpmdb
Summary: dnf list result in writes to rpmdb
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: rpm-software-management
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 2042099
TreeView+ depends on / blocked
 
Reported: 2022-01-18 21:20 UTC by Chris Murphy
Modified: 2022-01-24 12:51 UTC (History)
11 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2022-01-24 12:51:45 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Chris Murphy 2022-01-18 21:20:30 UTC
Relocate RPM database to /usr/lib/sysimage/rpm
https://fedoraproject.org/wiki/Changes/RelocateRPMToUsr#Scope

On devel@ list discussion, it's observed that 'dnf list' results in writes to /var/lib/rpm/rpmdb* files. e.g.

Before:
# stat *
  File: rpmdb.sqlite
  Size: 166379520 	Blocks: 324960     IO Block: 4096   regular file
Device: 23h/35d	Inode: 10381       Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:object_r:rpm_var_lib_t:s0
Access: 2021-06-25 16:36:27.053030328 -0600
Modify: 2022-01-17 00:07:44.782411170 -0700
Change: 2022-01-17 00:07:44.782411170 -0700
 Birth: 2021-11-02 14:53:53.267875775 -0600
  File: rpmdb.sqlite-shm
  Size: 32768     	Blocks: 64         IO Block: 4096   regular file
Device: 23h/35d	Inode: 10382       Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:object_r:rpm_var_lib_t:s0
Access: 2021-06-25 16:38:02.039038091 -0600
Modify: 2022-01-18 13:42:53.238884791 -0700
Change: 2022-01-18 13:42:53.238884791 -0700
 Birth: 2021-11-02 14:53:54.256874498 -0600
  File: rpmdb.sqlite-wal
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
Device: 23h/35d	Inode: 10383       Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:object_r:rpm_var_lib_t:s0
Access: 2021-06-25 16:36:23.997030078 -0600
Modify: 2022-01-17 00:07:44.860411165 -0700
Change: 2022-01-18 13:42:53.238884791 -0700
 Birth: 2021-11-02 14:53:54.256874498 -0600

After:
# stat *
  File: rpmdb.sqlite
  Size: 166379520 	Blocks: 324960     IO Block: 4096   regular file
Device: 23h/35d	Inode: 10381       Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:object_r:rpm_var_lib_t:s0
Access: 2021-06-25 16:36:27.053030328 -0600
Modify: 2022-01-17 00:07:44.782411170 -0700
Change: 2022-01-17 00:07:44.782411170 -0700
 Birth: 2021-11-02 14:53:53.267875775 -0600
  File: rpmdb.sqlite-shm
  Size: 32768     	Blocks: 64         IO Block: 4096   regular file
Device: 23h/35d	Inode: 10382       Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:object_r:rpm_var_lib_t:s0
Access: 2021-06-25 16:38:02.039038091 -0600
Modify: 2022-01-18 14:18:43.564957364 -0700
Change: 2022-01-18 14:18:43.564957364 -0700
 Birth: 2021-11-02 14:53:54.256874498 -0600
  File: rpmdb.sqlite-wal
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
Device: 23h/35d	Inode: 10383       Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:object_r:rpm_var_lib_t:s0
Access: 2021-06-25 16:36:23.997030078 -0600
Modify: 2022-01-17 00:07:44.860411165 -0700
Change: 2022-01-18 14:18:43.564957364 -0700
 Birth: 2021-11-02 14:53:54.256874498 -0600


https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/SDGZJLGPSDFUGLGFTPZBXQWB7PFTUJIX/

Comment 1 Chris Murphy 2022-01-18 21:25:52 UTC
No change indicated for rpmdb.sqlite.

There's mtime and ctime change in rpmdb.sqlite-shm, with mtime change suggesting file content change.

Only ctime change with rpmdb.sqlite-wal, and the file length remains zero.

So I guess we need to know if these updates are intentional, whether they should and can be avoided for what's ostensibly a read-only command, and what the consequence is if rpmdb is on a read-only mount. i.e. would 'dnf list' fail? Or?

Comment 2 Chris Murphy 2022-01-18 21:34:51 UTC
btrfs subvolume find-new -> List the recently modified files in a subvolume, after <last_gen> generation.

It doesn't show deletions, so there could be more going on that this shows. But these are the changes btrfs is tracking in ~1 minute while the system was idle, followed by 'dnf list'. It looks like rpmdb.sqlite-shm is the only file whose contents are being modified, consistent with the earlier results from stat.

# btrfs sub find / 65328
inode 10377 file offset 0 len 4096 disk start 89184714752 offset 0 gen 65330 flags NONE var/lib/dnf/history.sqlite
inode 10377 file offset 8192 len 4096 disk start 89184735232 offset 0 gen 65330 flags NONE var/lib/dnf/history.sqlite
inode 10377 file offset 24576 len 4096 disk start 99922223104 offset 0 gen 65330 flags NONE var/lib/dnf/history.sqlite
inode 10377 file offset 1409024 len 4096 disk start 59625312256 offset 0 gen 65330 flags NONE var/lib/dnf/history.sqlite
inode 10377 file offset 1572864 len 4096 disk start 79123251200 offset 0 gen 65330 flags NONE var/lib/dnf/history.sqlite
inode 10377 file offset 2158592 len 4096 disk start 101905473536 offset 0 gen 65330 flags NONE var/lib/dnf/history.sqlite
inode 10377 file offset 3588096 len 4096 disk start 101905477632 offset 0 gen 65330 flags NONE var/lib/dnf/history.sqlite
inode 10377 file offset 3616768 len 4096 disk start 101905481728 offset 0 gen 65330 flags NONE var/lib/dnf/history.sqlite
inode 10377 file offset 3629056 len 4096 disk start 101905485824 offset 0 gen 65330 flags NONE var/lib/dnf/history.sqlite
inode 10377 file offset 3653632 len 8192 disk start 101905657856 offset 0 gen 65330 flags NONE var/lib/dnf/history.sqlite
inode 10378 file offset 0 len 32768 disk start 45638676480 offset 0 gen 65330 flags NONE var/lib/dnf/history.sqlite-shm
inode 10382 file offset 0 len 4096 disk start 45530910720 offset 0 gen 65330 flags NONE var/lib/rpm/rpmdb.sqlite-shm
inode 10382 file offset 4096 len 28672 disk start 45638647808 offset 0 gen 65330 flags NONE var/lib/rpm/rpmdb.sqlite-shm
inode 10481 file offset 0 len 4096 disk start 41918775296 offset 0 gen 65330 flags NONE var/lib/PackageKit/transactions.db
inode 10481 file offset 12288 len 4096 disk start 41946468352 offset 0 gen 65330 flags NONE var/lib/PackageKit/transactions.db
inode 10481 file offset 20480 len 4096 disk start 57482940416 offset 0 gen 65330 flags NONE var/lib/PackageKit/transactions.db
inode 10514 file offset 0 len 2 disk start 0 offset 0 gen 65330 flags INLINE var/cache/dnf/expired_repos.json
inode 10542 file offset 0 len 2 disk start 0 offset 0 gen 65330 flags INLINE root/.cache/dconf/user
inode 1278286 file offset 0 len 4096 disk start 39791362048 offset 0 gen 65328 flags NONE var/lib/NetworkManager/timestamps
transid marker was 65330
#

Comment 3 Panu Matilainen 2022-01-19 08:54:49 UTC
(In reply to Chris Murphy from comment #1)
> No change indicated for rpmdb.sqlite.
> 
> There's mtime and ctime change in rpmdb.sqlite-shm, with mtime change
> suggesting file content change.
> 
> Only ctime change with rpmdb.sqlite-wal, and the file length remains zero.
> 
> So I guess we need to know if these updates are intentional, whether they
> should and can be avoided for what's ostensibly a read-only command, and
> what the consequence is if rpmdb is on a read-only mount. i.e. would 'dnf
> list' fail? Or?

AIUI this is normal sqlite WAL-mode (https://sqlite.org/wal.html) behavior. As of sqlite >= 3.22.0 (ie since 2018) WAL on read-only mount is supported (https://sqlite.org/releaselog/3_22_0.html) and should Just Work when librpm is used to access the rpmdb.

Comment 4 Lukáš Hrázký 2022-01-19 13:01:01 UTC
What Panu says. Also dnf doesn't access rpmdb directly, only through librpm API. Without doing any investigation I assume `dnf list` is just querying rpmdb for some data (quite certainly a read-only operation).

Comment 5 Lukáš Hrázký 2022-01-24 12:51:45 UTC
Seems things are working as expected, closing this. If you still think there's an issue to address, feel free to reopen.


Note You need to log in before you can comment on or make changes to this bug.