1594916 – (R)ead-only activation of non-read-only volume attribute no longer honored or present

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1594916 - (R)ead-only activation of non-read-only volume attribute no longer honored or present

Summary: (R)ead-only activation of non-read-only volume attribute no longer honored or...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	lvm2
Sub Component:
Version:	7.6
Hardware:	x86_64
OS:	Linux
Priority:	unspecified
Severity:	low
Target Milestone:	rc
Target Release:	---
Assignee:	David Teigland
QA Contact:	cluster-qe@redhat.com
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	1349661
TreeView+	depends on / blocked

Reported:	2018-06-25 17:01 UTC by Corey Marthaler
Modified:	2021-09-03 12:54 UTC (History)
CC List:	8 users (show)
Fixed In Version:	lvm2-2.02.180-4.el7
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2018-10-30 11:03:47 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2018:3193	0	None	None	None	2018-10-30 11:04:28 UTC

Description Corey Marthaler 2018-06-25 17:01:03 UTC

Description of problem:
SCENARIO - [add_to_ro_volume_list_w_mirror_resync_in_progress]
While resync is in progress, add mirror to the read only volume list and reactivate
Create a nosync mirror then resync it so that it can be up converted

mckinley-03: lvcreate  --type mirror -m 1 -n rolist_during_resync -L 6G mirror_sanity
Waiting until all mirror|raid volumes become fully syncd...
   0/1 mirror(s) are fully synced: ( 26.82% )
   0/1 mirror(s) are fully synced: ( 47.20% )
   0/1 mirror(s) are fully synced: ( 70.90% )
   0/1 mirror(s) are fully synced: ( 94.30% )
   1/1 mirror(s) are fully synced: ( 100.00% )
Sleeping 15 sec

Resyncing rolist_during_resync mirror
current copy percent: 8.33
rolist_during_resync:  lvs --noheadings -o lv_attr mirror_sanity/rolist_during_resync
ATTR: mwi-a-m---
rolist_during_resync_mimage_0:  lvs --noheadings -o lv_attr mirror_sanity/rolist_during_resync_mimage_0
ATTR: Iwi-aom---
rolist_during_resync_mimage_1:  lvs --noheadings -o lv_attr mirror_sanity/rolist_during_resync_mimage_1
ATTR: Iwi-aom---

While resync is in progress, add this mirror to the read only volume list and reactivate
lvchange --addtag RO mirror_sanity/rolist_during_resync
lvchange -an mirror_sanity/rolist_during_resync
lvchange -ay mirror_sanity/rolist_during_resync
current copy percent: 21.22

rolist_during_resync:  lvs --noheadings -o lv_attr mirror_sanity/rolist_during_resync
ATTR: mRi-a-m---
rolist_during_resync_mimage_0:  lvs --noheadings -o lv_attr mirror_sanity/rolist_during_resync_mimage_0
ATTR: IRi-aom---
rolist_during_resync_mimage_1:  lvs --noheadings -o lv_attr mirror_sanity/rolist_during_resync_mimage_1
ATTR: IRi-aom---

Attempting to write to ro mirror
dd: writing to ‘/dev/mirror_sanity/rolist_during_resync’: Operation not permitted
1+0 records in
0+0 records out
0 bytes (0 B) copied, 0.000533383 s, 0.0 kB/s



### RHEL 7.6

SCENARIO - [add_to_ro_volume_list_w_mirror_resync_in_progress]
While resync is in progress, add mirror to the read only volume list and reactivate
Create a nosync mirror then resync it so that it can be up converted
host-073: lvcreate  --type mirror -m 1 -n rolist_during_resync -L 6G mirror_sanity
Waiting until all mirror|raid volumes become fully syncd...
   0/1 mirror(s) are fully synced: ( 7.32% )
   0/1 mirror(s) are fully synced: ( 13.90% )
   0/1 mirror(s) are fully synced: ( 20.67% )
   [...]
   0/1 mirror(s) are fully synced: ( 86.20% )
   0/1 mirror(s) are fully synced: ( 93.23% )
   1/1 mirror(s) are fully synced: ( 100.00% )
Sleeping 15 sec

Resyncing rolist_during_resync mirror
current copy percent: 3.74
rolist_during_resync:  lvs --noheadings -o lv_attr mirror_sanity/rolist_during_resync
ATTR: mwi-a-m---
rolist_during_resync_mimage_0:  lvs --noheadings -o lv_attr mirror_sanity/rolist_during_resync_mimage_0
ATTR: Iwi-aom---
rolist_during_resync_mimage_1:  lvs --noheadings -o lv_attr mirror_sanity/rolist_during_resync_mimage_1
ATTR: Iwi-aom---

While resync is in progress, add this mirror to the read only volume list and reactivate
lvchange --addtag RO mirror_sanity/rolist_during_resync
lvchange -an mirror_sanity/rolist_during_resync
lvchange -ay mirror_sanity/rolist_during_resync
current copy percent: 7.58

rolist_during_resync:  lvs --noheadings -o lv_attr mirror_sanity/rolist_during_resync
ATTR: mwi-a-m---

mirror device should have (R)ead-only activation of non-read-only volume attribute set
sub:rolist_during_resync attr:mwi-a-m---


[root@host-073 ~]# grep RO /etc/lvm/lvm.conf
    read_only_volume_list = [ "@RO" ]  # edited by QA test script qe_lvmconf (on Mon Jun 25 11:39:15 CDT 2018)!

[root@host-073 ~]# lvs -a -o +devices,lv_tags
 LV                              VG            Attr       LSize  Log                         Cpy%Sync Devices                                                           LV Tags
 rolist_during_resync            mirror_sanity mwi-a-m--- 6.00g  [rolist_during_resync_mlog] 100.00   rolist_during_resync_mimage_0(0),rolist_during_resync_mimage_1(0) RO
 [rolist_during_resync_mimage_0] mirror_sanity iwi-aom--- 6.00g                                       /dev/sda2(0)
 [rolist_during_resync_mimage_1] mirror_sanity iwi-aom--- 6.00g                                       /dev/sda1(0)
 [rolist_during_resync_mlog]     mirror_sanity lwi-aom--- 4.00m                                       /dev/sdg1(0)

[root@host-073 ~]# dd if=/dev/zero of=/dev/mirror_sanity/rolist_during_resync  count=1
1+0 records in
1+0 records out
512 bytes (512 B) copied, 0.00584262 s, 87.6 kB/s


Version-Release number of selected component (if applicable):
3.10.0-906.el7.x86_64

lvm2-2.02.179-2.el7    BUILT: Thu Jun 21 08:02:34 CDT 2018
lvm2-libs-2.02.179-2.el7    BUILT: Thu Jun 21 08:02:34 CDT 2018
lvm2-cluster-2.02.179-2.el7    BUILT: Thu Jun 21 08:02:34 CDT 2018
lvm2-lockd-2.02.179-2.el7    BUILT: Thu Jun 21 08:02:34 CDT 2018
lvm2-python-boom-0.8.5-6.el7    BUILT: Mon Jun 18 01:16:13 CDT 2018
cmirror-2.02.179-2.el7    BUILT: Thu Jun 21 08:02:34 CDT 2018
device-mapper-1.02.148-2.el7    BUILT: Thu Jun 21 08:02:34 CDT 2018
device-mapper-libs-1.02.148-2.el7    BUILT: Thu Jun 21 08:02:34 CDT 2018
device-mapper-event-1.02.148-2.el7    BUILT: Thu Jun 21 08:02:34 CDT 2018
device-mapper-event-libs-1.02.148-2.el7    BUILT: Thu Jun 21 08:02:34 CDT 2018
device-mapper-persistent-data-0.7.3-3.el7    BUILT: Tue Nov 14 05:07:18 CST 2017


How reproducible:
Everytime

Comment 3 Corey Marthaler 2018-06-27 21:14:00 UTC

Looks like raid1 has this similar bug as well, however with the difference being the actual raid lv has the correct R attr, but the subvols do not.

SCENARIO (raid1) - [add_to_ro_volume_list_w_raid_resync_in_progress]                                                                                                                                                                                    
Create a raid mirror then while resyncing, add it to the read_only_volume_list and verify both the "(I)mage out-of-sync" and "(R)ead-only activation of non-read-only volume" attributes                                                                
host-092.virt.lab.msp.redhat.com: lvcreate  --type raid1 -m 1 -n rolist_during_resync -L 7G raid_sanity                                                                                                                                                 
Waiting until all mirror|raid volumes become fully syncd...                                                                                                                                                                                             
   0/1 mirror(s) are fully synced: ( 6.28% )                                                                                                                                                                                                            
   0/1 mirror(s) are fully synced: ( 18.81% )                                                                                                                                                                                                           
   0/1 mirror(s) are fully synced: ( 25.12% )                                                                                                                                                                                                           
   0/1 mirror(s) are fully synced: ( 37.67% )                                                                                                                                                                                                           
   0/1 mirror(s) are fully synced: ( 43.95% )                                                                                                                                                                                                           
   0/1 mirror(s) are fully synced: ( 56.51% )                                                                                                                                                                                                           
   0/1 mirror(s) are fully synced: ( 61.91% )                                                                                                                                                                                                           
   0/1 mirror(s) are fully synced: ( 74.05% )                                                                                                                                                                                                           
   0/1 mirror(s) are fully synced: ( 80.47% )                                                                                                                                                                                                           
   0/1 mirror(s) are fully synced: ( 88.07% )                                                                                                                                                                                                           
   0/1 mirror(s) are fully synced: ( 95.67% )                                                                                                                                                                                                           
   1/1 mirror(s) are fully synced: ( 100.00% )
Sleeping 15 sec

Resyncing rolist_during_resync raid
lvchange --resync -y raid_sanity/rolist_during_resync

While resync is in progress, add this raid to the read only volume list and reactivate
lvchange --addtag RO raid_sanity/rolist_during_resync
lvchange -an raid_sanity/rolist_during_resync
lvchange -ay raid_sanity/rolist_during_resync
       4.49% synced

raid device should have [R]ead-only activation of non-read-only volume attribute set (while image syncing [I])
sub:rolist_during_resync_rimage_0 attr:Iwi-aor---

[root@host-092 ~]# lvs -a -o +devices,lvtags
  LV                              VG            Attr       LSize  Cpy%Sync Devices                                                           LV Tags
  rolist_during_resync            raid_sanity   rRi-a-r--- 7.00g  100.00   rolist_during_resync_rimage_0(0),rolist_during_resync_rimage_1(0) RO
  [rolist_during_resync_rimage_0] raid_sanity   iwi-aor--- 7.00g           /dev/sdb1(1)
  [rolist_during_resync_rimage_1] raid_sanity   iwi-aor--- 7.00g           /dev/sdb2(1)
  [rolist_during_resync_rmeta_0]  raid_sanity   ewi-aor--- 4.00m           /dev/sdb1(0)
  [rolist_during_resync_rmeta_1]  raid_sanity   ewi-aor--- 4.00m           /dev/sdb2(0)

Comment 4 David Teigland 2018-08-02 16:25:09 UTC

read_only_volume_list doesn't work at all for mirrors (sync isn't a factor.)

The LV is activated readonly, but monitoring then reactivates it rw.  This is a regression from:

https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=fade45b1d14c90eac7c6cc41019ef946659a8fb5

commit fade45b1d14c90eac7c6cc41019ef946659a8fb5
Author: Zdenek Kabelac <zkabelac>
Date:   Sat Apr 28 22:14:47 2018 +0200

    mirror: improve table update
    
    Shift refresh of mirror table right into monitor_dev_for_events().
    Use  !vg_write_lock_held() to recognize use of lvchange/vgchange.
    (this shall change if this would no longer work, but requires
    futher some API changes).
    
    With this patch  dm mirror table is only refreshed when necassary.
    
    Also update WARNING message about mirror usage without monitoring
    and display LV name.


The following patch fixes the regression, but I don't know if there are other activation settings that should also be copied over:

diff --git a/lib/activate/activate.c b/lib/activate/activate.c
index 56ec73241bd7..70cce30cbbe9 100644
--- a/lib/activate/activate.c
+++ b/lib/activate/activate.c
@@ -1851,6 +1851,8 @@ int monitor_dev_for_events(struct cmd_context *cmd, const struct logical_volume
 
        if (!laopts)
                laopts = &zlaopts;
+       else
+               mirr_laopts.read_only = laopts->read_only;
 
        /* skip dmeventd code altogether */
        if (dmeventd_monitor_mode() == DMEVENTD_MONITOR_IGNORE)

Comment 5 David Teigland 2018-08-02 16:41:05 UTC

fixed the readonly here
https://sourceware.org/git/?p=lvm2.git;a=commit;h=5f648406b0f2e322851fce36b062a298cb3d6736

Comment 7 Corey Marthaler 2018-08-27 21:59:35 UTC

Just to be clear, as outlined in bug comment https://bugzilla.redhat.com/show_bug.cgi?id=1208269#c13 for this bug, *ALL* subvols starting in 7.6 *always* have "w" regardless of the top level volumes actual status? All we now verify is the top most volume?


SCENARIO (raid1) - [add_to_ro_volume_list_w_raid_resync_in_progress]
Create a raid mirror then while resyncing, add it to the read_only_volume_list and verify both the "(I)mage out-of-sync" and "(R)ead-only activation of non-read-only volume" attributes
hayes-02: lvcreate  --type raid1 -m 1 -n rolist_during_resync -L 7G raid_sanity
Waiting until all mirror|raid volumes become fully syncd...
   0/1 mirror(s) are fully synced: ( 27.16% )
   0/1 mirror(s) are fully synced: ( 54.40% )
   0/1 mirror(s) are fully synced: ( 81.62% )
   1/1 mirror(s) are fully synced: ( 100.00% )
Sleeping 15 sec

Resyncing rolist_during_resync raid
lvchange --resync -y raid_sanity/rolist_during_resync

While resync is in progress, add this raid to the read only volume list and reactivate
lvchange --addtag RO raid_sanity/rolist_during_resync
lvchange -an raid_sanity/rolist_during_resync
lvchange -ay raid_sanity/rolist_during_resync
        8.18% synced
raid device should have [R]ead-only activation of non-read-only volume attribute set (while image syncing [I])
sub:rolist_during_resync_rimage_0 attr:Iwi-aor---


[root@hayes-02 ~]# lvs -a -o +devices
  LV                              VG          Attr       LSize Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert Devices                                                          
  rolist_during_resync            raid_sanity rRi-a-r--- 7.00g                                    89.94            rolist_during_resync_rimage_0(0),rolist_during_resync_rimage_1(0)
  [rolist_during_resync_rimage_0] raid_sanity Iwi-aor--- 7.00g                                                     /dev/sdm1(1)                                                     
  [rolist_during_resync_rimage_1] raid_sanity Iwi-aor--- 7.00g                                                     /dev/sdl1(1)                                                     
  [rolist_during_resync_rmeta_0]  raid_sanity ewi-aor--- 4.00m                                                     /dev/sdm1(0)                                                     
  [rolist_during_resync_rmeta_1]  raid_sanity ewi-aor--- 4.00m

Comment 8 David Teigland 2018-08-28 13:52:43 UTC

I'm not sure what the meaning of rw for sub LVs is, maybe something we just shouldn't care about.  Maybe Heinz knows.

Comment 9 Heinz Mauelshagen 2018-08-29 15:04:09 UTC

Corey,

for mirror/raid, the SubLVs need to be writeable to allow for updates
during resync (mind resync causes kernel initiated write I/O).

So user write I/O can be prohibited by setting the top-level
raid/mirror LV readonly e.g. via read_only_volume_list, kernel
write I/O to the SubLVs may not.

Comment 10 Corey Marthaler 2018-08-29 20:03:17 UTC

Marking verified in latest rpms. Will also adjust scenarios to only verify top most volume attributes.

3.10.0-939.el7.x86_64

lvm2-2.02.180-6.el7    BUILT: Fri Aug 24 08:52:19 CDT 2018
lvm2-libs-2.02.180-6.el7    BUILT: Fri Aug 24 08:52:19 CDT 2018
lvm2-cluster-2.02.180-6.el7    BUILT: Fri Aug 24 08:52:19 CDT 2018
lvm2-lockd-2.02.180-6.el7    BUILT: Fri Aug 24 08:52:19 CDT 2018
lvm2-python-boom-0.9-9.el7    BUILT: Fri Aug 24 08:52:54 CDT 2018
cmirror-2.02.180-6.el7    BUILT: Fri Aug 24 08:52:19 CDT 2018
device-mapper-1.02.149-6.el7    BUILT: Fri Aug 24 08:52:19 CDT 2018
device-mapper-libs-1.02.149-6.el7    BUILT: Fri Aug 24 08:52:19 CDT 2018
device-mapper-event-1.02.149-6.el7    BUILT: Fri Aug 24 08:52:19 CDT 2018
device-mapper-event-libs-1.02.149-6.el7    BUILT: Fri Aug 24 08:52:19 CDT 2018
device-mapper-persistent-data-0.7.3-3.el7    BUILT: Tue Nov 14 05:07:18 CST 2017




# MIRROR

SCENARIO - [add_to_ro_volume_list_w_mirror_resync_in_progress]
While resync is in progress, add mirror to the read only volume list and reactivate
Create a nosync mirror then resync it so that it can be up converted
hayes-02: lvcreate  --type mirror -m 1 -n rolist_during_resync -L 6G mirror_sanity
Waiting until all mirror|raid volumes become fully syncd...
   0/1 mirror(s) are fully synced: ( 20.87% )
   0/1 mirror(s) are fully synced: ( 41.44% )
   0/1 mirror(s) are fully synced: ( 62.01% )
   0/1 mirror(s) are fully synced: ( 82.55% )
   1/1 mirror(s) are fully synced: ( 100.00% )
Sleeping 15 sec

Resyncing rolist_during_resync mirror
current copy percent: 8.98
Check that mirror device has read [w]rite activation attribute set
        rolist_during_resync:mwi-a-m---
        rolist_during_resync_mimage_0:Iwi-aom---
        rolist_during_resync_mimage_1:Iwi-aom---

While resync is in progress, add this mirror to the read only volume list and reactivate
lvchange --addtag RO mirror_sanity/rolist_during_resync
lvchange -an mirror_sanity/rolist_during_resync
lvchange -ay mirror_sanity/rolist_during_resync
Check that mirror device has (R)ead-only activation of non-read-only volume attribute set
        19.37% synced
        rolist_during_resync:mRi-a-m---

Attempting to write to ro mirror
dd: writing to ‘/dev/mirror_sanity/rolist_during_resync’: Operation not permitted
1+0 records in
0+0 records out
0 bytes (0 B) copied, 0.000181976 s, 0.0 kB/s
Deactivating mirror rolist_during_resync... and removing



# RAID1

SCENARIO (raid1) - [add_to_ro_volume_list_w_raid_resync_in_progress]
Create a raid mirror then while resyncing, add it to the read_only_volume_list and verify both the "(I)mage out-of-sync" and "(R)ead-only activation of non-read-only volume" attributes
hayes-02: lvcreate  --type raid1 -m 1 -n rolist_during_resync -L 7G raid_sanity
Waiting until all mirror|raid volumes become fully syncd...
   0/1 mirror(s) are fully synced: ( 26.93% )
   0/1 mirror(s) are fully synced: ( 54.52% )
   0/1 mirror(s) are fully synced: ( 81.77% )
   1/1 mirror(s) are fully synced: ( 100.00% )
Sleeping 15 sec

Resyncing rolist_during_resync raid
lvchange --resync -y raid_sanity/rolist_during_resync
Check that raid device has read [w]rite activation attribute set (while image syncing [I])
        rolist_during_resync:rwi-a-r---
        rolist_during_resync_rimage_0:Iwi-aor---
        rolist_during_resync_rimage_1:Iwi-aor---
        rolist_during_resync_rmeta_0:ewi-aor---
        rolist_during_resync_rmeta_1:ewi-aor---

While resync is in progress, add this raid to the read only volume list and reactivate
lvchange --addtag RO raid_sanity/rolist_during_resync
lvchange -an raid_sanity/rolist_during_resync
lvchange -ay raid_sanity/rolist_during_resync
        8.18% synced
Check that raid device has [R]ead-only activation of non-read-only volume attribute set (while image syncing [I])
        rolist_during_resync:rRi-a-r---

Attempting to write to ro raid
dd: writing to ‘/dev/raid_sanity/rolist_during_resync’: Operation not permitted
1+0 records in
0+0 records out
0 bytes (0 B) copied, 0.000171107 s, 0.0 kB/s

perform raid scrubbing (lvchange --syncaction repair) on raid raid_sanity/rolist_during_resync
  raid_sanity/rolist_during_resync state is currently "resync".  Unable to switch to "repair".
Waiting until all mirror|raid volumes become fully syncd...
   0/1 mirror(s) are fully synced: ( 48.79% )
   0/1 mirror(s) are fully synced: ( 76.32% )
   1/1 mirror(s) are fully synced: ( 100.00% )
Sleeping 15 sec

Deactivating raid rolist_during_resync... and removing

Comment 12 errata-xmlrpc 2018-10-30 11:03:47 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3193

Note You need to log in before you can comment on or make changes to this bug.