Bug 452121 - cmirror fails to report fully synced after creation on one node in cluster
cmirror fails to report fully synced after creation on one node in cluster
Status: CLOSED DEFERRED
Product: Red Hat Cluster Suite
Classification: Red Hat
Component: cmirror-kernel (Show other bugs)
4
All Linux
medium Severity high
: ---
: ---
Assigned To: Jonathan Earl Brassow
Cluster QE
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2008-06-19 10:50 EDT by Corey Marthaler
Modified: 2013-09-23 11:32 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-09-23 11:32:43 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
log from taft-03 (15.86 KB, text/plain)
2008-06-19 11:20 EDT, Corey Marthaler
no flags Details
log (with extra debugging) from taft-03 when this happened (13.78 KB, text/plain)
2008-06-25 10:47 EDT, Corey Marthaler
no flags Details
log (with extra debugging) from taft-04 when this happened (279.10 KB, text/plain)
2008-06-25 10:48 EDT, Corey Marthaler
no flags Details
log (with extra debugging) from taft-01 when this happened (7.44 KB, text/plain)
2008-06-25 10:58 EDT, Corey Marthaler
no flags Details
log (with extra debugging) from taft-02 when this happened (791.97 KB, text/plain)
2008-06-25 11:51 EDT, Corey Marthaler
no flags Details

  None (edit)
Description Corey Marthaler 2008-06-19 10:50:13 EDT
Description of problem:
This may be related to bz 240864.
I created two cmirrors and one of them is reported as 0.00% synced on taft-03.


Creating mirror(s) on taft-04...
taft-04: lvcreate -m 2 -n syncd_secondary_3legs_1 -L 800M helter_skelter
/dev/sdh1:0-1000 /dev/sde1:0-1000 /dev/sdf1:0-1000 /dev/sdg1:0-150
taft-04: lvcreate -m 2 -n syncd_secondary_3legs_2 -L 800M helter_skelter
/dev/sdh1:0-1000 /dev/sde1:0-1000 /dev/sdf1:0-1000 /dev/sdg1:0-150

Waiting until all mirrors become fully syncd...
        0/2 mirror(s) are fully synced: ( 1=8.00% 2=0.50% )
        0/2 mirror(s) are fully synced: ( 1=25.00% 2=16.50% )
        0/2 mirror(s) are fully synced: ( 1=42.50% 2=32.50% )
        0/2 mirror(s) are fully synced: ( 1=59.00% 2=48.00% )
        0/2 mirror(s) are fully synced: ( 1=76.50% 2=64.00% )
        0/2 mirror(s) are fully synced: ( 1=93.50% 2=81.00% )
        1/2 mirror(s) are fully synced: ( 1=100.00% 2=98.50% )
        2/2 mirror(s) are fully synced: ( 1=100.00% 2=100.00% )
syncd_secondary_3legs_2 is not fully sync'ed on taft-03 (0.00%)

[root@taft-01 ~]# lvs
  LV                      VG             Attr   LSize   Origin Snap%  Move Log 
                        Copy%  Convert
  syncd_secondary_3legs_1 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_1_mlog 100.00
  syncd_secondary_3legs_2 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_2_mlog 100.00

[root@taft-02 ~]# lvs
  LV                      VG             Attr   LSize   Origin Snap%  Move Log 
                        Copy%  Convert
  syncd_secondary_3legs_1 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_1_mlog 100.00
  syncd_secondary_3legs_2 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_2_mlog 100.00

[root@taft-03 ~]# lvs
  LV                      VG             Attr   LSize   Origin Snap%  Move Log 
                        Copy%  Convert
  syncd_secondary_3legs_1 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_1_mlog 100.00
  syncd_secondary_3legs_2 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_2_mlog   0.00

[root@taft-04 ~]# lvs
  LV                      VG             Attr   LSize   Origin Snap%  Move Log 
                        Copy%  Convert
  syncd_secondary_3legs_1 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_1_mlog 100.00
  syncd_secondary_3legs_2 helter_skelter mwi-a- 800.00M                   
syncd_secondary_3legs_2_mlog 100.00


2.6.9-71.ELsmp

lvm2-2.02.37-3.el4    BUILT: Thu Jun 12 10:09:19 CDT 2008
lvm2-cluster-2.02.37-3.el4    BUILT: Thu Jun 12 10:22:07 CDT 2008
device-mapper-1.02.25-2.el4    BUILT: Mon Jun  9 09:28:41 CDT 2008
cmirror-1.0.1-1    BUILT: Tue Jan 30 17:28:02 CST 2007
cmirror-kernel-2.6.9-41.4    BUILT: Tue Jun  3 13:54:29 CDT 2008
Comment 1 Corey Marthaler 2008-06-19 11:20:03 EDT
Created attachment 309850 [details]
log from taft-03
Comment 2 Corey Marthaler 2008-06-25 09:31:09 EDT
Hit this last night while attempting to reproduce bz 450939.

Creating mirror(s) on taft-04...
taft-04: lvcreate --corelog -m 1 -n syncd_primary_core_2legs_1 -L 800M
helter_skelter /dev/sdf1:0-1000 /dev/sdh1:0-1000
taft-04: lvcreate --corelog -m 1 -n syncd_primary_core_2legs_2 -L 800M
helter_skelter /dev/sdf1:0-1000 /dev/sdh1:0-1000

Waiting until all mirrors become fully syncd...
        0/2 mirror(s) are fully synced: ( 1=11.00% 2=0.00% )
        0/2 mirror(s) are fully synced: ( 1=33.00% 2=22.00% )
        0/2 mirror(s) are fully synced: ( 1=53.50% 2=42.50% )
        0/2 mirror(s) are fully synced: ( 1=74.50% 2=63.50% )
        0/2 mirror(s) are fully synced: ( 1=95.50% 2=84.00% )
        2/2 mirror(s) are fully synced: ( 1=100.00% 2=100.00% )
syncd_primary_core_2legs_2 is not fully sync'ed on taft-01 (0.00%)


[root@taft-01 ~]# lvs
  LV                         VG             Attr   LSize   Origin Snap%  Move
Log Copy%  Convert
  syncd_primary_core_2legs_1 helter_skelter mwi-a- 800.00M                     
  100.00                  
  syncd_primary_core_2legs_2 helter_skelter mwi-a- 800.00M                     
    0.00                  


[root@taft-02 ~]# lvs
  LV                         VG             Attr   LSize   Origin Snap%  Move
Log Copy%  Convert
  syncd_primary_core_2legs_1 helter_skelter mwi-a- 800.00M                     
  100.00                  
  syncd_primary_core_2legs_2 helter_skelter mwi-a- 800.00M                     
  100.00                  


[root@taft-03 ~]# lvs
  LV                         VG             Attr   LSize   Origin Snap%  Move
Log Copy%  Convert
  syncd_primary_core_2legs_1 helter_skelter mwi-a- 800.00M                     
  100.00                  
  syncd_primary_core_2legs_2 helter_skelter mwi-a- 800.00M                     
    0.00                  


[root@taft-04 ~]# lvs
  LV                         VG             Attr   LSize   Origin Snap%  Move
Log Copy%  Convert
  syncd_primary_core_2legs_1 helter_skelter mwi-a- 800.00M                     
  100.00                  
  syncd_primary_core_2legs_2 helter_skelter mwi-a- 800.00M                     
  100.00                  
Comment 3 Corey Marthaler 2008-06-25 10:47:25 EDT
Created attachment 310272 [details]
log (with extra debugging) from taft-03 when this happened
Comment 4 Corey Marthaler 2008-06-25 10:48:00 EDT
Created attachment 310273 [details]
log (with extra debugging) from taft-04 when this happened
Comment 5 Corey Marthaler 2008-06-25 10:58:31 EDT
Created attachment 310277 [details]
log (with extra debugging) from taft-01 when this happened
Comment 6 Corey Marthaler 2008-06-25 11:51:57 EDT
Created attachment 310279 [details]
log (with extra debugging) from taft-02 when this happened
Comment 7 Corey Marthaler 2008-07-09 15:01:53 EDT
Hit this after an upconvert.

Up converting linear(s) back to mirror(s) on taft-04...
taft-04: lvconvert --corelog -m 1 helter_skelter/nonsyncd_secondary_core_2legs_1
/dev/sde1:0-1000 /dev/sdf1:0-1000
taft-04: lvconvert --corelog -m 1 helter_skelter/nonsyncd_secondary_core_2legs_2
/dev/sde1:0-1000 /dev/sdf1:0-1000


[root@taft-01 sbin]# lvs
  LV                              VG             Attr   LSize   Origin Snap% 
Move Log Copy%  Convert
  LogVol00                        VolGroup00     -wi-ao  58.34G
  LogVol01                        VolGroup00     -wi-ao   9.75G
  nonsyncd_secondary_core_2legs_1 helter_skelter mwi-ao 800.00M                
       100.00
  nonsyncd_secondary_core_2legs_2 helter_skelter mwi-ao 800.00M                
       100.00


[root@taft-02 sbin]# lvs
  LV                              VG             Attr   LSize   Origin Snap% 
Move Log Copy%  Convert
  LogVol00                        VolGroup00     -wi-ao  58.34G
  LogVol01                        VolGroup00     -wi-ao   9.75G
  nonsyncd_secondary_core_2legs_1 helter_skelter mwi-ao 800.00M                
       100.00
  nonsyncd_secondary_core_2legs_2 helter_skelter mwi-ao 800.00M                
         0.00


[root@taft-03 sbin]# lvs
  LV                              VG             Attr   LSize   Origin Snap% 
Move Log Copy%  Convert
  LogVol00                        VolGroup00     -wi-ao  58.34G
  LogVol01                        VolGroup00     -wi-ao   9.75G
  nonsyncd_secondary_core_2legs_1 helter_skelter mwi-ao 800.00M                
       100.00
  nonsyncd_secondary_core_2legs_2 helter_skelter mwi-ao 800.00M                
         0.00


[root@taft-04 sbin]# lvs
  LV                              VG             Attr   LSize   Origin Snap% 
Move Log Copy%  Convert
  LogVol00                        VolGroup00     -wi-ao  58.34G
  LogVol01                        VolGroup00     -wi-ao   9.75G
  nonsyncd_secondary_core_2legs_1 helter_skelter mwi-ao 800.00M                
       100.00
  nonsyncd_secondary_core_2legs_2 helter_skelter mwi-ao 800.00M                
         0.00
Comment 8 Corey Marthaler 2008-07-09 15:12:47 EDT
These are the only "odd" messages during this failure, on taft-01 (the only node
that reports the 2nd mirror as 100%):

dm-cmirror: Told to clear recovery on wrong region 0/18446744073709551615
dm-cmirror: Unable to notify server of sync state change
dm-cmirror: Told to clear recovery on wrong region 0/18446744073709551615
dm-cmirror: Unable to notify server of sync state change
dm-cmirror: Told to clear recovery on wrong region 0/18446744073709551615
dm-cmirror: Unable to notify server of sync state change
dm-cmirror: Told to clear recovery on wrong region 0/18446744073709551615
dm-cmirror: Unable to notify server of sync state change
dm-cmirror: Told to clear recovery on wrong region 0/18446744073709551615
dm-cmirror: Unable to notify server of sync state change
Comment 11 Corey Marthaler 2011-01-17 15:24:32 EST
FWIW, this issue still exists and continues to cause basic cmirror regression
tests to fail.

2.6.9-94.ELsmp

lvm2-2.02.42-9.el4    BUILT: Thu Oct 21 15:49:57 CDT 2010
lvm2-cluster-2.02.42-9.el4    BUILT: Thu Oct 21 15:46:55 CDT 2010
device-mapper-1.02.28-3.el4    BUILT: Thu Mar  4 14:48:16 CST 2010
cmirror-1.0.2-1.el4    BUILT: Thu Feb 26 15:29:27 CST 2009
cmirror-kernel-2.6.9-43.14.el4    BUILT: Wed Dec 22 16:24:19 CST 2010
Comment 12 Lon Hohberger 2013-09-23 11:32:43 EDT
The Red Hat Cluster Suite product is past end-of-life; closing.

Note You need to log in before you can comment on or make changes to this bug.