Bug 855425

Summary: 3.1 - Pool upgrade to DC 3.1 generates multiple lease files for the same template
Product: Red Hat Enterprise Linux 6 Reporter: Federico Simoncelli <fsimonce>
Component: vdsmAssignee: Federico Simoncelli <fsimonce>
Status: CLOSED ERRATA QA Contact: vvyazmin <vvyazmin>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.3CC: abaron, bazulay, chetan, hateya, iheim, ilvovsky, lpeer, thildred, ykaul
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: storage
Fixed In Version: vdsm-4.9.6-40.0 Doc Type: Bug Fix
Doc Text:
In a beta release of Red Hat Enterprise Virtualization, upgraded file-based storage domains had multiple lease files generated for templates that were used in multiple virtual machines. Now, one lease file is generated for each template no matter how many virtual machines use it.
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-12-04 19:10:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Federico Simoncelli 2012-09-07 17:19:30 UTC
Description of problem:
When upgrading a file domain (NFS) from DC 3.0 to DC 3.1 (domain version 0 => 3) the volume lease files of the templates are different in each image (based on such template):

DC 3.0
======
Notice the f959ece7-9cef-45b2-98c8-2030de7e49eb inodes (volume and meta file), they are the same [1704082]/[1704084] in both the template (6d5aec33-b59a-4962-876c-de2ea23e7a9f) and the image based on such template (cc07c3a9-59b8-4e00-be63-c904149601c8):

# tree --inodes /rhev/data-center/316b44c1-6956-4b43-9374-db8ceb2651e4/ae8cf5a8-04a0-4462-9a5d-ebab7bdf71af/images/
/rhev/data-center/316b44c1-6956-4b43-9374-db8ceb2651e4/ae8cf5a8-04a0-4462-9a5d-ebab7bdf71af/images/
|-- [1704077]  6d5aec33-b59a-4962-876c-de2ea23e7a9f
|   |-- [1704082]  f959ece7-9cef-45b2-98c8-2030de7e49eb
|   `-- [1704084]  f959ece7-9cef-45b2-98c8-2030de7e49eb.meta
|-- [1704080]  cc07c3a9-59b8-4e00-be63-c904149601c8
|   |-- [1704085]  41e86ac1-8967-491a-b344-1d65cdbd819e
|   |-- [1704087]  41e86ac1-8967-491a-b344-1d65cdbd819e.meta
|   |-- [1704082]  f959ece7-9cef-45b2-98c8-2030de7e49eb
|   `-- [1704084]  f959ece7-9cef-45b2-98c8-2030de7e49eb.meta
`-- [1704073]  f8a52125-3ff1-4d9f-a614-83b60f46616f
    |-- [1704076]  9df738cc-a950-4322-9f45-834f1f39f08c
    `-- [1704074]  9df738cc-a950-4322-9f45-834f1f39f08c.meta

Upgraded to DC 3.1:
Notice the f959ece7-9cef-45b2-98c8-2030de7e49eb inodes (volume, meta and lease file), the lease files inode of the template [1704089] is different from the one in the image [1704081].

# tree --inodes /rhev/data-center/316b44c1-6956-4b43-9374-db8ceb2651e4/ae8cf5a8-04a0-4462-9a5d-ebab7bdf71af/images/
/rhev/data-center/316b44c1-6956-4b43-9374-db8ceb2651e4/ae8cf5a8-04a0-4462-9a5d-ebab7bdf71af/images/
|-- [1704077]  6d5aec33-b59a-4962-876c-de2ea23e7a9f
|   |-- [1704082]  f959ece7-9cef-45b2-98c8-2030de7e49eb
|   |-- [1704089]  f959ece7-9cef-45b2-98c8-2030de7e49eb.lease
|   `-- [1704084]  f959ece7-9cef-45b2-98c8-2030de7e49eb.meta
|-- [1704080]  cc07c3a9-59b8-4e00-be63-c904149601c8
|   |-- [1704085]  41e86ac1-8967-491a-b344-1d65cdbd819e
|   |-- [1704088]  41e86ac1-8967-491a-b344-1d65cdbd819e.lease
|   |-- [1704087]  41e86ac1-8967-491a-b344-1d65cdbd819e.meta
|   |-- [1704082]  f959ece7-9cef-45b2-98c8-2030de7e49eb
|   |-- [1704081]  f959ece7-9cef-45b2-98c8-2030de7e49eb.lease
|   `-- [1704084]  f959ece7-9cef-45b2-98c8-2030de7e49eb.meta
`-- [1704073]  f8a52125-3ff1-4d9f-a614-83b60f46616f
    |-- [1704076]  9df738cc-a950-4322-9f45-834f1f39f08c
    |-- [1704075]  9df738cc-a950-4322-9f45-834f1f39f08c.lease
    `-- [1704074]  9df738cc-a950-4322-9f45-834f1f39f08c.meta


Version-Release number of selected component (if applicable):
vdsm-4.9.6-33.0.el6_3

How reproducible:
100%

Steps to Reproduce:
1. convert a NFS DC 3.0 to DC 3.1
2. check the lease files present in the images based on templates
  
Actual results:
The lease file inodes in the template and in the images based on such template are different.

Expected results:
The lease file inodes in the template and in the images based on such template should be the same.

Comment 3 Federico Simoncelli 2012-10-15 12:43:22 UTC
commit bd488e9c991021b8a39c9469eb6e66da7928e175
Author: Federico Simoncelli <fsimonce>
Date:   Wed Oct 10 07:41:51 2012 -0400

    upgrade: handle the templates lease files

http://gerrit.ovirt.org/#/c/8271/

Comment 5 vvyazmin@redhat.com 2012-11-05 11:37:03 UTC
Verified on RHEVM 3.1 - SI24

RHEVM: rhevm-3.1.0-26.el6ev.noarch
VDSM: vdsm-4.9.6-41.0.el6_3.x86_64
LIBVIRT: libvirt-0.9.10-21.el6_3.5.x86_64
QEMU & KVM: qemu-kvm-rhev-0.12.1.2-2.295.el6_3.4.x86_64
SANLOCK: sanlock-2.3-4.el6_3.x86_64


NFS DC with 3.0 compatibility version

[root@cougar05 ~]# tree --inodes /rhev/data-center/fe59837b-45c0-4040-979b-4476b641adbc/da275a3a-ae7c-4233-965b-91e6940a2482/images
/rhev/data-center/fe59837b-45c0-4040-979b-4476b641adbc/da275a3a-ae7c-4233-965b-91e6940a2482/images
├── [   3460]  0a9476b0-c23f-4d5e-a8af-01dc452ee14b
│   ├── [   3472]  a14ced5a-2817-4486-92ed-117a22a9b3ef
│   ├── [   3481]  a14ced5a-2817-4486-92ed-117a22a9b3ef.lease
│   └── [   3488]  a14ced5a-2817-4486-92ed-117a22a9b3ef.meta
├── [   3340]  28aab435-3a03-4c15-a0b6-6d6d4aa12d65
│   ├── [   3412]  27588fa6-e33f-4718-8ee5-c192587aa456
│   ├── [   3428]  27588fa6-e33f-4718-8ee5-c192587aa456.lease
│   ├── [   3427]  27588fa6-e33f-4718-8ee5-c192587aa456.meta
│   ├── [   3352]  92528614-1b07-4d7f-9ab5-c6875ac199c6
│   ├── [   3361]  92528614-1b07-4d7f-9ab5-c6875ac199c6.lease
│   └── [   3419]  92528614-1b07-4d7f-9ab5-c6875ac199c6.meta
├── [   3520]  528a9906-4a7a-4b46-a03d-b656a21035d2
│   ├── [   3585]  73e506c5-e207-4188-baf7-3c6ca973a583
│   ├── [   3601]  73e506c5-e207-4188-baf7-3c6ca973a583.lease
│   ├── [   3600]  73e506c5-e207-4188-baf7-3c6ca973a583.meta
│   ├── [   3532]  b21c3e4c-1d26-4448-8f52-bee751b4eb3b
│   ├── [   3541]  b21c3e4c-1d26-4448-8f52-bee751b4eb3b.lease
│   └── [   3592]  b21c3e4c-1d26-4448-8f52-bee751b4eb3b.meta
├── [   3228]  80b5cd2b-c899-4519-9ec1-02e28a70e3a5
│   ├── [   3240]  e744c4e4-41f7-43b5-b33b-f0dd38e6847d
│   ├── [   3249]  e744c4e4-41f7-43b5-b33b-f0dd38e6847d.lease
│   └── [   3248]  e744c4e4-41f7-43b5-b33b-f0dd38e6847d.meta
└── [   3287]  b610dc23-a232-4381-a991-2e00220a0c48
    ├── [   3299]  d211f849-ba23-4e38-bb3d-daa3c1786d55
    ├── [   3308]  d211f849-ba23-4e38-bb3d-daa3c1786d55.lease
    └── [   3375]  d211f849-ba23-4e38-bb3d-daa3c1786d55.meta


After upgrade to NFS DC with 3.1 compatibility version

[root@cougar05 ~]# tree --inodes /rhev/data-center/fe59837b-45c0-4040-979b-4476b641adbc/da275a3a-ae7c-4233-965b-91e6940a2482/images
/rhev/data-center/fe59837b-45c0-4040-979b-4476b641adbc/da275a3a-ae7c-4233-965b-91e6940a2482/images
├── [   3460]  0a9476b0-c23f-4d5e-a8af-01dc452ee14b
│   ├── [   3472]  a14ced5a-2817-4486-92ed-117a22a9b3ef
│   ├── [   3481]  a14ced5a-2817-4486-92ed-117a22a9b3ef.lease
│   └── [   3488]  a14ced5a-2817-4486-92ed-117a22a9b3ef.meta
├── [   3340]  28aab435-3a03-4c15-a0b6-6d6d4aa12d65
│   ├── [   3412]  27588fa6-e33f-4718-8ee5-c192587aa456
│   ├── [   3428]  27588fa6-e33f-4718-8ee5-c192587aa456.lease
│   ├── [   3427]  27588fa6-e33f-4718-8ee5-c192587aa456.meta
│   ├── [   3352]  92528614-1b07-4d7f-9ab5-c6875ac199c6
│   ├── [   3361]  92528614-1b07-4d7f-9ab5-c6875ac199c6.lease
│   └── [   3419]  92528614-1b07-4d7f-9ab5-c6875ac199c6.meta
├── [   3520]  528a9906-4a7a-4b46-a03d-b656a21035d2
│   ├── [   3585]  73e506c5-e207-4188-baf7-3c6ca973a583
│   ├── [   3601]  73e506c5-e207-4188-baf7-3c6ca973a583.lease
│   ├── [   3600]  73e506c5-e207-4188-baf7-3c6ca973a583.meta
│   ├── [   3532]  b21c3e4c-1d26-4448-8f52-bee751b4eb3b
│   ├── [   3541]  b21c3e4c-1d26-4448-8f52-bee751b4eb3b.lease
│   └── [   3592]  b21c3e4c-1d26-4448-8f52-bee751b4eb3b.meta
├── [   3228]  80b5cd2b-c899-4519-9ec1-02e28a70e3a5
│   ├── [   3240]  e744c4e4-41f7-43b5-b33b-f0dd38e6847d
│   ├── [   3249]  e744c4e4-41f7-43b5-b33b-f0dd38e6847d.lease
│   └── [   3248]  e744c4e4-41f7-43b5-b33b-f0dd38e6847d.meta
└── [   3287]  b610dc23-a232-4381-a991-2e00220a0c48
    ├── [   3299]  d211f849-ba23-4e38-bb3d-daa3c1786d55
    ├── [   3308]  d211f849-ba23-4e38-bb3d-daa3c1786d55.lease
    └── [   3375]  d211f849-ba23-4e38-bb3d-daa3c1786d55.meta

[root@kipi-rhevm ~]# psql -U postgres engine -c 'select vm_names,image_group_id from all_disks;'
      vm_names       |            image_group_id            
---------------------+--------------------------------------
 Template-01-RHEL6.3 | b610dc23-a232-4381-a991-2e00220a0c48
 vm-02               | 28aab435-3a03-4c15-a0b6-6d6d4aa12d65
 vm-01               | 80b5cd2b-c899-4519-9ec1-02e28a70e3a5
 vm-bON-Template     | 0a9476b0-c23f-4d5e-a8af-01dc452ee14b
 vm-bON-Template-02  | 528a9906-4a7a-4b46-a03d-b656a21035d2
(5 rows)

Comment 6 Tim Hildred 2012-11-22 11:04:52 UTC
In an beta release of Red Hat Enterprise Virtualization, upgraded file-based storage domains had multiple lease files generated for templates that were used in multiple virtual machines. 

Now, one lease file is generated for each template no matter how many virtual machines use it.

Comment 8 errata-xmlrpc 2012-12-04 19:10:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-1508.html