Bug 1200206
| Summary: | Guest fail to restore when disk is iscsi lun using authenticate pool/volume source with direct mode | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Pei Zhang <pzhang> |
| Component: | libvirt | Assignee: | John Ferlan <jferlan> |
| Status: | CLOSED ERRATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 7.1 | CC: | dyuan, jdenemar, jferlan, mzhan, rbalakri, xuzhang, yanyang |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | libvirt-1.2.17-1.el7 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-11-19 06:19:48 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
The real problem is that <auth> field will be automatically generated in domain's disk element when vm starts up. However, as described about 'auth' element in libvirt.org, it seems that auth element is NOT supported for a disk type 'volume'. e.g.
auth
The auth element is supported for a disk type "network" that is using a source element with the protocol attributes "rbd" or "iscsi".
If I defined an iscsi type pool not specifying secret type in source element, secret type will be 'null' in the automatically generated 'auth' element in domain's disk element. If I defined an iscsi type pool specifying secret type with 'iscsi' in source element, secret type will be 'iscsi' in the automatically generated 'auth' element in domain's disk element.
For example:
scenario 1:
1. define and start iscsi pool with secret type 'null'
# virsh pool-dumpxml iscsi
<pool type='iscsi'>
<name>iscsi</name>
<uuid>4e7eed01-b53e-41c1-9d3b-41c23955496c</uuid>
<capacity unit='bytes'>64424509440</capacity>
<allocation unit='bytes'>64424509440</allocation>
<available unit='bytes'>0</available>
<source>
<host name='10.66.x.x' port='3260'/>
<device path='iqn.2008-09.5.165.3ffe.104:server.target1'/>
<auth type='chap' username='rhat'>
<secret usage='libvirtiscsi'/>
</auth>
</source>
<target>
<path>/dev/disk/by-path</path>
<permissions>
<mode>0755</mode>
<owner>-1</owner>
<group>-1</group>
</permissions>
</target>
2. start vm with disk type 'volume' and source mode 'direct'
<disk type='volume' device='disk'>
<driver name='qemu' type='raw'/>
<source pool='iscsi' volume='unit:0:0:1' mode='direct'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</disk>
</pool>
3. check the domain xml, the auth element is generated with secret type 'null'
<disk type='volume' device='disk'>
<driver name='qemu' type='raw'/>
<auth type='chap' username='rhat'>
<secret usage='libvirtiscsi'/>
</auth>
<source pool='iscsi' volume='unit:0:0:1' mode='direct'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</disk>
4. create snapshot
# virsh snapshot-create-as virt-tests-vm1 --disk-only s1 --diskspec vda,file=/tmp/s1
error: unsupported configuration: invalid secret type (null)
Scenario 2
1. define and start iscsi pool with secret type 'null'
# virsh pool-dumpxml iscsi
<pool type='iscsi'>
<name>iscsi</name>
<uuid>4e7eed01-b53e-41c1-9d3b-41c23955496c</uuid>
<capacity unit='bytes'>64424509440</capacity>
<allocation unit='bytes'>64424509440</allocation>
<available unit='bytes'>0</available>
<source>
<host name='10.66.x.x' port='3260'/>
<device path='iqn.2008-09.5.165.3ffe.104:server.target1'/>
<auth type='chap' username='rhat'>
<secret type='iscsi' usage='libvirtiscsi'/>
</auth>
</source>
<target>
<path>/dev/disk/by-path</path>
<permissions>
<mode>0755</mode>
<owner>-1</owner>
<group>-1</group>
</permissions>
</target>
2. start vm with disk type 'volume' and source mode 'direct'
<disk type='volume' device='disk'>
<driver name='qemu' type='raw'/>
<source pool='iscsi' volume='unit:0:0:1' mode='direct'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</disk>
</pool>
3. check the domain xml, the auth element is generated with secret type 'null'
<disk type='volume' device='disk'>
<driver name='qemu' type='raw'/>
<auth type='chap' username='rhat'>
<secret type='iscsi' usage='libvirtiscsi'/>
</auth>
<source pool='iscsi' volume='unit:0:0:1' mode='direct'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</disk>
4. create snapshot
# virsh snapshot-create-as virt-tests-vm1 --disk-only s1 --diskspec vda,file=/tmp/s1
error: internal error: invalid secret type 'iscsi'
Patch posted upstream: http://www.redhat.com/archives/libvir-list/2015-June/msg00329.html Patch has been pushed upstream:
commit 1feaccf0001794e167b0374abf75d97024414e27
Author: John Ferlan <jferlan>
Date: Mon Jun 8 18:41:28 2015 -0400
storage: Need to set secrettype for direct iscsi disk volume
Commit id '1b4eaa61' added the ability to have a mode='direct' for
an iscsi disk volume. It relied on virStorageTranslateDiskSourcePool
in order to copy any disk source pool authentication information to
the direct disk volume, but it neglected to also copy the 'secrettype'
field which ends up being used in the domain volume formatting code.
Adding a secrettype for this case will allow for proper formatting later
and allow disk snapshotting to work properly
Additionally libvirtd restart processing would fail to find the domain
since the translation processing code is run after domain xml processing,
so handle the the case where the authdef could have an empty secrettype
field when processing the auth and additionally ignore performing the
actual and expected auth secret type checks for a DISK_VOLUME since that
data will be reassembled later during translation processing of the
running domain.
$ git describe 1feaccf0001794e167b0374abf75d97024414e27
v1.2.16-148-g1feaccf
$
Verify version:
libvirt-1.2.17-5.el7.x86_64
qemu-kvm-rhev-2.3.0-18.el7.x86_64
steps:
1.prepare a iscsi server with secret conf.
2.on iscsi client create iscsi pool with secret info.
3.check pool and volume info.
# virsh pool-dumpxml iscsi-pool
<pool type='iscsi'>
<name>iscsi-pool</name>
<uuid>73a72456-e7ef-460a-99f7-926a6f20a74c</uuid>
<capacity unit='bytes'>32212254720</capacity>
<allocation unit='bytes'>32212254720</allocation>
<available unit='bytes'>0</available>
<source>
<host name='$IP' port='3260'/>
<device path='iqn.2008-09.5.165.3ffe.104:server.target3'/>
<auth type='chap' username='redhat'>
<secret usage='libvirtiscsi'/>
</auth>
</source>
<target>
<path>/dev/disk/by-path</path>
</target>
</pool>
# virsh vol-list iscsi-pool
Name Path
------------------------------------------------------------------------------
unit:0:0:1 /dev/disk/by-path/ip-$IP:3260-iscsi-iqn.2008-09.5.165.3ffe.104:server.target3-lun-1
4.define and start a guest like following
# virsh dumpxml r708 --inactive |grep disk -A 9
<disk type='volume' device='disk'>
<driver name='qemu' type='raw'/>
<source pool='iscsi-pool' volume='unit:0:0:1' mode='direct'/>
<target dev='sda' bus='scsi'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
5.do save and restore
# virsh save r708 r708.save
Domain r708 saved to r708.save
# virsh restore r708.save
Domain restored from r708.save
# virsh dumpxml r708|grep disk -A 9
<disk type='volume' device='disk'>
<driver name='qemu' type='raw'/>
<auth type='chap' username='redhat'>
<secret type='iscsi' usage='libvirtiscsi'/>
</auth>
<source pool='iscsi-pool' volume='unit:0:0:1' mode='direct'/>
<backingStore/>
<target dev='sda' bus='scsi'/>
<alias name='scsi0-0-0-0'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
6.do external snapshot
# virsh snapshot-create-as r708 --disk-only s1 --diskspec sda,file=/tmp/s1
Domain snapshot s1 created
# virsh snapshot-info r708 s1
Name: s1
Domain: r708
Current: yes
State: disk-snapshot
Location: external
Parent: -
Children: 0
Descendants: 0
Metadata: yes
7. restart libvirtd then guest should still running.
# service libvirtd restart
Redirecting to /bin/systemctl restart libvirtd.service
# virsh list
Id Name State
----------------------------------------------------
6 r708 running
Now, it works well.
And John, a small issue need your help. Thanks in advance.
As you said in patch :
"For the domain disk, there is no <auth> 'type' field, while there is a
<secret> 'type' field."
I found that "<auth type='chap' ...>" will be added into domain xml automatically. details as step 4 and step 5. This xml will validate fail via virt-xml-validate .
I was wondering if this issue also should be modified.
Thanks.
(In reply to Pei Zhang from comment #5) > > And John, a small issue need your help. Thanks in advance. > As you said in patch : > "For the domain disk, there is no <auth> 'type' field, while there is a > <secret> 'type' field." > I found that "<auth type='chap' ...>" will be added into domain xml > automatically. details as step 4 and step 5. This xml will validate fail via > virt-xml-validate . > I was wondering if this issue also should be modified. > Thanks. Hmm... yeah that shows up I guess. Beyond the XML validation failing virt-xml-validate it's no big deal. The reason it's there has more to do with common code shared between pool and disk as well as how the disk data is filled in from the pool data. Feel free to generate a RHEL7.3 bug to track a fix. Thanks for your info John. File a new bug on rhel7.3 to track <auth> type issue. https://bugzilla.redhat.com/show_bug.cgi?id=1256999 and according to comment 5, move this bug to verified. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2015-2202.html |
Description of problem: Guest fail to restore when disk is iscsi lun using authenticate pool/volume source with direct mode. Guest save and restore successfully when using host mode. Version-Release number of selected component (if applicable): libvirt-1.2.8-16.el7.x86_64 qemu-kvm-rhev-2.2.0-5.el7.x86_64 kernel-3.10.0-229.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1.Prepare authenticate iscsi target server #vim /etc/tgt/targets.conf ...... <target iqn.2008-09.5.165.3ffe.104:server.target1> backing-store /mnt/server/iscsi/iscsi.img incominguser rhat rhatrhat </target> ...... #service tgtd restart 2.on iscsi client using authenticate iscsi lun. 2.1 define a secret # cat iscsi-secret.xml <secret ephemeral='no' private='yes'> <description>iSCSI secret</description> <auth type='chap' username='rhat'/> <usage type='iscsi'> <target>libvirtiscsi</target> </usage> </secret> # virsh secret-define iscsi-secret.xml Secret 653f8bfe-7e1a-4e55-904f-608ced9d563b created # virsh secret-list UUID Usage -------------------------------------------------------------------------------- 653f8bfe-7e1a-4e55-904f-608ced9d563b iscsi libvirtiscsi 2.2 set value for secret # MYSECRET=`printf %s "rhatrhat" | base64` # virsh secret-set-value 653f8bfe-7e1a-4e55-904f-608ced9d563b $MYSECRET Secret value set 2.3 create a iscsi pool with authentication # virsh pool-dumpxml iscsi-pool <pool type='iscsi'> <name>iscsi-pool</name> ... <source> <host name='ip' port='3260'/> <device path='iqn.2008-09.5.165.3ffe.104:server.target1'/> <auth type='chap' username='rhat'> <secret type='iscsi' usage='libvirtiscsi'/> </auth> </source> <target> <path>/dev/disk/by-path</path> ... </pool> # virsh pool-define iscsi-pool-secret.xml Pool iscsi-pool defined from iscsi-pool-secret.xml # virsh pool-start iscsi-pool Pool iscsi-pool started # virsh vol-list iscsi-pool Name Path ------------------------------------------------------------------------------ unit:0:0:1 /dev/disk/by-path/ip-$ip:3260-iscsi-iqn.2008-09.5.165.3ffe.104:server.target1-lun-1 3.define and start a guest with following XML : start a guest with source mode is direct . ...... <disk type='volume' device='disk'> <driver name='qemu' type='raw'/> <source pool='iscsi-pool' volume='unit:0:0:1' mode='direct'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </disk> ...... 3.1 save and restore # virsh save iscsi-vol iscsi.save Domain iscsi-vol saved to iscsi.save # virsh restore iscsi.save error: Failed to restore domain from iscsi.save error: internal error: invalid secret type 'iscsi' Actual results: As step3.1 guest fail to restore . Expected results: Guest restore successfully. Additional info: Also fail to create snapshot . # virsh snapshot-create-as iscsi-vol s1 --disk-only --diskspec vda,file=/tmp/snap1 error: internal error: invalid secret type 'iscsi'