Bug 858256

Summary: volume attach fails with iscsiadm --login timeout
Product: [Fedora] Fedora Reporter: Eoghan Glynn <eglynn>
Component: openstack-cinderAssignee: Pádraig Brady <p>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 18CC: akscram, alexander.sakhnov, apevec, asalkeld, bfilippov, breu, eharney, Jan.van.Eldik, jonathansteffan, jose.castro.leon, markmc, matt_domsch, mlvov, p, rbryant, rkukura
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-12-20 15:07:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Eoghan Glynn 2012-09-18 13:11:55 UTC
Description of problem:

Attempt to attach a volume to a nova instance fails with the following error in the nova-compute log:

  'iscsiadm: Could not login to [iface: default, target: iqn.2010-10.org.openstack:volume-81d30c1d-a932-4e03-babe-c8584bcc45ed, portal: 192.168.0.6,3260].\niscsiadm: initiator reported error (8 - connection timed out)\niscsiadm: Could not log into all portals\n'


Version-Release number of selected component (if applicable):

openstack-nova-compute-2012.2-0.7.f3.fc18.noarch


How reproducible:

100%


Steps to Reproduce:
1. Create a volume
2. Attach the volume to an instance
3. Monitor the volume status through 'attaching' back to 'available'


Actual results:

The volume status fails to reach 'in-use'.


Expected results:

The volume status transitions to 'in-use'.

Comment 2 Mark McLoughlin 2012-09-20 10:08:09 UTC
The issue turns out to be that the iSCSI target was never actually getting created

We need tgtd to include volumes configurations from /var/lib/nova/volumes as described here:

  http://lists.fedoraproject.org/pipermail/cloud/2012-July/001593.html

Comment 3 Eoghan Glynn 2012-09-20 14:20:43 UTC
Small correction, it should include /var/lib/cinder/volumes in the tgtd.conf, i.e.:

  sudo sed -i '1iinclude /var/lib/cinder/volumes/*' /etc/tgt/targets.conf

Comment 4 Fedora Update System 2012-09-21 17:53:29 UTC
openstack-cinder-2012.2-0.5.rc1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/openstack-cinder-2012.2-0.5.rc1.fc18

Comment 5 Fedora Update System 2012-09-22 06:33:10 UTC
openstack-cinder-2012.2-0.5.rc1.fc18 has been pushed to the Fedora 18 testing repository.

Comment 6 Mark McLoughlin 2012-09-24 15:29:45 UTC
FYI - for this to work on F17, you need this scsi-target-utils update:

https://admin.fedoraproject.org/updates/FEDORA-2012-14471/scsi-target-utils-1.0.24-4.fc17

Comment 7 Eoghan Glynn 2012-09-25 14:06:47 UTC
Re-opening as the work-around:

  # 1. remove non-globbed conf fragments
  sudo rm /etc/tgt/conf.d/{nova,cinder}.conf
  # 2. ensure that target config in /etc/cinder/volume picked up
  # after tgtd restart
  sudo sed -i '1iinclude /etc/cinder/volumes/*' /etc/tgt/targets.conf

is still required.

#2 in particular required to ensure that a volume created before a tgtd restart is still usable afterwards.

Comment 8 Pádraig Brady 2012-09-25 14:11:49 UTC
It seems nested includes as discussed in bug #643302 is supported,
but nested globbing is not. Investigating.

Comment 9 Mark McLoughlin 2012-09-25 14:30:14 UTC
This recent patch - https://review.openstack.org/12542 - is actually a part of the confusion here

We didn't have this patch during the first test day and volume attach was failing

However, in the second test day, this patch was in and attach works fine without any inclusion targets.conf

The problem is that if you restart tgtd without the targets.conf inclusion, the targets disappear - this is the issue that the persistent targets work in Folsom was intended to fix

Comment 10 Pádraig Brady 2012-09-26 23:28:55 UTC
Possible solution detailed at bug 643302#c14

Comment 11 Fedora Update System 2012-09-27 17:37:53 UTC
openstack-nova-2012.2-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/openstack-nova-2012.2-1.fc18

Comment 12 Fedora Update System 2012-09-27 17:40:44 UTC
openstack-nova-2012.2-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/openstack-nova-2012.2-1.fc18

Comment 13 Fedora Update System 2012-09-27 17:42:59 UTC
openstack-cinder-2012.2-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/openstack-cinder-2012.2-1.fc18

Comment 14 Fedora Update System 2012-09-28 17:21:34 UTC
openstack-cinder-2012.2-1.fc18 has been pushed to the Fedora 18 testing repository.

Comment 15 Fedora Update System 2012-12-20 15:07:49 UTC
openstack-cinder-2012.2-0.5.rc1.fc18 has been pushed to the Fedora 18 obsolete repository.