Hide Forgot
Description of problem: Nova is unable to find iSCSI volume after rebooting the compute node Version-Release number of selected component (if applicable): OSP 6.0 How reproducible: Steps to Reproduce: 1. Boot an instance from volume (iSCSI backend) 2. Reboot the compute node 3. Resume the instance Actual results: The instance failed to resume and the following error could be found: 2016-02-13 16:03:31.230 4323 WARNING nova.virt.libvirt.volume [req-d051fda8-0098-4b3a-814e-86f503bb4fb0 None] ISCSI volume not yet found at: vda. Will rescan & retry. Try number: 0 2016-02-13 16:03:32.292 4323 WARNING nova.virt.libvirt.volume [req-d051fda8-0098-4b3a-814e-86f503bb4fb0 None] ISCSI volume not yet found at: vda. Will rescan & retry. Try number: 1 2016-02-13 16:03:36.363 4323 WARNING nova.virt.libvirt.volume [req-d051fda8-0098-4b3a-814e-86f503bb4fb0 None] ISCSI volume not yet found at: vda. Will rescan & retry. Try number: 2 2016-02-13 16:03:45.433 4323 WARNING nova.virt.libvirt.volume [req-d051fda8-0098-4b3a-814e-86f503bb4fb0 None] ISCSI volume not yet found at: vda. Will rescan & retry. Try number: 3 2016-02-13 16:04:01.503 4323 WARNING nova.virt.libvirt.volume [req-d051fda8-0098-4b3a-814e-86f503bb4fb0 None] ISCSI volume not yet found at: vda. Will rescan & retry. Try number: 4 2016-02-13 16:04:26.701 4323 ERROR oslo.messaging.rpc.dispatcher [req-d051fda8-0098-4b3a-814e-86f503bb4fb0 ] Exception during message handling: iSCSI device not found at [u'/dev/disk/by-path/ip-192.168.10.170:3260-iscsi-iqn.2010-10.org.openstack:volume-cab83bd2-bb3d-4b63-85ba-2fff11d5d448-lun-0'] Expected results: The instance should be successfully resume. Additional info: sosreports for both controller and compute (after the accident) are in the collab-shell.usersys.redhat.com:/cases/01583215
Hi Lee, Do you think this is something like "slow scanning disk" issue within compute node ? Or is there anywhere suspicious ? The customer got a plan to reboot the environment again and they are worrying that they could encounter the issue again. I have asked them to collect the sosreports when the problem is occurring. Is there anything you need ? Best regards, Chen
(In reply to Chen from comment #2) > Hi Lee, > > Do you think this is something like "slow scanning disk" issue within > compute node ? Or is there anywhere suspicious ? > > The customer got a plan to reboot the environment again and they are > worrying that they could encounter the issue again. I have asked them to > collect the sosreports when the problem is occurring. Is there anything you > need ? Hello Chen, Apologies for the delay, it appears you forgot to add a needinfo. (In reply to Chen from comment #0) > Steps to Reproduce: > 1. Boot an instance from volume (iSCSI backend) > 2. Reboot the compute node > 3. Resume the instance I'm unable to reproduce this using the latest 2014.2.3 builds of Nova. Would the customer be willing to update to 2014.2.3, increase num_iscsi_scan_tries, enable debug logging and reproduce?