Bug 1121591 - Viewing console while launching a VDI instances causes error status
Summary: Viewing console while launching a VDI instances causes error status
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 5.0 (RHEL 6)
Hardware: x86_64
OS: Linux
medium
unspecified
Target Milestone: ---
: 5.0 (RHEL 7)
Assignee: Russell Bryant
QA Contact: Ami Jeain
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-07-21 09:58 UTC by Tzach Shefi
Modified: 2019-09-09 15:44 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-08-13 15:05:12 UTC


Attachments (Terms of Use)
Compute log (201.47 KB, application/x-gzip)
2014-07-21 09:58 UTC, Tzach Shefi
no flags Details
new compute log (136.28 KB, text/plain)
2014-08-04 08:50 UTC, Tzach Shefi
no flags Details
Cinder volume log (98.80 KB, application/x-gzip)
2014-08-07 06:50 UTC, Tzach Shefi
no flags Details


Links
System ID Priority Status Summary Last Updated
Launchpad 1346148 None None None Never

Description Tzach Shefi 2014-07-21 09:58:03 UTC
Created attachment 919576 [details]
Compute log

Description of problem: If you access an instance's console while launching an a VDI image, instance status: error. Ran this a few times if however you wait until instance has completed booting before accessing console it boots up fine. This doesn't happen with Cirros QCOW.  

VDI image used, must be uncompressed:
http://downloads.sourceforge.net/virtualboximage/dsl-4.2.5-x86.7z

Version-Release number of selected component (if applicable):
RHEL 6.5 
openstack-nova-compute-2014.1.1-2.el6ost.noarch

How reproducible:
Every time, tested three times.

Steps to Reproduce:
1. In my case Glance uses a Gluster share, don't think it's related.
2. Upload VDI image to Glance.
3. Launch an instance from VDI image.
4. Before instances completes boot process, go to instance's console.
5. Instance's failed to boot, status error. 
6. Repeat above steps, this time wait for instance to complete boot process then go to console, status is active. 

Actual results:

Failed to boot VDI instance, when accessing instance's console before instance is active.  

On compute.log instance ID  13b23f83-9587-4068-a6d1-d2b8034d0e80 status error, while instance ID 542bfd5e-866e-4af4-ab59-f1ecb86bfa2f booted up fine. 

Expected results:

VDI instance should boot up even if you look at console during boot process, just like Cirros qcow.

Comment 1 Russell Bryant 2014-07-30 15:17:02 UTC
There's several other errors in the compute log that show that the instance would have failed to boot for another reason.  Primarily:

2014-07-21 12:10:21.655 7692 TRACE nova.compute.manager [instance: 13b23f83-9587-4068-a6d1-d2b8034d0e80] FlavorDiskTooSmall: Flavor's disk is too small for requested image.

I want to make sure the boot failed for the reason you think it did.  Can you try testing again that shows this failure with a clean log with only failures related to the console?

Comment 2 Tzach Shefi 2014-08-04 08:50:10 UTC
Created attachment 923803 [details]
new compute log

Comment 3 Tzach Shefi 2014-08-04 08:51:46 UTC
See above compute.log, while trying to recreate this I may have found a new bug. 
Using same VDI image source, can't boot instance:                                                                                                                                        
2014-08-04 11:42:25.318 16648 ERROR nova.compute.manager [req-9b5a8aa8-3bad-47bc-b1d4-4e732166810d b643e376b9f746c89744b218b3b3288e 12f4debd3b314201bbab93341f94d67e] [instance: 75694195-1568-4234-810e-b60d4e798486] Error: Unexpected error while running command.                                                                     
Command: env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/_base/759fd7c54b5726bb3e060daeb676e15f44be97f2.part                                               
Exit code: 1                                                                                                                                                         
Stdout: ''                                                                                                                                                           
Stderr: "Could not open '/var/lib/nova/instances/_base/759fd7c54b5726bb3e060daeb676e15f44be97f2.part': Operation not permitted\n"                                    
2014-08-04 11:42:25.318 16648 TRACE nova.compute.manager [instance: 75694195-1568-4234-810e-b60d4e798486] Traceback (most recent call last):                         
2014-08-04 11:42:25.318 16648 TRACE nova.compute.manager [instance: 75694195-1568-4234-810e-b60d4e798486]   File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1305, in _build_instance                                                                 

Any tips would be welcomed :)

Comment 4 Tzach Shefi 2014-08-04 08:53:21 UTC
Forgot to mention, Horizon gives the usual:

Error: Failed to launch instance "test": Please try again later [Error: No valid host was found.

Comment 5 Russell Bryant 2014-08-06 15:21:23 UTC
(In reply to Tzach Shefi from comment #3)
> See above compute.log, while trying to recreate this I may have found a new
> bug. 
> Using same VDI image source, can't boot instance:                           
> 
> 2014-08-04 11:42:25.318 16648 ERROR nova.compute.manager
> [req-9b5a8aa8-3bad-47bc-b1d4-4e732166810d b643e376b9f746c89744b218b3b3288e
> 12f4debd3b314201bbab93341f94d67e] [instance:
> 75694195-1568-4234-810e-b60d4e798486] Error: Unexpected error while running
> command.                                                                     
> Command: env LC_ALL=C LANG=C qemu-img info
> /var/lib/nova/instances/_base/759fd7c54b5726bb3e060daeb676e15f44be97f2.part 
> 
> Exit code: 1                                                                
> 
> Stdout: ''                                                                  
> 
> Stderr: "Could not open
> '/var/lib/nova/instances/_base/759fd7c54b5726bb3e060daeb676e15f44be97f2.
> part': Operation not permitted\n"                                    
> 2014-08-04 11:42:25.318 16648 TRACE nova.compute.manager [instance:
> 75694195-1568-4234-810e-b60d4e798486] Traceback (most recent call last):    
> 
> 2014-08-04 11:42:25.318 16648 TRACE nova.compute.manager [instance:
> 75694195-1568-4234-810e-b60d4e798486]   File
> "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 1305, in
> _build_instance                                                             
> 
> 
> Any tips would be welcomed :)

Have you checked for any SELinux errors?

Also, how was this deployment done?

Comment 6 Tzach Shefi 2014-08-07 06:49:30 UTC
Packstack deployment, since then I tired to run various other VDI images downloaded from virtualbox site (guess they should be ok if they are posted). 

No avc found on audit.log, but just to be sure setenforce 0 for testing, didn't help.  

Adding insult to injury it even failed to create a volume from VDI image, status error.  

Cinder log (also attached)

2014-08-07 09:38:37.867 28906 TRACE oslo.messaging.rpc.dispatcher     raise exception.ImageCopyFailure(reason=ex.stderr)
2014-08-07 09:38:37.867 28906 TRACE oslo.messaging.rpc.dispatcher ImageCopyFailure: Failed to copy image to volume: Could not open '/var/lib/cinder/conversion/tmpgbL3B0': Operation not permitted


Should we keep working on this here, or open a new bug Nova VDI image problem?

Comment 7 Tzach Shefi 2014-08-07 06:50:49 UTC
Created attachment 924721 [details]
Cinder volume log

should open another bug about cinder volume from VDI image?

Comment 8 Russell Bryant 2014-08-13 15:05:12 UTC
Actually, looking at this again, VDI images aren't something supported at all.  Also, an image made for use with virtualbox may not work at all, either.  So, I don't think this is a bug, and you just need to get a valid image.


Note You need to log in before you can comment on or make changes to this bug.