Bug 1567601

Summary: Second rhel-guest-image-7.5 VM - No valid host was found
Product: Red Hat OpenStack Reporter: Noam Manos <nmanos>
Component: openstack-novaAssignee: OSP DFG:Compute <osp-dfg-compute>
Status: CLOSED INSUFFICIENT_DATA QA Contact: OSP DFG:Compute <osp-dfg-compute>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 13.0 (Queens)CC: astafeye, berrange, dasmith, eglynn, jhakimra, kchamart, mbooth, nmanos, sbauza, sferdjao, sgordon, srevivo, tfreger, vromanso
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-26 14:49:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
command output
none
sosreport undercloud
none
sosreport compute-1
none
Could not reproduce on OSP13 - 2018-04-10.2. here's Console Output. none

Description Noam Manos 2018-04-15 09:54:21 UTC
Created attachment 1422148 [details]
command output

Description of problem:
Trying to add a second rhel-guest-image-7.5 VM - while there are enough resources - shows ERROR "No valid host was found".

Version-Release number of selected component:
OSP 13  -  2018-03-20.2

How reproducible:
Always

Steps to Reproduce:

1) Add rhel 7.5 image and flavor:

curl -o /tmp/rhel75.qcow2 http://rhos-qe-mirror-tlv.usersys.redhat.com/rel-eng/latest-RHEL-7.5/compose/Server/x86_64/images/rhel-guest-image-7.5-146.x86_64.qcow2

openstack image create rhel75 --disk-format qcow2 --container-format bare --public --file /tmp/rhel75.qcow2

openstack flavor create --ram 2048 --disk 10 --vcpus 1 --id auto rhel7


2) Create network:

. overcloudrc

openstack network create int_net 

int_net_id=$(openstack network show int_net -c id -f value)


3) Create first RHEL vm:

openstack server create vm-rht-1 --flavor rhel7 --nic net-id=$int_net_id --image rhel75


4) Create second RHEL vm:

openstack server create vm-rht-2 --flavor rhel7 --nic net-id=$int_net_id --image rhel75

5) Check VMs status:
nova hypervisor-stats
nova list
nova show vm-rht-2

-----

Actual results:

Despite there are enough resources:

+----------------------+-------+
| Property             | Value |
+----------------------+-------+
| count                | 2     |
| current_workload     | 0     |
| disk_available_least | 87    |
| free_disk_gb         | 102   |
| free_ram_mb          | 3838  |
| local_gb             | 118   |
| local_gb_used        | 16    |
| memory_mb            | 16382 |
| memory_mb_used       | 12544 |
| running_vms          | 5     |
| vcpus                | 12    |
| vcpus_used           | 5     |
+----------------------+-------+


vm-rht-2 status shows ERROR:

"No valid host was found. ", "code": 500, "details": "  File \"/usr/lib/python2.7/site-packages/nova/conductor/manager.py\", line 1116, in schedule_and_build_instances


Expected results:

If there are enough resources, all VMs should show ACTIVE status.

Additional info:
Commands output, logs, and SOS report attached hereby.

Comment 1 Noam Manos 2018-04-15 09:58:01 UTC
Created attachment 1422149 [details]
sosreport undercloud

Comment 2 Noam Manos 2018-04-15 10:05:44 UTC
Created attachment 1422150 [details]
sosreport compute-1

Comment 3 Kashyap Chamarthy 2018-04-20 13:52:43 UTC
I've downloaded the sosreports (from both the Compute node and "UnderCloud"), and I don't see the relevant traceback from either of those sosreports.  So there isn't enough info here to tell what's going on.

Are you sure you've uploaded the right sosreports?

Comment 4 Noam Manos 2018-04-22 07:06:31 UTC
Could not reproduce issue on OSP 13 - 2018-04-10.2.

Console output attached.

Comment 5 Noam Manos 2018-04-22 07:08:39 UTC
Created attachment 1425251 [details]
Could not reproduce on OSP13 - 2018-04-10.2. here's Console Output.

Comment 6 Matthew Booth 2018-04-26 14:49:13 UTC
Without a reproducer I'm going to treat this as a ghost in the machine. Please feel free to reopen if it recurs.