Bug 1813596

Summary:	after upgrade to 13 DiskNotFound. scheduller discards these hipervisors and are not able to allocate new instances.
Product:	Red Hat OpenStack	Reporter:	Eduard Barrera <ebarrera>
Component:	openstack-nova	Assignee:	OSP DFG:Compute <osp-dfg-compute>
Status:	CLOSED DUPLICATE	QA Contact:	OSP DFG:Compute <osp-dfg-compute>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	13.0 (Queens)	CC:	dasmith, dparkes, eglynn, fgarciad, jhakimra, kchamart, lyarwood, molasaga, sbauza, sgordon, vromanso
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-03-14 20:08:01 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1381612

Description Eduard Barrera 2020-03-14 16:38:18 UTC

Description of problem:

it seems   https://review.opendev.org/#/c/711279/ 


/cases/02600290/0030-sosreport-cfs1pnc40-2020-03-14-fiwfprj.tar.xz/sosreport-cfs1pnc40-2020-03-14-fiwfprj/var/log/containers/nova/nova-compute.log


-a394-468f-93c9-2e91a855fc5f (vdd) as it is a volume _get_instance_disk_info_from_config /usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py:7902

2020-03-14 13:00:13.456 8 ERROR nova.compute.manager [req-a18ab3d6-394d-4058-b7ba-89f1cf1ec086 - - - - -] Error updating resources for node cfs1pnc40.infra.es.iaas.igrupobbva.: DiskNotFound: No disk at /var/lib/nova/instances/5ffe0ff6-1483-468b-9bfd-4c77cfd1c6eb/disk
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager Traceback (most recent call last):
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 7573, in update_available_resource_for_node
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager     rt.update_available_resource(context, nodename)
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py", line 690, in update_available_resource
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager     resources = self.driver.get_available_resource(nodename)
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6497, in get_available_resource
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager     disk_over_committed = self._get_disk_over_committed_size_total()
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 8080, in _get_disk_over_committed_size_total
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager     err_ctxt.reraise = False
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager     self.force_reraise()
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager     six.reraise(self.type_, self.value, self.tb)
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 8021, in _get_disk_over_committed_size_total
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager     config, block_device_info)
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7923, in _get_instance_disk_info_from_config
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager     qemu_img_info = disk_api.get_disk_info(path)
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/virt/disk/api.py", line 136, in get_disk_info
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager     return images.qemu_img_info(path)
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager   File "/usr/lib/python2.7/site-packages/nova/virt/images.py", line 57, in qemu_img_info
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager     raise exception.DiskNotFound(location=path)
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager DiskNotFound: No disk at /var/lib/nova/instances/5ffe0ff6-1483-468b-9bfd-4c77cfd1c6eb/disk
2020-03-14 13:00:13.456 8 ERROR nova.compute.manager


I'm trying to follow the nova-scheduler did for instance c54003fe-1d11-4050-88fd-2278eeaa2314 but just found this relevant line:

2020-03-14 14:07:54.649 27 DEBUG nova.scheduler.manager [req-76fe64b6-8183-4187-a008-c4b3cc5988e8 b898d2d4bbd14552bd05efa6538cbf13 4325d5ef7ece4b6fbd275cf8ea907b92 - default default] Starting to schedule for instances: [u'c54003fe-1d11-4050-88fd-2278eeaa2314'] select_destinations /usr/lib/python2.7/site-packages/nova/scheduler/manager.py:110



Version-Release number of selected component (if applicable):
OSP13

How reproducible:
always

Steps to Reproduce:
1. migrate from 10 to 13
2. spawn instances
3.

Actual results:
computes with the specified disk not found error are removed from the scheduler

Expected results:
computes not removed from the scheduler


Additional info:

Comment 2 Lee Yarwood 2020-03-14 20:08:01 UTC


*** This bug has been marked as a duplicate of bug 1810442 ***