Bug 1434986
| Summary: | Nova Evacuation Failed while evacuating instance back to previously targeted host | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Manisha Tripathy <manisha_tripathy> |
| Component: | openstack-nova | Assignee: | Eoghan Glynn <eglynn> |
| Status: | CLOSED DUPLICATE | QA Contact: | Prasanth Anbalagan <panbalag> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 10.0 (Newton) | CC: | arkady_kanevsky, audra_cooper, berrange, cdevine, dasmith, eglynn, John_walsh, j_t_williams, kchamart, kurt_hey, manisha_tripathy, mbooth, sbauza, sferdjao, sgordon, smerrow, sreichar, srevivo, vromanso |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-03-24 17:07:48 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release. *** Bug 1434985 has been marked as a duplicate of this bug. *** Can you please provide a couple of things to help us get to the bottom of this: 1. The nova-compute.log containing the permission error 2. The output of 'ls -l /var/lib/nova/instances/' on both the source and destination? Thanks. Hi Mathew compute1 is the source where the instance is created compute2 is destination When I evacuate the vm from compute 1 to compute 2,permissions look like root@r11b-compute-1 ~]# cd /var/lib/nova/instances/ [root@r11b-compute-1 instances]# ls -lZ 00a37206-353c-47da-8f66-af857e5dd564/ -rw-r--r--. qemu qemu system_u:object_r:svirt_image_t:s0:c355,c987 console.log root@r11b-compute-2 ~]# cd /var/lib/nova/instances/ [root@r11b-compute-2 instances]# ls -lZ 00a37206-353c-47da-8f66-af857e5dd564/ -rw-r--r--. qemu qemu system_u:object_r:svirt_image_t:s0:c355,c987 console.log Then when I get Compute 1 power on back, the instance directory still remains in Compute 1 with permissions changed as follows: root@r11b-compute-1 ~]# cd /var/lib/nova/instances/ [root@r11b-compute-1 instances]# ls -lZ 00a37206-353c-47da-8f66-af857e5dd564/ -rw-r--r--. root root system_u:object_r:svirt_image_t:s0:c355,c987 console.log Now, when I try to evacuate the same instance from compute 2 to compute 1 , it goes to error state. I hope this helps to understand the issue. Thanks Manisha Thanks for the quick response, Manisha. I'm looking specifically for the directory permissions on the instance directory itself. If the error you're seeing is on the console.log file then this looks like a duplicate of bug 1422154, which we're already working on. However, if it's on the whole instance directory then it's a new problem we don't understand yet. Could you paste the output of: # ls -ld /var/lib/nova/instances/00a37206-353c-47da-8f66-af857e5dd564/ Thanks, Matt root@r11b-compute-2]# ls -ld /var/lib/nova/00a37206-353c-47da-8f66-af857e5dd564/ drwxr-xr-x. 2 nova nova 25 Mar 24 15:44 00a37206-353c-47da-8f66-af857e5dd564/ Thanks, Manisha. I'm going to move this over to the other bug. *** This bug has been marked as a duplicate of bug 1422154 *** |
Description of problem: With shared-storage backend, when I evacuate an instance first time, it works fine. But when I try to evacuate it back to its initial host it goes into error state giving permission denied error message. Details on checking the permissions on the directory /var/lib/nova/instances/<instance_id> are a follows Status Permissions on Compute1 Permissions on Compute2 Create Vm on compute1 qemu N/A Compute1-Power On qemu N/A Compute-1 Power Off qemu N/A Evacuate to Compute2 qemu qemu Power On Compute 1 root qemu Power Off Compute 2 root qemu Evacuate back to Compute 1- root qemu (this is where it fails giving permission denied) Version-Release number of selected component (if applicable): RH0SP-10 How reproducible: Every time Steps to Reproduce: Create the instance on shared storage (Ceph ephemeral), on Compute1 Verify the instance directory is created on compute1 in cd /var/lib/nova/instances/<instanceID> Power off Compute1 Evacuate the instance to Compute2 (nova host-evacuate --target_host overcloud-compute-2.localdomain overcloud-compute-1.localdomain) Verify the instance is running on Compute2 and the instance directory resides on Compute2 cd /var/lib/nova/instances/<instanceID> Power on Compute1 and see that the instance directory is still there Power off Compute2 Evacuate the instance back to Compute1 (nova host-evacuate --target_host overcloud-compute-1.localdomain overcloud-compute-2.localdomain) The instance comes up with ERROR state on overcloud-compute-2.localdomain