Bug 1311177 - implementation of soft shutdown on vmware hypervisors
Summary: implementation of soft shutdown on vmware hypervisors
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 6.0 (Juno)
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
: 6.0 (Juno)
Assignee: Eoghan Glynn
QA Contact: nlevinki
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-02-23 14:46 UTC by GE Scott Knauss
Modified: 2019-10-10 11:18 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Openstack 6.0 with vmware hypervisors
Last Closed: 2016-04-01 16:30:46 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description GE Scott Knauss 2016-02-23 14:46:56 UTC
What problem/issue/behavior are you having trouble with?  What do you expect to see?

nova stop does not use the vmware client for shutdown the vm

We implemented the soft shutdown, using the vmware client, for nova stop.

could your please look at the following diffs and implement this in OSP 6? thanks

diff --git a/nova/virt/vmwareapi/driver.py b/nova/virt/vmwareapi/driver.py
index 82fabee..46abaed 100644
--- a/nova/virt/vmwareapi/driver.py
+++ b/nova/virt/vmwareapi/driver.py
@@ -513,7 +513,7 @@ class VMwareVCDriver(driver.ComputeDriver):
         """Power off the specified instance."""
         # TODO(PhilDay): Add support for timeout (clean shutdown)
         LOG.info("Shutdown timeout: %s, Retry interval: %s", timeout, retry_interval)
-        self._vmops.power_off(instance)
+        self._vmops.power_off(instance, timeout, retry_interval)

     def power_on(self, context, instance, network_info,
                  block_device_info=None):

diff --git a/nova/virt/vmwareapi/vmops.py b/nova/virt/vmwareapi/vmops.py
index db820b5..b3b8afd 100644
--- a/nova/virt/vmwareapi/vmops.py
+++ b/nova/virt/vmwareapi/vmops.py
@@ -64,6 +64,8 @@ CONF.import_opt('my_ip', 'nova.netconf')

 LOG = logging.getLogger(__name__)

+SHUTDOWN_TIME_INCREMENT = 5
+
 VMWARE_POWER_STATES = {
                    'poweredOff': power_state.SHUTDOWN,
                     'poweredOn': power_state.RUNNING,
@@ -974,13 +976,78 @@ class VMwareVMOps(object):
         if power_on:
             vm_util.power_on_instance(self._session, instance, vm_ref=vm_ref)

-    def power_off(self, instance):
+    def power_off(self, instance, timeout=0, retry_interval=0):
         """Power off the specified instance.

         :param instance: nova.objects.instance.Instance
         """
+
+        if retry_interval <= 0:
+            retry_interval = SHUTDOWN_TIME_INCREMENT
+
+        if timeout and self._soft_shutdown(instance,
+                                           timeout,
+                                           retry_interval):
+            return
+
+        LOG.info("Performing Hard shutdown on instance", instance=instance)
         vm_util.power_off_instance(self._session, instance)

+    def _soft_shutdown(self, instance,
+                       timeout=60,
+                       retry_interval=SHUTDOWN_TIME_INCREMENT):
+        """Perform a soft shutdown on the VM.
+           :return: True if the instance was shutdown within time limit,
+           False otherwise.
+        """
+        LOG.info("Performing Soft shutdown on instance", instance=instance)
+        vm_ref = vm_util.get_vm_ref(self._session, instance)
+        lst_properties = ["summary.guest.toolsStatus", "runtime.powerState",
+                          "summary.guest.toolsRunningStatus"]
+        props = self._session._call_method(vim_util, "get_object_properties",
+                           None, vm_ref, "VirtualMachine",
+                           lst_properties)
+        query = vm_util.get_values_from_object_properties(self._session, props)
+        pwr_state = query['runtime.powerState']
+        tools_status = query['summary.guest.toolsStatus']
+        tools_running_status = query['summary.guest.toolsRunningStatus']
+
+        if pwr_state not in ["poweredOn"]:
+            return False
+
+        if (tools_status == "toolsOk" and
+                tools_running_status == "guestToolsRunning"):
+
+            LOG.info("Soft shutdown instance, timeout: %d",
+                    timeout, instance=instance)
+            self._session._call_method(self._session._get_vim(), "ShutdownGuest",
+                    vm_ref)
+
+            while timeout > 0:
+                wait_time = min(retry_interval, timeout)
+                try:
+                    props = self._session._call_method(vim_util, "get_object_properties",
+                            None, vm_ref, "VirtualMachine",
+                            lst_properties)
+                    query = vm_util.get_values_from_object_properties(self._session, props)
+                    if query['runtime.powerState'] in ["poweredOff"]:
+                        LOG.info("Soft shutdown succeeded.",
+                                  instance=instance)
+                        return True
+                except Exception as e:
+                    LOG.info("Soft shutdown failed: %s", e, instance=instance)
+
+                time.sleep(wait_time)
+                timeout -= retry_interval
+
+            LOG.warning("Timed out while waiting for soft shutdown.",
+                        instance=instance)
+        else:
+            LOG.info("VMWare Tools not running", instance=instance)
+
+        return False
+
+
     def power_on(self, instance):
         vm_util.power_on_instance(self._session, instance

Where are you experiencing the behavior?  What environment?

applications on the os could not stop gentle

When does the behavior occur? Frequently?  Repeatedly?   At certain times?

every time

Comment 5 GE Scott Knauss 2016-04-01 12:01:01 UTC
Matt,
     They are going to submit to the upstream. Should we close this BZ and open a new once they have their code accepted? If so, feel free... 

-Scott

Comment 6 Matthew Booth 2016-04-01 16:30:46 UTC
(In reply to GE Scott Knauss from comment #5)
> Matt,
>      They are going to submit to the upstream. Should we close this BZ and
> open a new once they have their code accepted? If so, feel free... 

Thanks, Scott.


Note You need to log in before you can comment on or make changes to this bug.