Bug 1088572 - hosted-engine script fails with Traceback when cluster doesn't return a CPU type
Summary: hosted-engine script fails with Traceback when cluster doesn't return a CPU type
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-hosted-engine-setup
Version: 3.3.0
Hardware: x86_64
OS: Linux
medium
high
Target Milestone: ---
: 3.4.0
Assignee: Sandro Bonazzola
QA Contact: Artyom
URL:
Whiteboard: integration
: 1073986 (view as bug list)
Depends On:
Blocks: 1092378
TreeView+ depends on / blocked
 
Reported: 2014-04-16 19:33 UTC by wdaniel
Modified: 2019-04-28 08:38 UTC (History)
18 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Previously, the Red Hat Enterprise Virtualization Manager would not provide CPU information for cluster objects in the response to the first query made by the ovirt-hosted-engine-setup command under certain conditions. This would cause setup to fail after timing out due to not having CPU information. Now, the ovirt-hosted-engine-setup command queries the Red Hat Enterprise Virtualization Manager again until the timeout is reached, allowing CPU information to be retrieved corrected and setup to complete successfully.
Clone Of:
: 1092378 (view as bug list)
Environment:
Last Closed: 2014-06-09 14:48:05 UTC
oVirt Team: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1073986 1 None None None 2021-01-20 06:05:38 UTC
Red Hat Product Errata RHBA-2014:0505 0 normal SHIPPED_LIVE ovirt-hosted-engine-setup bug fix and enhancement update 2014-06-09 18:45:23 UTC
oVirt gerrit 26887 0 master ABANDONED packaging: setup: avoid traceback on adding host Never
oVirt gerrit 26964 0 master MERGED packaging: setup: query cluster on attempts loop Never
oVirt gerrit 26965 0 ovirt-hosted-engine-setup-1.0 ABANDONED packaging: setup: query cluster on attempts loop Never
oVirt gerrit 26966 0 ovirt-hosted-engine-setup-1.1 MERGED packaging: setup: query cluster on attempts loop Never

Internal Links: 1073986

Description wdaniel 2014-04-16 19:33:21 UTC
Description of problem:

It seems that if the RHEV API can not return a CPU type using 'cluster.get_cpu()', the hosted-engine script fails out, breaking the installation process.

Looking at file /usr/share/ovirt-hosted-engine-setup/plugins/ovirt-hosted-engine-setup/engine/add_host.py at the '_closeup()' method:

<snippet - line 461>
        else:
            #This works only if the host is up.
            self.logger.debug('Setting CPU for the cluster')
            try:
                cluster = engine_api.clusters.get('Default')
                self.logger.debug(cluster.__dict__)
                cpu = self._wait_cluster_cpu_ready(cluster)
                self.logger.debug(cpu.__dict__)
                cpu.set_id(self.environment[ohostedcons.VDSMEnv.ENGINE_CPU])
                cluster.set_cpu(cpu)
                cluster.update()
</snippet>

As you can see, 'cpu' is defined by whatever the _wait_cluster_cpu_ready() method returns. Looking at the _wait_for_cluster_cpu_ready() code:

<snippiet - line 244>
    def _wait_cluster_cpu_ready(self, cluster):
        tries = self.VDSM_RETRIES
        cpu = None
        while cpu is None and tries > 0:
            tries -= 1
            cpu = cluster.get_cpu()
            if cpu is None:
                self.logger.debug(
                    'cluster {cluster} cluster.__dict__ {cdict}'.format(
                        cluster=cluster,
                        cdict=cluster.__dict__,
                    )
                )
                if tries % 30 == 0:
                    self.logger.info(
                        _(
                            "Waiting for cluster '{name}' "
                            "to become operational..."
                        ).format(
                            name=cluster.name,
                        )
                    )
                time.sleep(self.VDSM_DELAY)
        if cpu is None and tries == 0:
            self.logger.error(_(
                'Timed out while waiting for cluster to become ready. '
                'Please check the logs.'
            ))
        return cpu
</snippet>

In this code the variable 'cpu' is initially assigned as 'None'. After going through the while loop we then hit the 'if' statement, which does not change the 'cpu' variable. Finally, the 'cpu' variable is returned to the _closeup() method. If 'cpu' still equals None, then the script fails out as that object has no method '__dict__':


Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/otopi/context.py", line 127, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/engine/add_host.py", line 468, in _closeup
    self.logger.debug(cpu.__dict__)
AttributeError: 'NoneType' object has no attribute '__dict__'
2014-04-11 20:23:34 ERROR otopi.context context._executeMethod:146 Failed to execute stage 'Closing up': 'NoneType' object has no attribute '__dict__'

Version-Release number of selected component (if applicable):

RHEV 3.3

How reproducible:
I've not been able to reproduce yet - I can't get the API to time out and prevent the variable from changing

Steps to Reproduce:
1.
2.
3.

Actual results:

hosted-engine script fails out during the final phase of installation

Expected results:

hosted-engine script successfully finishes

Additional info:

Comment 1 Sandro Bonazzola 2014-04-17 09:17:55 UTC
(In reply to wdaniel from comment #0)

> Actual results:
> 
> hosted-engine script fails out during the final phase of installation
> 
> Expected results:
> 
> hosted-engine script successfully finishes

If cpu object is not returned, the installation can't successfully finish.
The only thing I can do is completing the setup without adding the host to the engine without the traceback, allowing the user to check why the cluster is not working as expected and manually add the host to the engine.

Comment 3 Sandro Bonazzola 2014-04-17 12:24:28 UTC
Proposed a patch for getting more debug info on hosted-engine setup side and avoid the traceback.

Comment 6 Sandro Bonazzola 2014-04-18 13:20:59 UTC

*** This bug has been marked as a duplicate of bug 1073986 ***

Comment 7 Sandro Bonazzola 2014-04-28 10:57:23 UTC
*** Bug 1073986 has been marked as a duplicate of this bug. ***

Comment 11 Artyom 2014-05-01 14:14:23 UTC
Verified on is8.1
On this scenario https://bugzilla.redhat.com/show_bug.cgi?id=1080513

Comment 12 errata-xmlrpc 2014-06-09 14:48:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0505.html


Note You need to log in before you can comment on or make changes to this bug.