Bug 1368127 - [downstream clone - 3.6.9] If ovirt-ha-agent fails to read local answers.conf during upgrade, it writes None to shared fhanswers.conf
Summary: [downstream clone - 3.6.9] If ovirt-ha-agent fails to read local answers.conf...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-hosted-engine-ha
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ovirt-3.6.9
: 3.6.9
Assignee: Simone Tiraboschi
QA Contact: Jiri Belka
URL:
Whiteboard:
Depends On: 1367732
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-18 13:25 UTC by rhev-integ
Modified: 2019-04-28 13:24 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
With the update, an issue in the Red Hat Enterprise Virtualization 3.5 to Red Hat Enterprise Virtualization 3.6 upgrade procedure has been fixed. This issue moved the answer file to the shared storage, even if it was empty or unavailable, on the host. This could lead to future issues, such as, refusing to upgrade as the answer file is missing.
Clone Of: 1367732
Environment:
Last Closed: 2016-09-21 17:54:56 UTC
oVirt Team: Integration
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1366879 0 high CLOSED --upgrade-appliance - Failed to execute stage 'Environment customization': File contains no section headers. file: <???>... 2021-02-22 00:41:40 UTC
Red Hat Product Errata RHBA-2016:1924 0 normal SHIPPED_LIVE ovirt-hosted-engine-ha bug fix update for 3.6.9 2016-09-21 21:47:05 UTC
oVirt gerrit 62584 0 ovirt-hosted-engine-ha-1.3 MERGED upgrade: stop the upgrade with unavailable files 2016-08-23 08:44:38 UTC

Internal Links: 1366879

Comment 2 Jiri Belka 2016-09-13 08:28:44 UTC
Even this change introduces raising runtime error if answers.conf is not available, this does not happen if "Host configuration is already up-to-date".

That means, if an user ended in previous 3.6 after upgrade from 3.5 with no answers.conf and 'None' in fhanswers.conf on shared storage, he will continue to have no answers.conf and misconfigured configuration on shared storage.


[root@dell-r210ii-03 ~]# egrep "Configuration|up-to-date" /var/log/ovirt-hosted-engine-ha/agent.log                                             
MainThread::ERROR::2016-09-13 09:40:00,121::upgrade::396::ovirt_hosted_engine_ha.lib.upgrade.StorageServer::(_get_conffile_content) Configuration file '/etc/ovirt-hosted-engine/answers.conf' not available: [Errno 2] No such file or directory: '/etc/ovirt-hosted-engine/answers.conf'

^^ from 3.6.8 which detected missing answers.conf (there was 'None' in fhanswers.conf in this time)

MainThread::INFO::2016-09-13 10:02:47,877::upgrade::994::ovirt_hosted_engine_ha.lib.upgrade.StorageServer::(upgrade_35_36) Host configuration is already up-to-date
MainThread::INFO::2016-09-13 10:03:11,771::upgrade::994::ovirt_hosted_engine_ha.lib.upgrade.StorageServer::(upgrade_35_36) Host configuration is already up-to-date
MainThread::INFO::2016-09-13 10:07:53,853::upgrade::1007::ovirt_hosted_engine_ha.lib.upgrade.StorageServer::(upgrade_35_36) Host configuration is already up-to-date
MainThread::INFO::2016-09-13 10:11:34,160::upgrade::1007::ovirt_hosted_engine_ha.lib.upgrade.StorageServer::(upgrade_35_36) Host configuration is already up-to-date
MainThread::INFO::2016-09-13 10:11:58,059::upgrade::1007::ovirt_hosted_engine_ha.lib.upgrade.StorageServer::(upgrade_35_36) Host configuration is already up-to-date

^^ IIUC this is from 3.6.9 which did not run later on '_get_conffile_content' and thus did not raised runtime error

[root@dell-r210ii-03 ~]# find /rhev/data-center/ -path "*/$domain/*" -type f -name "$file" | xargs -I {} tar xOf {} fhanswers.conf
None[root@dell-r210ii-03 ~]# 

04:26 < tiraboschi> jbelka: yes, we have another patch for that

Please provide BZ which covers this situation, otherwise I would consider this failedQE. Thx!

Comment 3 Simone Tiraboschi 2016-09-13 08:43:30 UTC
(In reply to Jiri Belka from comment #2)
> 04:26 < tiraboschi> jbelka: yes, we have another patch for that
> 
> Please provide BZ which covers this situation, otherwise I would consider
> this failedQE. Thx!

https://bugzilla.redhat.com/1366879

Comment 4 Jiri Belka 2016-09-13 10:46:30 UTC
(In reply to Simone Tiraboschi from comment #3)
> (In reply to Jiri Belka from comment #2)
> > 04:26 < tiraboschi> jbelka: yes, we have another patch for that
> > 
> > Please provide BZ which covers this situation, otherwise I would consider
> > this failedQE. Thx!
> 
> https://bugzilla.redhat.com/1366879

The BZ above solves described issue in #2 only when setup - ie. hosted-engine - command is involved, see https://bugzilla.redhat.com/show_bug.cgi?id=1368399#c2

IMO this BZ should solve also #2, ie. to raise runtime error when already existing 3.6 SHE env is "broken".

Comment 5 Yedidyah Bar David 2016-09-13 12:01:24 UTC
(In reply to Jiri Belka from comment #4)
> (In reply to Simone Tiraboschi from comment #3)
> > (In reply to Jiri Belka from comment #2)
> > > 04:26 < tiraboschi> jbelka: yes, we have another patch for that
> > > 
> > > Please provide BZ which covers this situation, otherwise I would consider
> > > this failedQE. Thx!
> > 
> > https://bugzilla.redhat.com/1366879
> 
> The BZ above solves described issue in #2 only when setup - ie.
> hosted-engine - command is involved, see
> https://bugzilla.redhat.com/show_bug.cgi?id=1368399#c2
> 
> IMO this BZ should solve also #2, ie. to raise runtime error when already
> existing 3.6 SHE env is "broken".

Not sure I fully understand what you want, but I deliberately changed the summary (at the time, on clone bug 1367732) to make it clear what this bug is about. Current summary is "If ovirt-ha-agent fails to read local answers.conf during upgrade, it writes None to shared fhanswers.conf". Does this still happen? If not, it's a different bug. Please open one and describe it clearly. Thanks.

Comment 6 Yedidyah Bar David 2016-09-13 12:30:43 UTC
(In reply to Yedidyah Bar David from comment #5)
> (In reply to Jiri Belka from comment #4)
> > IMO this BZ should solve also #2, ie. to raise runtime error when already
> > existing 3.6 SHE env is "broken".
> 
> Not sure I fully understand what you want, but I deliberately changed the
> summary (at the time, on clone bug 1367732) to make it clear what this bug
> is about. Current summary is "If ovirt-ha-agent fails to read local
> answers.conf during upgrade, it writes None to shared fhanswers.conf". Does
> this still happen? If not, it's a different bug. Please open one and
> describe it clearly. Thanks.

BTW, I can see two potential bugs you might want to open for this:

1. Make ovirt-ha-agent test for bad fhanswers.conf and fail if found. I really do not see the point in this. What will this be good for? Take a perfectly working setup and break it by a mere yum update? If the only problem is when trying to add a host, and this fails with a reasonable error, that's good enough.

2. Make ovirt-ha-agent automatically fix a bad fhanswers.conf. I think that's way too risky. We didn't get many reports about such cases, and do not expect many in the future, as this code is in GA for more than a year now. To do this manually, people should follow bug 1366879 comment 23.

Comment 7 Jiri Belka 2016-09-19 08:34:17 UTC
(In reply to Yedidyah Bar David from comment #6)
> (In reply to Yedidyah Bar David from comment #5)
> > (In reply to Jiri Belka from comment #4)
> > > IMO this BZ should solve also #2, ie. to raise runtime error when already
> > > existing 3.6 SHE env is "broken".
> > 
> > Not sure I fully understand what you want, but I deliberately changed the
> > summary (at the time, on clone bug 1367732) to make it clear what this bug
> > is about. Current summary is "If ovirt-ha-agent fails to read local
> > answers.conf during upgrade, it writes None to shared fhanswers.conf". Does
> > this still happen? If not, it's a different bug. Please open one and
> > describe it clearly. Thanks.
> 
> BTW, I can see two potential bugs you might want to open for this:
> 
> 1. Make ovirt-ha-agent test for bad fhanswers.conf and fail if found. I
> really do not see the point in this. What will this be good for? Take a
> perfectly working setup and break it by a mere yum update? If the only
> problem is when trying to add a host, and this fails with a reasonable
> error, that's good enough.

I agree. If 3.6 has bad fhanswers.conf it would be catched later on with 4.0 hosted-engine setup (ie. migration to 4.0).

> 2. Make ovirt-ha-agent automatically fix a bad fhanswers.conf. I think
> that's way too risky. We didn't get many reports about such cases, and do
> not expect many in the future, as this code is in GA for more than a year
> now. To do this manually, people should follow bug 1366879 comment 23.

I agree.

Comment 8 Jiri Belka 2016-09-19 08:41:30 UTC
ok, based on #4 and #7

3.6.9

Comment 10 errata-xmlrpc 2016-09-21 17:54:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-1924.html


Note You need to log in before you can comment on or make changes to this bug.