Bug 1276740 - race condition refreshing /var/lib/ovirt-hosted-engine-ha/broker.conf
Summary: race condition refreshing /var/lib/ovirt-hosted-engine-ha/broker.conf
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-hosted-engine-ha
Classification: oVirt
Component: Broker
Version: 1.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ovirt-3.6.0-rc3
: 1.3.2.1
Assignee: Simone Tiraboschi
QA Contact: Artyom
URL:
Whiteboard: integration
: 1277137 (view as bug list)
Depends On: 1278130
Blocks: 1234906 RHEV3.6Upgrade
TreeView+ depends on / blocked
 
Reported: 2015-10-30 17:19 UTC by Simone Tiraboschi
Modified: 2015-12-22 13:29 UTC (History)
7 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2015-12-22 13:29:26 UTC
oVirt Team: Integration
Embargoed:
rule-engine: ovirt-3.6.0+
rule-engine: blocker+
bmcclain: planning_ack+
dfediuck: devel_ack+
gklein: testing_ack+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 47908 0 master MERGED sharedstorageconf: avoid upgrading local conf file is not valid Never
oVirt gerrit 47953 0 ovirt-hosted-engine-ha-1.3 MERGED sharedstorageconf: avoid upgrading local conf file is not valid Never
oVirt gerrit 47960 0 master MERGED upgrade: fixing broker.conf path after rpm upgrade Never
oVirt gerrit 47961 0 ovirt-hosted-engine-ha-1.3 MERGED upgrade: fixing broker.conf path after rpm upgrade Never

Description Simone Tiraboschi 2015-10-30 17:19:21 UTC
Description of problem:
rpm upgrade of ovirt-ha-agent will move the broker configuration file from
/etc/ovirt-hosted-engine-ha/broker.conf
to
/var/lib/ovirt-hosted-engine-ha/broker.conf

The agent upgrade procedure will save /var/lib/ovirt-hosted-engine-ha/broker.conf 
on the shared storage.
The broker will periodically refresh from there to /var/lib/ovirt-hosted-engine-ha/broker.conf 

Due to race condition /var/lib/ovirt-hosted-engine-ha/broker.conf can be overwritten with none before it got copied to the shared storage.

Version-Release number of selected component (if applicable):


How reproducible:
race condition

Steps to Reproduce:
1. upgrade form 3.5 to 3.6
2.
3.

Actual results:
'None' could get written in /var/lib/ovirt-hosted-engine-ha/broker.conf if the race condition happens and than it got copied to the shared storage like that.

Expected results:
var/lib/ovirt-hosted-engine-ha/broker.conf is got overwritten only if the content from the shared storage is valid.

Additional info:

Comment 1 Red Hat Bugzilla Rules Engine 2015-11-01 16:24:09 UTC
This bug is not marked for z-stream, yet the milestone is for a z-stream version, therefore the milestone has been reset.
Please set the correct milestone or add the z-stream flag.

Comment 2 Simone Tiraboschi 2015-11-02 14:33:24 UTC
*** Bug 1277137 has been marked as a duplicate of this bug. ***

Comment 3 Artyom 2015-12-01 16:23:54 UTC
Verified on ovirt-hosted-engine-ha-1.3.3-1.el7ev.noarch
1) Deploy hosted-engine 3.5 on two hosts and on NFS storage
2) Put first host to maintenance via webadmin
3) Upgrade packages and restart host(restart host W/A because bug https://bugzilla.redhat.com/show_bug.cgi?id=1282187)
4) Wait for correct status via hosted-engine --vm-status(can take around 5-7 minutes)
5) Activate host via webadmin
6) Put second host to maintenance(wait until all vms migrated and he vm migrate on first host)
7) Upgrade packages and restart second host
8) Wait for correct status via hosted-engine --vm-status(can take around 5-7 minutes)
9) Activate second host via webadmin
10) Put environment to global maintenance
11) Update rhevm-setup.noarch package on engine
12) Run engine-setup on vm and finish upgrade process
13) Disable global maintenance via webadmin

Comment 4 Sandro Bonazzola 2015-12-22 13:29:26 UTC
oVirt 3.6.0 has been released and the bz verified, moving to closed current release.


Note You need to log in before you can comment on or make changes to this bug.