Bug 1856375 - Can't add additional host as hosted-engine ha-host from "Guide me" from UI.
Summary: Can't add additional host as hosted-engine ha-host from "Guide me" from UI.
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Infra
Version: 4.4.1.8
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ovirt-4.4.4
: 4.4.4.2
Assignee: Artur Socha
QA Contact: Nikolai Sednev
URL:
Whiteboard:
Depends On: 1868571
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-07-13 13:38 UTC by Nikolai Sednev
Modified: 2020-12-21 12:36 UTC (History)
5 users (show)

Fixed In Version: ovirt-engine-4.4.4.2
Clone Of:
Environment:
Last Closed: 2020-12-21 12:36:24 UTC
oVirt Team: Infra
Embargoed:
pm-rhel: ovirt-4.4+


Attachments (Terms of Use)
screencast-2020-07-13_16.32.12.mkv (1.24 MB, application/x-matroska)
2020-07-13 13:40 UTC, Nikolai Sednev
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 110281 0 master MERGED webadmin: hosted engine tab on guideme new host 2020-12-21 16:17:57 UTC
oVirt gerrit 111399 0 master MERGED webadmin: Add/deploy HE host from guide me 2020-12-21 16:18:00 UTC
oVirt gerrit 111984 0 master MERGED webadmin: 'activate' flag on add host via guide-me 2020-12-21 16:18:00 UTC
oVirt gerrit 112001 0 master MERGED webadmin: guide-me add host synced with 'add host' 2020-12-21 16:18:00 UTC

Description Nikolai Sednev 2020-07-13 13:38:31 UTC
Description of problem:
There is no "Hosted Engine" option for adding additional host from UI through the "Guide Me" flow. 


Version-Release number of selected component (if applicable):
Software Version:4.4.1.8-0.7.el8ev
ovirt-hosted-engine-setup-2.4.5-1.el8ev.noarch
ovirt-hosted-engine-ha-2.4.4-1.el8ev.noarch
Red Hat Enterprise Linux release 8.2 (Ootpa)
Linux 4.18.0-193.13.1.el8_2.x86_64 #1 SMP Tue Jul 7 14:03:09 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux


How reproducible:
100%

Steps to Reproduce:
1. Deploy hosted-engine on one host.
2. Click on "Clusters" or on "Data Centers" and on "Guide Me".
3.Select "+Add another Host".
3. Click on "Configure Host".

Actual results:
There is no "Hosted Engine" option for adding additional host.

Expected results:
"Hosted Engine" option should exist through "Guide Me" flow.

Additional info:
Short movie is attached.

Comment 1 Nikolai Sednev 2020-07-13 13:40:08 UTC
Created attachment 1700849 [details]
screencast-2020-07-13_16.32.12.mkv

Comment 2 Martin Perina 2020-07-13 14:26:08 UTC
As a workaround please use New button in Hosts view

Comment 3 Nikolai Sednev 2020-07-13 14:57:07 UTC
(In reply to Martin Perina from comment #2)
> As a workaround please use New button in Hosts view

Yes, indeed that in "Hosts" this function is working properly like it had worked in previous versions.

Comment 8 Nikolai Sednev 2020-08-24 15:01:33 UTC
Additional HE-ha host being added now and the option appears in UI, but host being added as regular host, although I literally chose to add it as "Deploy" in Hosted Engine section via "Guide me" option.
Tested on these components:
ovirt-engine-setup-4.4.2.3-0.6.el8ev.noarch
ovirt-ansible-hosted-engine-setup-1.1.8-1.el8ev.noarch
ovirt-hosted-engine-setup-2.4.6-1.el8ev.noarch
ovirt-hosted-engine-ha-2.4.4-1.el8ev.noarch
Red Hat Enterprise Linux release 8.2 (Ootpa)
Linux 4.18.0-193.14.3.el8_2.x86_64 #1 SMP Mon Jul 20 15:02:29 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Moving back to assigned, as required functionality not met.

The additionally added host also appears under local maintenance after the addition process finishes, although it have to activate the host at the end of the addition.

Comment 9 RHEL Program Management 2020-08-24 15:01:39 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 10 Nikolai Sednev 2020-10-14 11:53:41 UTC
Host was added to the cluster in set to maintenance mode, while via hosts tab its being activated right from the start after addition. I reactivated it and then it failed to become ha-active.
The newly added ha-host should not remain in maintenance after addition, just like adding regularly ha-capable hosts via hosts tab.

When I tried to add host I also probably hit https://bugzilla.redhat.com/show_bug.cgi?id=1664159:
MainThread::ERROR::2020-10-13 17:41:47,029::hosted_engine::457::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngin
e::(start_monitoring) Unhandled monitoring loop exception
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 454, in start_monitoring
    self._monitoring_loop()
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 473, in _monitoring_loop
    for old_state, state, delay in self.fsm:
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/fsm/machine.py", line 127, in __next__
    new_data = self.refresh(self._state.data)
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/state_machine.py", line 81, in refresh
    stats.update(self.hosted_engine.collect_stats())
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 761, in collect_stats
    all_stats = self._broker.get_stats_from_storage()
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 148, in get_stats_from_storage
    result = self._proxy.get_stats()
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1112, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1452, in __request
    verbose=self.__verbose
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1154, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1166, in single_request
    http_conn = self.send_request(host, handler, request_body, verbose)
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1279, in send_request
    self.send_content(connection, request_body)
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1309, in send_content
    connection.endheaders(request_body)
  File "/usr/lib64/python3.6/http/client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/usr/lib64/python3.6/http/client.py", line 974, in send
    self.connect()
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 74, in connect
    self.sock.connect(base64.b16decode(self.host))
FileNotFoundError: [Errno 2] No such file or directory
MainThread::ERROR::2020-10-13 17:41:47,033::agent::143::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent
    return action(he)
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper
    return he.start_monitoring()
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 461, in start_monitoring
    self.publish(stopped)
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 361, in publish
    self._push_to_storage(blocks)
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 732, in _push_to_storage
    self._broker.put_stats_on_storage(self.host_id, blocks)
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 118, in put_stats_on_storage
    self._proxy.put_stats(host_id, xmlrpc_client.Binary(data.encode()))
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1112, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1452, in __request
    verbose=self.__verbose
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1154, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1279, in send_request
    self.send_content(connection, request_body)
  File "/usr/lib64/python3.6/xmlrpc/client.py", line 1309, in send_content
    connection.endheaders(request_body)
  File "/usr/lib64/python3.6/http/client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/usr/lib64/python3.6/http/client.py", line 974, in send
    self.connect()
  File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 74, in connect
    self.sock.connect(base64.b16decode(self.host))
FileNotFoundError: [Errno 2] No such file or directory

MainThread::ERROR::2020-10-13 17:41:47,033::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent
MainThread::INFO::2020-10-13 17:41:47,033::agent::89::ovirt_hosted_engine_ha.agent.agent.Agent::(run) Agent shutting down
MainThread::INFO::2020-10-13 17:41:57,424::agent::67::ovirt_hosted_engine_ha.agent.agent.Agent::(run) ovirt-hosted-engine-ha agent 2.4.5 started
MainThread::INFO::2020-10-13 17:41:57,435::hosted_engine::242::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_hostname) Certificate common name not found, using hostname to identify host
MainThread::INFO::2020-10-13 17:41:57,509::hosted_engine::548::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker) Initializing ha-broker connection
MainThread::INFO::2020-10-13 17:41:57,510::brokerlink::82::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Starting monitor network, options {'addr': '10.35.95.254', 'network_test': 'dns', 'tcp_t_address': '', 'tcp_t_port': ''}

These came from vdsm:
2020-10-14 14:31:49,011+0300 INFO  (monitor/2e15dc3) [storage.LVM] Error deactivating VG 2e15dc35-1140-447d-ba75-21188
65391f5: rc=5 out=[] err=['  Logical volume 2e15dc35-1140-447d-ba75-2118865391f5/62ab791c-717b-4080-aea8-1c20479efbf3 
in use.', '  Can\'t deactivate volume group "2e15dc35-1140-447d-ba75-2118865391f5" with 1 open logical volume(s)'] (lv
m:1491)
2020-10-14 14:31:49,133+0300 ERROR (monitor/2e15dc3) [storage.LVM] Removing VG mapping failed: Could not remove mappin
g: Command ['/usr/sbin/dmsetup', 'remove', '2e15dc35--1140--447d--ba75--2118865391f5-62ab791c--717b--4080--aea8--1c204
79efbf3'] failed with rc=1 out=b'' err=b'device-mapper: remove ioctl on 2e15dc35--1140--447d--ba75--2118865391f5-62ab7
91c--717b--4080--aea8--1c20479efbf3  failed: Device or resource busy\nCommand failed.\n' (lvm:1167)
2020-10-14 14:31:49,219+0300 INFO  (jsonrpc/1) [vdsm.api] FINISH disconnectStoragePool return=True from=::ffff:10.35.9
2.51,58208, task_id=1b1d61b5-81e7-4af4-9502-872b818c2c8a (api:54)
2020-10-14 14:31:49,219+0300 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call StoragePool.disconnect succeeded in 14.01 seconds (__init__:312)
2020-10-14 14:31:49,220+0300 INFO  (jsonrpc/7) [vdsm.api] FINISH stopMonitoringDomain return=None from=::1,46534, task_id=022ce5f0-7002-4581-97fc-316b626b138b (api:54)
2020-10-14 14:31:49,220+0300 INFO  (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.stopMonitoringDomain succeeded in 8.51 seconds (__init__:312)
2020-10-14 14:31:49,267+0300 INFO  (jsonrpc/2) [vdsm.api] START disconnectStorageServer(domType=3, spUUID='705f4c8a-0896-11eb-9edc-00163e7bb853', conList=[{'password': '********', 'port': '3260', 'iqn': 'iqn.2009-01.com.kaminario:storage.k2.22612', 'connection': '10.35.162.67', 'ipv6_enabled': 'false', 'id': '31c25a1b-5809-4048-8ff2-a14d089941d0', 'user': '', 'tpgt': '1'}, {'password': '********', 'port': '3260', 'iqn': 'iqn.2009-01.com.kaminario:storage.k2.22612', 'connection': '10.35.162.66', 'ipv6_enabled': 'false', 'id': '6f147f7a-4f5a-4647-bbe9-e33870fd61ed', 'user': '', 'tpgt': '1'}], options=None) from=::ffff:10.35.92.51,58208, flow_id=2f55a573, task_id=2c2edb6e-4a79-44c8-a8a0-467e2cdb83ff (api:48)

Moving back to assigned due to the inconsistent behavior after addition, while compared to regular addition of ha-host.

Comment 11 Nikolai Sednev 2020-10-14 11:54:24 UTC
Tested on:
rhvm-4.4.3.6-0.13.el8ev.noarch
ovirt-hosted-engine-setup-2.4.7-2.el8ev.noarch
ovirt-hosted-engine-ha-2.4.5-1.el8ev.noarch
Linux 4.18.0-240.el8.x86_64 #1 SMP Wed Sep 23 05:13:10 EDT 2020 x86_64 x86_64 x86_64 GNU/Linux
Red Hat Enterprise Linux release 8.3 (Ootpa)

Comment 12 Nikolai Sednev 2020-10-14 12:02:29 UTC
After some time ~10 minutes host became ha-active and overcame https://bugzilla.redhat.com/show_bug.cgi?id=1664159 issue.
Still there is inconsistency between regular addition and through "Guide Me" from datacenter tab.
Host should not be set in to local maintenance after it's addition using "Guide Me" option, it should become active like it does from regular addition.

Comment 13 Casper (RHV QE bot) 2020-11-26 22:31:45 UTC
This bug has low overall severity and passed an automated regression suite, and is not going to be further verified by QE. If you believe special care is required, feel free to re-open to ON_QA status.

Comment 14 Nikolai Sednev 2020-11-27 21:58:55 UTC
Works fine on:
rhvm-4.4.4.1-0.1.el8ev.noarch
Red Hat Enterprise Linux release 8.3 (Ootpa)
Linux 4.18.0-240.5.1.el8_3.x86_64 #1 SMP Tue Nov 17 14:29:54 EST 2020 x86_64 x86_64 x86_64 GNU/Linux

Comment 15 Sandro Bonazzola 2020-12-21 12:36:24 UTC
This bugzilla is included in oVirt 4.4.4 release, published on December 21st 2020.

Since the problem described in this bug report should be resolved in oVirt 4.4.4 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.