Bug 1268969
| Summary: | oVirt Hosted Engine is not reachable after engine setup | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [oVirt] ovirt-hosted-engine-setup | Reporter: | joselito.angeles | ||||||
| Component: | General | Assignee: | Simone Tiraboschi <stirabos> | ||||||
| Status: | CLOSED NOTABUG | QA Contact: | Pavel Stehlik <pstehlik> | ||||||
| Severity: | medium | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | --- | CC: | bugs, didi, joselito.angeles, lveyde, rmartins, sbonazzo, stirabos | ||||||
| Target Milestone: | --- | Flags: | rule-engine:
planning_ack?
rule-engine: devel_ack? rule-engine: testing_ack? |
||||||
| Target Release: | --- | ||||||||
| Hardware: | x86_64 | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2015-10-07 09:31:52 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Could you please also attach ovirt-hosted-engine-setup logs from /var/log/ovirt-hosted-engine-setup/ on your host? Created attachment 1080080 [details]
ovirt hosted engine setup log
Hi Simone,
Here's the log from the host.
Thanks.
Could you please try to execute curl http://ovirt-engine1.perflab.iint/ovirt-engine/services/health on the host you are deploying? Is it getting DB Up!Welcome to Health Status! also from there? I am not getting the DB Up!Welcome to Health Status from the host when I run curl. The host can resolve the ipaddress using the hostname and can also do a reverse lookup though. [root@usatl-s-smperf1 ~]# curl http://ovirt-engine1.perflab.iint/ovirt-engine/services/health <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"> <html> <head> <style> a:link {font:8pt/11pt verdana; color:red} a:visited {font:8pt/11pt verdana; color:#4e4e4e} </style> <meta HTTP-EQUIV="Content-Type" Content="text-html; charset=Windows-1252"> <title>Cannot find server</title> </head> <body bgcolor="white"> <table width="400" cellpadding="3" cellspacing="5"> <tr> <td id="tableProps2" align="left" valign="middle" width="360"><h1 id="textSection1" style="COLOR: black; FONT: 13pt/15pt verdana"><span id="errorText">The page cannot be displayed</span></h1> </td> </tr> <tr> <td id="tablePropsWidth" width="400" colspan="2"><font style="COLOR: black; FONT: 8pt/11pt verdana">The page you are looking for is currently unavailable. The Web site might be experiencing technical difficulties, or you may need to adjust your browser settings.</font></td> </tr> <tr> <td id="tablePropsWidth" width="400" colspan="2"><font id="LID1" style="COLOR: black; FONT: 8pt/11pt verdana"><hr color="#C0C0C0" noshade> <p id="LID2">Please try the following:</p><ul> <li id="instructionsText2">If you typed the page address in the Address bar, make sure that it is spelled correctly.<br> </li> <li id="instructionsText4"> If you are trying to reach a secure site, make sure your Security settings can support it. </li> <li id="list3">Click the <a href="javascript:history.back(1)"> Back</a> button to try another link. </li> </ul> <p><br> </p> <h2 style="font:8pt/11pt verdana; color:black">DNS Fail</h2> </font></td> </tr> </table> </body> </html> [root@usatl-s-smperf1 ~]# nslookup 172.16.128.131 Server: 172.16.128.21 Address: 172.16.128.21#53 131.128.16.172.in-addr.arpa name = ovirt-engine1.perflab.iint. [root@usatl-s-smperf1 ~]# nslookup ovirt-engine1.perflab.iint Server: 172.16.128.21 Address: 172.16.128.21#53 Name: ovirt-engine1.perflab.iint Address: 172.16.128.131 [root@usatl-s-smperf1 ~]# ping ovirt-engine1.perflab.iint. PING ovirt-engine1.perflab.iint (172.16.128.131) 56(84) bytes of data. 64 bytes from ovirt-engine1 (172.16.128.131): icmp_seq=1 ttl=64 time=0.302 ms 64 bytes from ovirt-engine1 (172.16.128.131): icmp_seq=2 ttl=64 time=0.349 ms If the engine is reachable, the heath servlet should respond 'DB Up!Welcome to Health Status!' on http://ovirt-engine1.perflab.iint/ovirt-engine/services/health. On the first message you said that you get 'DB Up!Welcome to Health Status!' if you access it from the browser, is it still true? Is there on your network any kind of reverse proxy that requires special authentication or something similar? "<title>Cannot find server</title>" on the response you posted looks quite surprising, at least it should be <title>404 - Page not found</title> Could you please check if your request for the health page got traced under /var/log/httpd/access_log on your engine VM? if not your host is accessing something else. Yes, I can access the health status from a browser and from other servers on the same network except from the deployment host. the requests are being traced on the engine VM. The deployment host is 172.16.128.81, the browser is on 172.16.128.39 and the other host (solaris) is on 172.16.128.103. I still have the session for deploy running on 172.1`6.128.81 so its continually trying to get the health status. 172.16.128.103 - - [06/Oct/2015:11:56:31 -0400] "GET /ovirt-engine/services/health HTTP/1.1" 200 31 "-" "curl/7.21.2 (sparc-sun-solaris2.11) libcurl/7.21.2 OpenSSL/1.0.1h zlib/1.2.3-T4mods libidn/1.19" 172.16.128.81 - - [06/Oct/2015:11:56:39 -0400] "GET /ovirt-engine/services/health HTTP/1.1" 200 31 "-" "Python-urllib/2.7" 172.16.128.81 - - [06/Oct/2015:11:56:49 -0400] "GET /ovirt-engine/services/health HTTP/1.1" 200 31 "-" "Python-urllib/2.7" 172.16.128.81 - - [06/Oct/2015:11:56:59 -0400] "GET /ovirt-engine/services/health HTTP/1.1" 200 31 "-" "Python-urllib/2.7" 172.16.128.81 - - [06/Oct/2015:11:57:09 -0400] "GET /ovirt-engine/services/health HTTP/1.1" 200 31 "-" "Python-urllib/2.7" ... 172.16.128.39 - - [06/Oct/2015:11:59:32 -0400] "GET /ovirt-engine/services/health HTTP/1.1" 200 51 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)" 172.16.128.39 - - [06/Oct/2015:11:59:32 -0400] "GET /favicon.ico HTTP/1.1" 404 209 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)" 172.16.128.81 - - [06/Oct/2015:11:59:39 -0400] "GET /ovirt-engine/services/health HTTP/1.1" 200 31 "-" "Python-urllib/2.7" 172.16.128.39 - - [06/Oct/2015:11:59:41 -0400] "GET /ovirt-engine/services/health HTTP/1.1" 200 51 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/6.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)" 172.16.128.81 - - [06/Oct/2015:11:59:49 -0400] "GET /ovirt-engine/services/health HTTP/1.1" 200 31 "-" "Python-urllib/2.7" Is 172.16.128.81 your host? Here I see 200 OK 172.16.128.81 - - [06/Oct/2015:11:59:49 -0400] "GET /ovirt-engine/services/health HTTP/1.1" 200 31 "-" "Python-urllib/2.7" Hi Simone, It was my proxy server that's causing the issue. I disabled my proxy and was able to get the DB Up! Thanks for your help. I appreciate your quick responses. --Joselito [root@usatl-s-smperf1 ovirt-hosted-engine-ha]# curl -I http://ovirt-engine1.perflab.iint/ovirt-engine/services/health HTTP/1.0 200 OK Date: Tue, 06 Oct 2015 17:55:57 GMT Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips Content-Type: text/html; charset=UTF-8 Content-Length: 31 X-Cache: MISS from usatl-s-ibmperf3 X-Cache-Lookup: MISS from usatl-s-ibmperf3:3128 Via: 1.0 usatl-s-ibmperf3 (squid/3.1.10) Connection: keep-alive [root@usatl-s-smperf1 ovirt-hosted-engine-ha]# env|grep proxy http_proxy=http://plproxy.perflab.iint:3128 ftp_proxy=ftp://plproxy.perflab.iint:3128 https_proxy=https://plproxy.perflab.iint:3128 [root@usatl-s-smperf1 ovirt-hosted-engine-ha]# unset http_proxy [root@usatl-s-smperf1 ovirt-hosted-engine-ha]# unset ftp_proxy [root@usatl-s-smperf1 ovirt-hosted-engine-ha]# unset https_proxy [root@usatl-s-smperf1 ovirt-hosted-engine-ha]# env|grep proxy [root@usatl-s-smperf1 ovirt-hosted-engine-ha]# curl http://ovirt-engine1.perflab.iint/ovirt-engine/services/health DB Up!Welcome to Health Status![root@usatl-s-smperf1 ovirt-hosted-engine-ha]# [root@usatl-s-smperf1 ovirt-hosted-engine-ha]# curl -I http://ovirt-engine1.perflab.iint/ovirt-engine/services/health HTTP/1.1 200 OK Date: Tue, 06 Oct 2015 17:55:17 GMT Server: Apache/2.4.6 (Red Hat Enterprise Linux) OpenSSL/1.0.1e-fips Content-Type: text/html; charset=UTF-8 Content-Length: 31 I set the no_proxy variable to include all local hosts (including the hosted-engine VM) and it's working fine now with proxy enabled. Thanks again for your help. |
Created attachment 1080022 [details] vdsm, engine, server and setup logs Description of problem: I have deployed ovirt 3.5 hosted engine from a RHEL 7.1 host to a RHEL 7.1 VM. After the engine setup was completed successfully on the VM, I went back to the host where I deployed the engine and selected option [1] but got the message below. [ ERROR ] Engine is still unreachable Engine health status page is not yet reachable. To continue make a selection from the options below: If I run vm-status it returns the engine status as up and healthy. # hosted-engine --vm-status --== Host 1 status ==-- Status up-to-date : True Hostname : usatl-s-smperf1 Host ID : 1 Engine status : {"health": "good", "vm": "up", "detail": "up"} Score : 2400 Local maintenance : False Host timestamp : 247344 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=247344 (Mon Oct 5 13:58:35 2015) host-id=1 score=2400 maintenance=False state=EngineUp But if I run check-liveliness it returns the message below. # hosted-engine --check-liveliness No handlers could be found for logger "otopi.__main__" Hosted Engine is not up! I can access and log on to the engine web admin portal via a browser. I also get "DB Up! Welcome to Health Status!" if I go to https://<FQDN>/ovirt-engine/services/health I tried stopping iptables on both host and engine but got the same error back. I have also tried redeploying the engine but ended up with the same problem. Please advise. Thanks. Version-Release number of selected component (if applicable): ovirt-hosted-engine-setup-1.2.5.3-1.el7.centos.noarch ovirt-engine-sdk-python-3.5.4.0-1.el7.centos.noarch libgovirt-0.3.1-3.el7.x86_64 ovirt-hosted-engine-ha-1.2.6.1-1.el7.centos.noarch ovirt-host-deploy-1.3.2-1.el7.centos.noarch ovirt-engine-3.5.4.2-1.el7.centos.noarch Red Hat Enterprise Linux Server release 7.1 (Maipo) Linux ... 3.10.0-229.el7.x86_64 #1 SMP Thu Jan 29 18:37:38 EST 2015 x86_64 x86_64 x86_64 GNU/Linux Actual results: After selecting option 1, got the message below. [ ERROR ] Engine is still unreachable Engine health status page is not yet reachable. Expected results: I expect to get the output "Engine Replied:DB Up! Welcome to Health Status!" then the installer will register itself ad the virtualization host Additional info: I am using gluster volumes as shared storage.