Bug 1552235
| Summary: | Prometheus is unable to scrape hosted router components due to iptables rules from openshift-ansible | ||||||
|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | David H <david_hocky> | ||||
| Component: | Monitoring | Assignee: | Paul Gier <pgier> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Junqi Zhao <juzhao> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | medium | ||||||
| Version: | 3.7.0 | CC: | aivaraslaimikis, aos-bugs, cstark, jmalde, jokerman, juzhao, klaas, ksalunkh, mfojtik, minden, Miranda_Shutt, mmccomas, pasik, sauchter, spasquie, surbania, travi | ||||
| Target Milestone: | --- | ||||||
| Target Release: | 3.10.z | ||||||
| Hardware: | All | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: |
Cause: The default firewall settings block the router stats/metrics port.
Consequence: This prevents prometheus from collecting the metrics from the openshift router.
Fix: Open the firewall to allow connections to the router stats port.
Result: Prometheus can now collect metrics from the router.
|
Story Points: | --- | ||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2019-02-20 10:11:10 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
|
Description
David H
2018-03-06 18:50:45 UTC
*** Bug 1589023 has been marked as a duplicate of this bug. *** *** Bug 1625510 has been marked as a duplicate of this bug. *** The upstream issue was closed, this is not correct. I still can't access all routers in a multi-infrastructure node setup. It can only access one router -- my guess: the one thats running on the same node as prometheus. Hello Team, Any updates on this issue Regards, Kedar I don't see an easy way to open the router metrics port (1936) during install for only the router nodes since the node firewall configuration takes place mostly before anything is done with the routers. Also, even if we could do that, I'm not sure how it would work post install if for example you wanted to move a router to a different node, you'd still need to manually open that port. So I've created a PR against 3.10 to optionally open that port for all nodes during install. https://github.com/openshift/openshift-ansible/pull/11052 Tested with openshift-ansible-3.10.110-1.git.0.1e03ab3.el7.noarch.rpm openshift-ansible-docs-3.10.110-1.git.0.1e03ab3.el7.noarch.rpm openshift-ansible-playbooks-3.10.110-1.git.0.1e03ab3.el7.noarch.rpm openshift-ansible-roles-3.10.110-1.git.0.1e03ab3.el7.noarch.rpm oopenshift-ansible-test-3.10.110-1.git.0.1e03ab3.el7.noarch.rpm 1936 port are opened for all nodes after install # iptables-save | grep 1936 -A KUBE-SEP-DFSWOTRTOQBRAYA4 -s 10.0.77.74/32 -m comment --comment "default/router:1936-tcp" -j KUBE-MARK-MASQ -A KUBE-SEP-DFSWOTRTOQBRAYA4 -p tcp -m comment --comment "default/router:1936-tcp" -m tcp -j DNAT --to-destination 10.0.77.74:1936 -A KUBE-SERVICES ! -s 10.128.0.0/14 -d 172.30.139.20/32 -p tcp -m comment --comment "default/router:1936-tcp cluster IP" -m tcp --dport 1936 -j KUBE-MARK-MASQ -A KUBE-SERVICES -d 172.30.139.20/32 -p tcp -m comment --comment "default/router:1936-tcp cluster IP" -m tcp --dport 1936 -j KUBE-SVC-4JCRTMMYZAAYMIJ2 -A KUBE-SVC-4JCRTMMYZAAYMIJ2 -m comment --comment "default/router:1936-tcp" -j KUBE-SEP-DFSWOTRTOQBRAYA4 -A OS_FIREWALL_ALLOW -p tcp -m state --state NEW -m tcp --dport 1936 -j ACCEPT Created attachment 1533903 [details]
router targets are up
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0328 |