Bug 1543902
| Summary: | Include /etc/hosts and /etc/nsswitch.conf into initramfs if fence_kdump is used otherwise crashkernel doesn't resolve the hostnames | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Josef Zimek <pzimek> |
| Component: | kexec-tools | Assignee: | Pingfan Liu <piliu> |
| Status: | CLOSED ERRATA | QA Contact: | Qiao Zhao <qzhao> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.4 | CC: | bhsharma, kcleveng, kdump-team-bugs, piliu, pzimek, ruyang, sbradley |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | kexec-tools-2.0.15-15.el7 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-10-30 11:29:30 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1548445, 1549423 | ||
Hello, The "dracut_args --include /etc/hosts /etc/nsswitch.conf" workaround you mentioned above does not work,mainly because the side effect of that entry line results in the content of file /etc/hosts being installed as /etc/nsswitch.conf. After inspecting how /usr/sbin/mkdumprd build the dracut_args array variable it receives as first argument to dracut I realized that probably you intended to use --install instead of --include. The former allows you to install the content of several files into the target kdump ramdisk, while the latter can only be used once, assuming that what is stated in dracut man pages is correct. Indeed, I checked that using several --include options (it does not matter if they appear in an unique line or in several dracut_args entries in /etc/kdump.conf), results in dracut complaining about syntax issues and aborting. On the other side, using --install instead of --include takes you to a correct, consistent and complete ramdisk-kdump image: This is the line I used in /etc/kdump.conf: dracut_args --install "/etc/nsswitch.conf" --install "/etc/hosts" Jesús. (In reply to Josef Zimek from comment #0) > Description of problem: > > fence_kdump_nodes is required parameter in case fence_kdump is used. If user > doesn't specify fence_kdump_nodes in /etc/kdump.conf the list of cluster > nodes is obtained from cluster automatically and that value is passed to > fence_kdump_nodes: > > > # retrieves fence_kdump nodes from Pacemaker cluster configuration > get_pcs_fence_kdump_nodes() { > local nodes > > # get cluster nodes from cluster cib, get interface and ip address > nodelist=`pcs cluster cib | xmllint --xpath > "/cib/status/node_state/@uname" -` > > # nodelist is formed as 'uname="node1" uname="node2" ... uname="nodeX"' > # we need to convert each to node1, node2 ... nodeX in each iteration > for node in ${nodelist}; do > # convert $node from 'uname="nodeX"' to 'nodeX' > eval $node > nodename=$uname > # Skip its own node name > if [ "$nodename" = `hostname` -o "$nodename" = `hostname -s` ]; then > continue > fi > nodes="$nodes $nodename" > done > > echo $nodes > } > > > Problem with above approach is that cluster node names from pacemaker doen't > necessarily have to be FQDN but can be user defined aliases. So this way we > feed fence_kdump_nodes with list of aliases which are typically mapped in > /etc/hosts but /etc/hosts is not included in initramfs by dracut by default. > There are multiple reasons why it is not included and there is low change > get it there by default. However it make sense to include /etc/hosts (and > /etc/nsswitch.conf to make it work) when fence_kdump is used because without > that hostname aliases are not resolvable from crashkernel which leads to > fence_kdump failure (fence_kdump_send fails to send notifications and > fencing results in timeout = vmcore doesn't get captured). > Hi, do you hit this issue? During mkdumprd, it stores the ipaddr, instead of the hostname. Hence alias will not cause the problem. Thanks, Pingfan But, it is harmless to include hosts and nsswitch.conf. If debugging in kdump shell, we try to scp something to alias. My test environment is broken. After adding rd.debug in cmdline, I found that fence_kdump_send cmd has no node as its input param. Hence during my test, I did not hit the condition (fence_kdump_send fails to send notifications and fencing results in timeout = vmcore doesn't get captured). It turns out that we need this fix. Thanks, Pingfan Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:3240 |
Description of problem: fence_kdump_nodes is required parameter in case fence_kdump is used. If user doesn't specify fence_kdump_nodes in /etc/kdump.conf the list of cluster nodes is obtained from cluster automatically and that value is passed to fence_kdump_nodes: # retrieves fence_kdump nodes from Pacemaker cluster configuration get_pcs_fence_kdump_nodes() { local nodes # get cluster nodes from cluster cib, get interface and ip address nodelist=`pcs cluster cib | xmllint --xpath "/cib/status/node_state/@uname" -` # nodelist is formed as 'uname="node1" uname="node2" ... uname="nodeX"' # we need to convert each to node1, node2 ... nodeX in each iteration for node in ${nodelist}; do # convert $node from 'uname="nodeX"' to 'nodeX' eval $node nodename=$uname # Skip its own node name if [ "$nodename" = `hostname` -o "$nodename" = `hostname -s` ]; then continue fi nodes="$nodes $nodename" done echo $nodes } Problem with above approach is that cluster node names from pacemaker doen't necessarily have to be FQDN but can be user defined aliases. So this way we feed fence_kdump_nodes with list of aliases which are typically mapped in /etc/hosts but /etc/hosts is not included in initramfs by dracut by default. There are multiple reasons why it is not included and there is low change get it there by default. However it make sense to include /etc/hosts (and /etc/nsswitch.conf to make it work) when fence_kdump is used because without that hostname aliases are not resolvable from crashkernel which leads to fence_kdump failure (fence_kdump_send fails to send notifications and fencing results in timeout = vmcore doesn't get captured). Same issue arrives not only when we automatically feed the fence_kdump_nodes with values obtained from pacemaker but also when user manually enters aliases into /etc/kdump.conf because we allow that. Kdump will not complain because when creating initramfs it resolves the alias but from crashkernel alias is not available. So currently it only works if fence_kdump_nodes contains a) IP addresses of cluster nodes b) FQDNs of cluster nodes. There is workaround by setting the dracut_args in /etc/kdump.conf: dracut_args --include /etc/hosts /etc/nsswitch.conf However it sounds reasonable to include it by default. Version-Release number of selected component (if applicable): Tested on kexec-tools-2.0.14 however it is very likely applicable to all RHEL 7 releases How reproducible: Always Steps to Reproduce: 1. set cluster with hostnames which are not FQDNs but aliases 2. map the aliases to IPs in /etc/hosts file 3. set fence_kdump as primary fencing method 4. restart kdump 5. crash the node 6. fencing of crashed node will timeout because Actual results: Fencing fails due to unresolvable hostnames in fence_kdump_nodes (vmcore not captured) Expected results: Fencing will succeed (vmcore gets captured) because aliases get resolved from within crashkernel because /etc/hosts and /etc/nsswitch.conf are available in case fence_kdump is installed Additional info: tested with: dracut_args --include /etc/hosts /etc/nsswitch.conf and it works as expected