Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
For bugs related to Red Hat Enterprise Linux 5 product line. The current stable release is 5.10. For Red Hat Enterprise Linux 6 and above, please visit Red Hat JIRA https://issues.redhat.com/secure/CreateIssue!default.jspa?pid=12332745 to report new issues.

Bug 510816

Summary: kdump fails to send vmcore via ssh/scp when configured using short hostname
Product: Red Hat Enterprise Linux 5 Reporter: Dave Maley <dmaley>
Component: kexec-toolsAssignee: Neil Horman <nhorman>
Status: CLOSED ERRATA QA Contact: Red Hat Kernel QE team <kernel-qe>
Severity: medium Docs Contact:
Priority: high    
Version: 5.4CC: qcai, syeghiay, tao
Target Milestone: rcKeywords: Regression
Target Release: 5.5   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
A previous enhancement to kdump ensured that kdump would perform reliably even when the DNS on a network changed. However, this required that host names specified in the kdump.conf script needed to be entered as fully- qualified domain names. Therefore, if a hostname was entered in a short form, kdump would fail to dump to a ssh/scp target. The documentation for kdump has been revised to make it clear that hostnames must always be specified as FQDNs. When users specify hostnames in this format, dumps to ssh/scp targets perform as expected.
Story Points: ---
Clone Of:
: 600579 (view as bug list) Environment:
Last Closed: 2010-03-30 07:47:13 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 499522    
Attachments:
Description Flags
kdump config
none
log of failed attempt
none
log of successful attempt
none
resolv.conf from lab
none
patch to ensure that we always use an fqdn
none
[updated] patch to ensure that we always use an fqdn none

Description Dave Maley 2009-07-10 22:36:39 UTC
Description of problem:
when kdump is configured to dump to a ssh/scp target using short hostname the vmcore is written to the local disk and nothing makes it to the remote system.  rolling back to kexec-tools-1.102pre-56.el5 resolves things.

*regression from 5.3 thus escalating to BZ immediately


Version-Release number of selected component (if applicable):
kexec-tools-1.102pre-74.el5


How reproducible:
100%


Steps to Reproduce:
1. configure kdump for scp/ssh to server using short hostname
2. propagate and restart service
3. crash system

  
Actual results:
vmcore captured on localhost


Expected results:
vmcore captured on server defined in kdump.conf


Additional info:
-reproduced in RDU GSS lab
  -> client: hp-dl385g5-1.gsslab.rdu.redhat.com
  -> server: dl385g2.gsslab.rdu.redhat.com
- scp using short hostname does work

Comment 1 Neil Horman 2009-07-11 01:14:19 UTC
I'm out all next week, so this is way too late to get into 5.4.  Proposing for 5.5.

when you say 'short' hostname, I assume you're referring to just the hostname, not the fqdn?  Its not recommended that you use a non-fqdn in your configuration for kdump (all the examples use an fqdn).  If you want to get me your kdump.conf and the serial console log of a kdump session, I might be able to make this a bit more robust, but there are a number of factors in resolving a non-fqdn for kdump, and so this might be a cantfix/wontfix.

Comment 3 Dave Maley 2009-07-13 17:12:10 UTC
> when you say 'short' hostname, I assume you're referring to just the hostname,
> not the fqdn?

correct

> Its not recommended that you use a non-fqdn in your configuration for kdump 
> (all the examples use an fqdn).

Do note that using short hostnames worked in 5.3.  On the same system, w/ same config, I can simply rollback to kexec-tools-1.102pre-56.el5 and it works.  Maybe that was by chance (ie. not expected) however this has been reported to us as being a regression.

Comment 4 Dave Maley 2009-07-13 17:13:58 UTC
Created attachment 351496 [details]
kdump config

config used for lab testing

Comment 5 Dave Maley 2009-07-13 17:15:50 UTC
Created attachment 351497 [details]
log of failed attempt

serial console log of failed attempt (kexec-tools-1.102pre-74.el5)

Comment 6 Dave Maley 2009-07-13 17:17:11 UTC
Created attachment 351498 [details]
log of successful attempt

serial console log of successful attempt (kexec-tools-1.102pre-56.el5)

Comment 7 Dave Maley 2009-07-13 17:19:10 UTC
There's nothing obvious in the log from the failed attempt, it just says:

Saving to remote location kdump-test@dl385g2
lost connection

I'm going to start reviewing the changes between pre-56 and pre-74 to see if I can find any clues ....

Comment 8 Dave Maley 2009-07-13 18:37:38 UTC
I've determined this regression was introduced w/ the mkdumprd change from bug 493690.  Reverting the changes that were introduced in that bug resolves this regression.

Interesting to note that according to:
https://bugzilla.redhat.com/show_bug.cgi?id=493690#c12

..this scenario (short hostname) was tested and reported as working (see case #5).  Tho I suppose it's possible the engineer meant username rather than username@hostname.

Comment 11 Neil Horman 2009-07-13 20:10:27 UTC
Can you please attach the /etc/resolve.conf file for this system thats failing?  Thanks!

Comment 13 Dave Maley 2009-07-13 20:17:45 UTC
Created attachment 351514 [details]
resolv.conf from lab

resolv.conf from lab system used during my reproduction efforts

Comment 15 Neil Horman 2009-07-13 20:23:58 UTC
does it work if you add this line to /etc/resolv.conf:
domain gsslab.rdu.redhat.com

Note that you'll have to touch /etc/kdump.conf and restart the kdump services.

Comment 17 Dave Maley 2009-07-13 21:19:52 UTC
adding the suggested line from comment 15 made no difference.

Comment 18 Neil Horman 2009-07-14 13:35:27 UTC
Created attachment 351584 [details]
patch to ensure that we always use an fqdn

think I found the problem.  In 5.3 we had a request to enhance kdump so that dns changes didn't require kdump restarts (we had previously resolved dns entries statically and just put the ip address in the initramfs.  We fixed this by simply putting the host name in, and the resolution setup in the kdump kernel is preventing simple hostname resolution in our environment. I could enhance the resolve setup so that it would work in our environment, but that doesn't solve the general problem if someones dns requires fqdn's to resolve an address. I think the best thing to do is twofold:
1) Transform non-fqdn's into fqdn's in the mkdumprd script
2) Add documentation to tell people to always use fqdns.

This patch does (1).  If you could test it out please and confirm dave, I'd appreciate it.  Thanks!

Comment 19 Dave Maley 2009-07-14 19:41:54 UTC
Created attachment 351661 [details]
[updated] patch to ensure that we always use an fqdn

Yep this works in my test setup.  I'll get a test pkg to the TAM and request that we get confirmation from the partner as well.

I had to fix a couple typos in the patch and so I've uploaded the corrected version.

Comment 22 Dave Maley 2009-07-20 13:49:21 UTC
partner has verified the proposed fix resolves this regression for them

Comment 25 Dave Maley 2009-08-07 16:51:33 UTC
The document "I cannot get a vmcore sent to a remote host via scp when kdump is configured using a short hostname" has been published in Knowledgebase English space. The article is now visible to the public at the following URL:

http://kbase.redhat.com/faq/docs/DOC-17880

Comment 26 Issue Tracker 2009-09-04 09:59:51 UTC
Event posted on 09-04-2009 06:59pm JST by mfuruta

Hi Samukawa-san,

----
Since RHEL5.4 is released, customers have to change their configuration
to specify the server by IP address or FQDN.  Even if we change this
behavior in RHEL5.5, it will leave a different limitation in how the
hostname could be specified.  The knowledgebase must be changed again
in RHEL5.5 and it will confuse the customers.

Hence, NEC would like to withdraw the original request to enable using
short hostnames, and will settle with the current knowledgebase article.
----

Thanks for comments, I got your situation. 
Let me confirm that you'd NOT like to change this behavior and want
KnowledgeBase Article only, right?

Thanks in advance.

Regards,

Masaki Furuta



Internal Status set to 'Waiting on Customer'
Status set to: Waiting on Client
Ticket type set to: 'Problem'

This event sent from IssueTracker by mfuruta 
 issue 315411

Comment 28 Dave Maley 2009-09-09 18:25:07 UTC
Neil - Since NEC has stated that they prefer to move forward w/ the kbase only, and to my knowledge we haven't received any additional reports, I'd suggest we:

1. update the kdump howto to state that fqdn is required
2. update the documentation in kdump.conf to state fqdn is required
3. update the kbase to remove the statement about the possibility of a fix

If you agree I'll go ahead and propose the changes to the howto and config files, and will get the kbase updated.  I'll also clear the 5.4.z request as there's no reason these changes can't wait for 5.5.

Comment 29 Dave Maley 2009-09-29 21:39:03 UTC
* removing 5.4.z request based on comment 27

Neil - lmk how you like to proceed wrt comment 28

Comment 30 Neil Horman 2009-10-21 19:56:33 UTC
yeah, that sounds like a fine plan.  Thanks Dave!

Comment 31 Neil Horman 2009-10-23 17:24:21 UTC
updated docs in the pkg in -84.el5.  Thanks!

Comment 32 Dave Maley 2009-10-26 19:45:39 UTC
Neil - I've updated the kbase so we should be all set here.  Thanks!

Comment 38 Ruediger Landmann 2010-03-19 05:00:41 UTC
Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.

New Contents:
A previous enhancement to kdump ensured that kdump would perform reliably
even when the DNS on a network changed. However, this required that host
names specified in the kdump.conf script needed to be entered as fully-
qualified domain names. Therefore, if a hostname was entered in a short form, 
kdump would fail to dump to a ssh/scp target. The documentation for kdump
has been revised to make it clear that hostnames must always be specified as
FQDNs. When users specify hostnames in this format, dumps to ssh/scp targets
perform as expected.

Comment 39 errata-xmlrpc 2010-03-30 07:47:13 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2010-0179.html