Bug 1356714 - After upgrade, all of my hosts are duplicated one with shortname, one with fqdn
After upgrade, all of my hosts are duplicated one with shortname, one with fqdn
Status: CLOSED ERRATA
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Hosts (Show other bugs)
6.2.0
x86_64 Linux
high Severity high (vote)
: 6.2.7
: --
Assigned To: satellite6-bugs
jcallaha
: PrioBumpGSS, PrioBumpPM, PrioBumpQA, Reopened, Triaged
: 1362592 (view as bug list)
Depends On:
Blocks: CEE_Sat6_Top_BZs/GSS_Sat6_Top_Bugs 1399395 1380473 1394349 1405538
  Show dependency treegraph
 
Reported: 2016-07-14 16:20 EDT by rakesh kumar
Modified: 2018-05-15 00:58 EDT (History)
28 users (show)

See Also:
Fixed In Version: rubygem-katello-3.0.0.80 foreman-1.11.0.54-1,katello-installer-base-3.0.0.69-1
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1380473 1405538 (view as bug list)
Environment:
Last Closed: 2017-01-26 10:55:05 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
unify_hosts.rake (2.32 KB, text/plain)
2016-08-23 08:58 EDT, Justin Sherrill
no flags Details
hosts page (44.00 KB, image/png)
2016-09-29 12:13 EDT, jcallaha
no flags Details
duplicate (51.34 KB, image/png)
2016-09-29 14:10 EDT, jcallaha
no flags Details
combined (48.30 KB, image/png)
2016-09-29 14:12 EDT, jcallaha
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Foreman Issue Tracker 14516 None None None 2016-08-24 13:46 EDT
Foreman Issue Tracker 16270 None None None 2016-08-24 13:46 EDT
Red Hat Knowledge Base (Solution) 2482641 None None None 2016-08-09 10:01 EDT

  None (edit)
Description rakesh kumar 2016-07-14 16:20:56 EDT
Description of problem:
Puppet creates Duplicate  profile in "All hosts" with FQDN if client is configured and registered with short name.

Version-Release number of selected component (if applicable):
Satellite 6.2 beta 2

How reproducible:
Always

Steps to Reproduce:
1.Change the client hostname to shortname instead of FQDN( # hostname and #hostname -f o/p should be different)
2.Register the same to Satellite
3.Then, installed the puppet agent and make necessary changes to /etc/puppet/puppet.conf 
4.Create the puppet client certificate by executing "puppet agent -t" and signed it in Satellite.
5. Again, execute below command on the client:
---
# puppet agent -t

Actual results: After performing above steps there were two client profiles in "All hosts" one is with short name and one is with FQDN.


Expected results: Puppet should not create new profile with FQDN.


Additional info:

Workaround: We can mention parameter "certname=<client shortname>" in /etc/puppet/puppet.conf than puppet will not create profile FQDN
Comment 3 Ivan Necas 2016-08-04 06:36:05 EDT
@jsherrill seems like related to https://bugzilla.redhat.com/show_bug.cgi?id=1362592, should we tread this as one BZ and close a dupe?
Comment 4 Justin Sherrill 2016-08-04 08:40:25 EDT
Ivan, 

Yeah I agree these are duplicates.  Closing https://bugzilla.redhat.com/show_bug.cgi?id=1362592

-Justin
Comment 5 Justin Sherrill 2016-08-04 08:40:47 EDT
*** Bug 1362592 has been marked as a duplicate of this bug. ***
Comment 10 Justin Sherrill 2016-08-15 11:54:51 EDT
Opened subscription-manager bugs for long term support:

https://bugzilla.redhat.com/show_bug.cgi?id=1367128
https://bugzilla.redhat.com/show_bug.cgi?id=1367126
Comment 11 Justin Sherrill 2016-08-19 16:10:11 EDT
Temporary workaround for new registrations:

run the following on a client prior to registration:


if [ -d /etc/rhsm/facts/ ]; then
  echo "{\"network.hostname\":\"`hostname -f`\"}" > /etc/rhsm/facts/katello.facts
fi


We are in the process of coming up with a more permanent work around as well as a script to 'unify' systems that are duplicated after upgrade
Comment 14 Justin Sherrill 2016-08-23 08:58 EDT
Created attachment 1193310 [details]
unify_hosts.rake
Comment 16 Mike McCune 2016-08-24 09:55:57 EDT
Attaching our script for unifying the hosts duplicated after upgrade:

Place in 'unify_hosts.rake' in /usr/share/foreman/lib/tasks/ and run:

# foreman-rake unify_hosts

NOTE: IT IS HIGHLY RECOMMENDED THE CUSTOMER HAVE A FULL DB BACKUP

With the testing we have done on 1 customer's db, it seemed to have unified 98% of the duplicate hosts.  The only times it did not was when it seemed the fqdn host had been re-registered (likely the customer doing it manually).  Those associated shortname hosts will need to be cleaned up manually.
Comment 17 Bryan Kearney 2016-08-24 14:17:32 EDT
Upstream bug assigned to jsherril@redhat.com
Comment 18 Bryan Kearney 2016-08-24 14:17:36 EDT
Upstream bug component is Infrastructure
Comment 19 Bryan Kearney 2016-08-24 14:17:41 EDT
Upstream bug assigned to jsherril@redhat.com
Comment 21 Bryan Kearney 2016-09-06 14:17:29 EDT
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/16270 has been resolved.
Comment 22 Ohad Levy 2016-09-15 08:45:18 EDT
*** Bug 1375706 has been marked as a duplicate of this bug. ***
Comment 23 Christian Pearce 2016-09-21 08:55:01 EDT
Is this target milestone correct?  I thought 6.2.2 is out.
Comment 27 Mike McCune 2016-09-28 16:31:50 EDT
Christian:

We are shipping an errata in between 6.2.2 and 6.2.3 to include this and a few other fixes that are landing outside our normally scheduled 6.2.Z 30-day release cycle.
Comment 30 jcallaha 2016-09-29 12:13 EDT
Created attachment 1205982 [details]
hosts page
Comment 31 Justin Sherrill 2016-09-29 13:17:31 EDT
The purpose of this BZ is to only handle the situation where a user has found themselves in after upgrading and having all hosts duplicated.  As part of this we are shipping a rake task that can be run (basically the same task as attached to this bug).   

I have cloned this bz to a new bug to handle new registrations (https://bugzilla.redhat.com/show_bug.cgi?id=1380473), and it is also targeting a 6.2.z release.  

In the meantime as has been stated, you can simply run this:


if [ -d /etc/rhsm/facts/ ]; then
  echo "{\"network.hostname\":\"`hostname -f`\"}" > /etc/rhsm/facts/katello.facts
fi

prior to registration to resolve the issue for new registrations.  This can be added to a bootstrap script, kickstart file, or puppet module.  Please do let us know if this workaround is not sufficient until 1380473 can be released.
Comment 32 jcallaha 2016-09-29 14:10:11 EDT
Verified in Satellite 6.2.2 Snap 1.

After duplicating the hosts as outlined in the setup steps (see attached image). I ran the unify_hosts job and the two entries were successfully combined (see unified image).

[root@rhsm-qe-1 ~]# foreman-rake  katello:unify_hosts
Skipping virt-who-supermicro-2027gr-tr2-1.gsslab.rdu2.redhat.com-1 due to blank primary interface mac
Skipping virt-who-inorbit.hq.gsslab.rdu.redhat.com-1 due to blank primary interface mac
Skipping virt-who-bulgaria.hq.gsslab.rdu.redhat.com-1 due to blank primary interface mac
Skipping virt-who-nightwing.hq.gsslab.rdu.redhat.com-1 due to blank primary interface mac
Unifying mgmt4.rhq.lab.eng.bos.redhat.com with mgmt4
Comment 33 jcallaha 2016-09-29 14:10 EDT
Created attachment 1205991 [details]
duplicate
Comment 34 jcallaha 2016-09-29 14:12 EDT
Created attachment 1205992 [details]
combined
Comment 37 errata-xmlrpc 2016-10-04 02:43:57 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1993
Comment 38 sthirugn@redhat.com 2016-11-17 18:16:54 EST
Negative.  This issue is still happening in Satellite 6.2.4.

Case #01713467
Comment 41 Bryan Kearney 2016-11-17 20:08:01 EST
Moving this bug to POST for triage into Satellite 6 since the upstream issue http://projects.theforeman.org/issues/16270 has been resolved.
Comment 42 Julio Entrena Perez 2016-11-18 06:59:00 EST
(In reply to sthirugn@redhat.com from comment #38)
> Negative.  This issue is still happening in Satellite 6.2.4.

This BZ provides the unify_hosts.rake script that removes already existing duplicates.
The script is provided by this errata and works, therefore CLOSED ERRATA is imho correct.

The prevention of new duplicates is not covered by this bug.
Comment 43 sthirugn@redhat.com 2016-11-18 07:55:37 EST
(In reply to Julio Entrena Perez from comment #42)
> (In reply to sthirugn@redhat.com from comment #38)
> > Negative.  This issue is still happening in Satellite 6.2.4.
> 
> This BZ provides the unify_hosts.rake script that removes already existing
> duplicates.
> The script is provided by this errata and works, therefore CLOSED ERRATA is
> imho correct.
> 
> The prevention of new duplicates is not covered by this bug.

I agree that this bug creates a workaround.  But I am looking for a fix to prevent this in the first place.

I am ok if this issue needs to be tracked in this bug or a new bug - regardless a fix is needed soon.
Comment 44 Justin Sherrill 2016-11-18 10:17:34 EST
We could pretty easily add this as a post step in the upgrade process.  I think that is safer than trying to modify the existing migrations as that migration is extremely fragile and has been working really well.  This script has been tested really well so far.  So my vote would be to just run this as a post script in the installer upgrade.
Comment 46 Justin Sherrill 2016-11-30 09:10:15 EST
Technically this already went into out in 6.2.2.  What suresh is suggesting is to run the script automatically upon upgrade.  I'll open a PR for that now
Comment 47 Justin Sherrill 2016-11-30 09:32:51 EST
PR Opened here: https://gitlab.sat.lab.tlv.redhat.com/satellite6/katello-installer/merge_requests/76

There is some concern because this will get run for all current users.  Given that the default behavior is to use mac addresses for the unification, its unlikely to cause a problem but it is a concern.
Comment 48 Julio Entrena Perez 2016-11-30 11:19:28 EST
IHAC reporting that running "foreman-rake katello:unify_hosts" results in all host entries being unified (expected) but then these hosts don't show any subscriptions or installed products. 
Running "subscription-manager refresh" in the hosts results in "Couldn't find consumer":

[root@hostname ~]# subscription-manager refresh
Couldn't find consumer 'bd9a911d-6fd2-4ce5-a9d1-f36652a54a8d'

They have workarounded that by explicitly creating file /etc/rhsm/facts/katello.facts with the correct network.hostname entry which prevents the duplicates.

I assume that running the script on a satellite with no duplicates would be a no op?
Comment 49 jcallaha 2017-01-16 15:27:53 EST
Verified in Satellite 6.2.7 Snap 1. The unify hosts task is now included at the end of the upgrade.

Upgrade Step: unify_hosts (this may take a while) ...
Skipping virt-who-supermicro-2027gr-tr2-1.gsslab.rdu2.redhat.com-1 due to blank primary interface mac
Skipping virt-who-bulgaria.hq.gsslab.rdu.redhat.com-1 due to blank primary interface mac
Skipping virt-who-inorbit.hq.gsslab.rdu.redhat.com-1 due to blank primary interface mac
Skipping virt-who-ironforge.satellite.lab.eng.rdu2.redhat.com-1 due to blank primary interface mac
Skipping virt-who-nightwing.hq.gsslab.rdu.redhat.com-1 due to blank primary interface mac
Skipping virt-who-dhcp-8-101-249.satellite.lab.eng.rdu2.redhat.com-1 due to blank primary interface mac
Unifying test with test.short.hostname.com

Upgrade completed!
Comment 50 Bryan Kearney 2017-01-26 10:55:05 EST
This was delivered in satellite 6.2.7 (https://access.redhat.com/errata/RHBA-2017:0197)
Comment 51 Luke Rawlins 2017-05-04 21:00:09 EDT
foreman-rake katello:unify_hosts will attempt to shutdown and delete virtual machines if they are not dissociated from the satellite server prior to running the script. If you have a compute node attached this can be dangerous. I've been using --name with the fqdn for new host registrations to avoid duplicate enteries.

Note You need to log in before you can comment on or make changes to this bug.