Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1228165 - [upgrade] Could not find Content Host with exact name <capsule_fqdn> (Katello::Errors::CapsuleContentMissingConsumer): On synchronising contents from upgraded sat6 to upgraded capsule
[upgrade] Could not find Content Host with exact name <capsule_fqdn> (Katello...
Status: CLOSED CURRENTRELEASE
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Upgrades (Show other bugs)
6.1.0
Unspecified Unspecified
unspecified Severity high (vote)
: Unspecified
: Unused
Assigned To: Stephen Benjamin
Sachin Ghai
http://projects.theforeman.org/issues...
: ReleaseNotes, Triaged
Depends On:
Blocks: sat61-release-notes 1195450
  Show dependency treegraph
 
Reported: 2015-06-04 06:36 EDT by Sachin Ghai
Modified: 2017-02-23 14:57 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-08-12 12:03:00 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
production.logs on syncing capsule contents (68.23 KB, text/plain)
2015-06-04 06:36 EDT, Sachin Ghai
no flags Details
logs from production.log (41.05 KB, text/plain)
2015-07-13 07:22 EDT, Sachin Ghai
no flags Details
CV promotion failed after sat6 upgrade (68.54 KB, image/png)
2015-07-13 08:49 EDT, Sachin Ghai
no flags Details
foreman-debug from rhel71 (619.54 KB, application/x-xz)
2015-07-13 08:57 EDT, Sachin Ghai
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Foreman Issue Tracker 11123 None None None 2016-04-22 12:56 EDT

  None (edit)
Description Sachin Ghai 2015-06-04 06:36:14 EDT
Created attachment 1034659 [details]
production.logs on syncing capsule contents

Description of problem:
I was trying upgrade from sat6.0.8 to sat6.1 snap7. I successfully upgraded capsule and satellite server.

But when I ran following command on satellite 6.1 server to re-sync contents on capsule, I got:

[root@sat6-qe-rhel66 ~]# hammer -u admin -p changeme capsule content synchronize --id=2
Could not synchronize capsule content:
  Could not find Content Host with exact name 'sat6-capsule-rhel66.usersys.redhat.com', verify the Capsule is registered with that name.


[root@sat6-qe-rhel66 ~]# hammer -u admin -p changeme capsule list
---|----------------------------------------|----------------------------------------------------
ID | NAME                                   | URL                                                
---|----------------------------------------|----------------------------------------------------
2  | sat6-capsule-rhel66.usersys.redhat.com | https://sat6-capsule-rhel66.usersys.redhat.com:9090
1  | sat6-qe-rhel66.usersys.redhat.com      | https://sat6-qe-rhel66.usersys.redhat.com:9090     
---|----------------------------------------|----------------------------------------------------


[root@sat6-qe-rhel66 ~]# hammer -u admin -p changeme ping
candlepin:      
    Status:          ok
    Server Response: Duration: 21ms
candlepin_auth: 
    Status:          ok
    Server Response: Duration: 28ms
pulp:           
    Status:          ok
    Server Response: Duration: 27ms
pulp_auth:      
    Status:          ok
    Server Response: Duration: 29ms
elasticsearch:  
    Status:          ok
    Server Response: Duration: 41ms
foreman_tasks:  
    Status:          ok
    Server Response: Duration: 0ms


[root@sat6-qe-rhel66 ~]#  hammer -u admin -p changeme organization list
---|----------------------|----------------------|------------
ID | NAME                 | LABEL                | DESCRIPTION
---|----------------------|----------------------|------------
1  | Default_Organization | Default_Organization |            
---|----------------------|----------------------|------------

Version-Release number of selected component (if applicable):
upgrade sat6.0.8 -> sat6.1 snap7

How reproducible:
always

Steps to Reproduce:
1. upgrade sat6.0.8 server to sat6.1 GA snap7
2. upgrade sat6.0.8 capsule to sat6.1 capsule
3. sync contents from sat6.1 server to capsule

Actual results:


Expected results:


Additional info:
I've updated the /etc/hosts file with ip and hostnames.. I can ping capsule node from sat6 via hostname and vice-versa.
Comment 2 Stephen Benjamin 2015-06-04 09:11:59 EDT
It looks like the migration didn't work.  There's this:
   db/migrate/20150423134004_add_content_host_id_to_smart_proxy.rb 

Which is supposed to add the content_host to the smart proxy, and that didn't happen for some reason on your Satellite.
Comment 3 Stephen Benjamin 2015-06-05 16:31:42 EDT
Post-upgrade I didn't encounter this issue, my Capsule had the Content Host associated.

However, deleting the Content Host associated with the Capsule, and later reregistering it will cause this problem, so its something we need to prevent from happening.
Comment 4 Stephen Benjamin 2015-06-10 10:42:55 EDT
I've now done a bunch of upgrades with SNAP 7 through various bug testing and never hit this, so I don't think it's an upgrade issue, or if it was, its been corrected.

The situation could still happen if you unregister the content host that the capsule is attached to.  You can fix it in `foreman-rake console` with this:

User.current = User.anonymous_admin
 
SmartProxy.all.each do |proxy|
  content_host = ::Katello::System.where(:name => proxy.name).order("created_at DESC").first
 
  if content_host
    proxy.content_host_id = content_host.id
    proxy.save!
    puts "Capsule #{proxy} has Content Host #{content_host}"
  else
    puts "No Content Host Found for #{proxy}"
  end
end 


Maybe that could be turned into a rake task.


The permanent solution isn't clear to me, there's a few things we could do. Maybe we should delete the SmartProxy object if a user deletes the relevant ContentHost for a Capsule.
Comment 5 Sachin Ghai 2015-07-13 07:20:46 EDT
This issue is still reproducible on rhel66 with sat6.1.1 snap12.

here is the summary:
========================


While trying upgrade from sat6.0.8 to sat6.1.1 snap12, found a weird issue where content-view promotion ( Library -> dev) failed. Please note that CV publish was successful. I've updated the CV with 6.1.1 capsule and sat-tools repos. So I was trying to generate new version of CV with these repos.

Here is the error I found in production.log:

---
2015-07-13 03:25:17 [E] Could not find Content Host with exact name 'cloud-qe-20.idmqe.lab.eng.bos.redhat.com', verify the Capsule is registered with that name. (Katello::Errors::CapsuleContentMissingConsumer)
--

here 'cloud-qe-20.idmqe.lab.eng.bos.redhat.com' is the node which was configured as external capsule. I skipped the promotion task from dynflow and resume it again.

Later,  I tried capsule-upgrade and it was completed successfully (with workaround[1]). But when I initiated capsule sync, it got failed with same error:

--
[root@cloud-qe-17 ~]# hammer -u admin -p changeme capsule content synchronize --id=2
Could not synchronize capsule content:
  Could not find Content Host with exact name 'cloud-qe-20.idmqe.lab.eng.bos.redhat.com', verify the Capsule is registered with that name.
--

I unregister and re-register the capsule node again and re-initiated sync. this time sync successfully completed.

----
[root@cloud-qe-17 ~]# hammer -u admin -p changeme capsule content synchronize --id=2
[............................................................................................................................................] [100%]
-----
Comment 6 Sachin Ghai 2015-07-13 07:22:54 EDT
Created attachment 1051372 [details]
logs from production.log
Comment 7 Sachin Ghai 2015-07-13 08:49:26 EDT
Created attachment 1051398 [details]
CV promotion failed after sat6 upgrade
Comment 8 Sachin Ghai 2015-07-13 08:54:54 EDT
This issue is reproducible on rhel71 too.

steps performed:
===================

- I installed sat6.0.8 with external capsule and provisioned a content-host via external capsule.

- upgraded sat6.0.8 with sat6.1.1.  (upgrade was completed successfully)
- created custom repos for sat6.1_tools and capsule repo.
- added repos to existing CV and published it
- but when I promoted the new version to next 'dev' env. promotion failed with error:

--
2015-07-13 08:48:02 [E] Could not find Content Host with exact name 'cloud-qe-6.idmqe.lab.eng.bos.redhat.com', verify the Capsule is registered with that name. (Katello::Errors::CapsuleContentMissingConsumer)
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.57/app/lib/katello/capsule_content.rb:48:in `consumer'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.57/app/lib/actions/katello/capsule_content/configure_capsule.rb:19:in `block in plan'
--
Comment 10 Sachin Ghai 2015-07-13 08:57:26 EDT
Created attachment 1051400 [details]
foreman-debug from rhel71
Comment 11 Stephen Benjamin 2015-07-13 09:35:16 EDT
Sachin will open a separate docs BZ for the problem, will leave this one as-is, deleting a content host breaks synchronization.
Comment 14 Sachin Ghai 2015-07-14 08:02:20 EDT
Earlier I was trying satellite6.1 content synchronization (capsule repo and sat-tools repo) after upgrade. Which breaks the content promotion.

As suggested, this time I synced all repos(capsule and sat-tools) before upgrade. Upgrade on Satellite server and external capsule completed successfully.

But when I re-tried the capsule sync after upgrade, I got this:

[root@cloud-qe-12 ~]# hammer -u admin -p changeme capsule content synchronize --id=2
Could not synchronize capsule content:
  Could not find Content Host with exact name 'cloud-qe-15.idmqe.lab.eng.bos.redhat.com', verify the Capsule is registered with that name.

[root@cloud-qe-12 ~]#  hammer -u admin -p changeme capsule list
---|------------------------------------------|------------------------------------------------------
ID | NAME                                     | URL                                                  
---|------------------------------------------|------------------------------------------------------
1  | cloud-qe-12.idmqe.lab.eng.bos.redhat.com | https://cloud-qe-12.idmqe.lab.eng.bos.redhat.com:9090
2  | cloud-qe-15.idmqe.lab.eng.bos.redhat.com | https://cloud-qe-15.idmqe.lab.eng.bos.redhat.com:9090
---|------------------------------------------|------------------------------------------------------
[root@cloud-qe-12 ~]# 

Does this mean we need to re-register the capsule after upgrade ?
Comment 16 Stephen Benjamin 2015-07-15 10:41:13 EDT
Created redmine issue http://projects.theforeman.org/issues/11123 from this bug
Comment 17 Bryan Kearney 2015-07-15 14:04:30 EDT
Moving to POST since upstream bug http://projects.theforeman.org/issues/11123 has been closed
-------------
Anonymous
Applied in changeset commit:katello|0bc31483f1110205dfa38ce1b0e16ce88dfd0839.
Comment 18 Bryan Kearney 2015-07-20 11:49:25 EDT
commit c0ca95be6ce42a9569010c4d0bc095afb732d89a
Author: Stephen Benjamin <stbenjam@redhat.com>
Date:   Wed Jul 15 10:41:14 2015 -0400

    fixes #11123 - actually assign content_host to capsule
Comment 20 Sachin Ghai 2015-07-23 05:45:57 EDT
Verified with sat6.1.1 GA snap14.

Now I can successfully sync contents to capsule after upgrade.

steps performed:

Upgraded sat server and capsule from  sat6.0.8 -> sat6.1.1 GA snap14
re-run the capsule-sync and it was completed without any error.

[root@cloud-qe-19 ~]# hammer -u admin -p changeme capsule content synchronize --id=2
[............................................................................................................................................] [100%]
Comment 21 Stephen Benjamin 2015-07-27 09:44:03 EDT
A workaround for anyone encountering this until GA:

echo 'User.current = User.anonymous_admin; SmartProxy.all.each { |proxy| proxy.content_host.blank? ? proxy.update_attribute(:content_host, Katello::System.find_by_name(proxy.name)) : nil }' | foreman-rake console
Comment 22 Bryan Kearney 2015-08-12 12:03:00 EDT
This bug was fixed in Satellite 6.1.1 which was delivered on 12 August, 2015.

Note You need to log in before you can comment on or make changes to this bug.