Bug 1873241

Summary: [RFE] When choosing what capsule to use for Remote Execution into a host, use the host's "Registered through" capsule
Product: Red Hat Satellite Reporter: Pablo Hess <phess>
Component: Remote ExecutionAssignee: Adam Ruzicka <aruzicka>
Status: CLOSED ERRATA QA Contact: Peter Ondrejka <pondrejk>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.8.0CC: ahumbe, aruzicka, bkearney, lstejska, mkalyat, msunil, pcreech, pdwyer, sraut, svaezi, swachira, zhunting
Target Milestone: 6.11.0Keywords: FutureFeature, Triaged
Target Release: Unused   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-07-05 14:28:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Pablo Hess 2020-08-27 17:08:38 UTC
Description of problem:

Consider a complex network scenario with multiple capsules but no subnets created on Satellite, and everything under a single Org and Location.
When running a Remote Execution job against a host, all existing criteria to choose what capsule to originate the REx job from will be fruitless.

I request adding the capsule the host was registered through (i.e. the "Registered Through" capsule) as one more criterion in this choice. This way, in the complex scenario described above a particular host's "Registered Through" capsule would be the first choice as the originator for REx jobs.

Comment 2 Adam Ruzicka 2020-08-31 10:23:43 UTC
> When running a Remote Execution job against a host, all existing criteria to choose what capsule to originate the REx job from will be fruitless.

Not all criteria depend on subnets and/or organizations and locations so I wouldn't say it will be completely fruitless. If they enable fallback to any capsule as per[1], then satellite should be able to select the capsule based on capsules directly assigned to the host, such as various network service related capsules (dhcp, dns...) or content capsules. Would this suffice as a workaround or is either environment so complex that hosts have capsules from another subnet assigned?

[1] - https://access.redhat.com/documentation/en-us/red_hat_satellite/6.7/html/managing_hosts/running_jobs_on_hosts#sect-Managing_Hosts-Choosing_a_Satellite_Capsule_for_Remote_Execution

Comment 3 Pablo Hess 2020-09-04 14:08:08 UTC
(In reply to Adam Ruzicka from comment #2)
> > When running a Remote Execution job against a host, all existing criteria to choose what capsule to originate the REx job from will be fruitless.
> 
> Not all criteria depend on subnets and/or organizations and locations so I
> wouldn't say it will be completely fruitless. If they enable fallback to any
> capsule as per[1], then satellite should be able to select the capsule based
> on capsules directly assigned to the host, such as various network service
> related capsules (dhcp, dns...) or content capsules. Would this suffice as a
> workaround or is either environment so complex that hosts have capsules from
> another subnet assigned?

Capsules in this scenario are not offering DNS or DHCP services. And hosts' content sources are load balancers instead of individual capsules. Because of this, I would say the RFE @ #1873625 is a better candidate to address this problem completely given this is a load-balanced capsule scenario.

Also, because of load balancers I now see that this RFE here would conflict with #1821457 which specifically states that load balancer scenarios should not have "registered through" updated after provisioning lest REx jobs get sent to load balancers instead of actual capsules. Is there a conflict?

I'm now clearing needinfo.

Comment 4 Sasan Vaezi 2021-05-11 14:09:40 UTC
Is there any update on the plan for this fix?

Thanks

Sasan

Comment 5 Adam Ruzicka 2021-07-30 10:39:37 UTC
*** Bug 1988162 has been marked as a duplicate of this bug. ***

Comment 8 Adam Ruzicka 2021-09-07 08:11:00 UTC
Created redmine issue https://projects.theforeman.org/issues/33425 from this bug

Comment 11 Bryan Kearney 2022-01-19 20:02:50 UTC
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/33425 has been resolved.

Comment 14 Peter Ondrejka 2022-02-10 14:14:32 UTC
Confirmed on Sat 7.0 snap 8 that with the "Prefer registered through Capsule for remote execution" setting enabled, the rex job goes through the registered_through capsule. Though with my setup I'm getting the following output for the rex job:


   1:Initialization error: RestClient::NotFound - 404 Not Found
   2:Initialization error: RestClient::NotFound - 404 Not Found
   3:Initialization error: RestClient::NotFound - 404 Not Found
   4:Initialization error: RestClient::NotFound - 404 Not FoundError loading data from Capsule: NoMethodError - undefined method `code' for "404 Not Found":String
   5:Did you mean?  encode

Not sure if it is setup-specific, the capsule is installed on FIPS-enabled rhel7 (to check also bz#1906023)

Comment 15 Peter Ondrejka 2022-02-11 09:53:12 UTC
Ok, with disabled ssh debug logging on the capsule the execution proceeds as expected, problem in c14 seems to be bound to bz#1906023, verifying this one

Comment 16 Pablo Hess 2022-02-25 22:10:56 UTC
I'm currently looking to implement a workaround that should work on Satellite 6.8 to 6.10 by tweaking the "Run Command - Ansible Default" job templates to take into account the target host's registered_through content facet attribute. 

The modified job template would add this to the playbook:
~~~
- hosts: .....
  vars:
    ansible_ssh_extra_args: "-o ProxyCommand='ssh <%= bastion_user %>@<%= registered_through(host) %> -W %h:%p'"
  ....
~~~

Of course this requires setting up a working SSH bastion host scheme between Satellite and the capsules, but I think this has potential to resolve the immediate need to always run Ansible commands through the registered_through capsule ONLY.

I'd appreciate thoughts on this if anyone has any to share.

Comment 19 errata-xmlrpc 2022-07-05 14:28:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.11 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5498