Bug 1659324 - While executing insights remediation playbooks via satellite it does not honour HTTP Proxy configured
Summary: While executing insights remediation playbooks via satellite it does not hono...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Ansible
Version: 6.4.0
Hardware: All
OS: Linux
high
medium vote
Target Milestone: Released
Assignee: satellite6-bugs
QA Contact: Lukas Pramuk
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-14 05:11 UTC by Ashish Humbe
Modified: 2019-10-07 17:19 UTC (History)
11 users (show)

Fixed In Version: tfm-rubygem-foreman_ansible-2.2.12
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1712375 (view as bug list)
Environment:
Last Closed: 2019-05-14 12:39:36 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2019:1222 None None None 2019-05-14 12:39:46 UTC
Github redhataccess foreman-plugin issues 29 None None None 2019-02-07 23:13:36 UTC
Foreman Issue Tracker 26272 None None None 2019-03-07 15:37:35 UTC

Description Ashish Humbe 2018-12-14 05:11:27 UTC
Description of problem:

When we try to apply the insights remediation playbook for the client system via satellite webui, it fails to connect to cert-api.access.redhat.com


2018-12-06T07:45:26 [I|app|] Started POST "/api/v2/job_invocations/" for 10.15.14.122 at 2018-12-06 07:45:26 +0300
2018-12-06T07:45:26 [I|app|d9aa8] Processing by Api::V2::JobInvocationsController#create as JSON
2018-12-06T07:45:26 [I|app|d9aa8]   Parameters: {"job_invocation"=>{"feature"=>"ansible_run_insights_plan", "host_ids"=>"plan_id=39649", "inputs"=>{"organization_id"=>1, "plan_id"=>39649}}, "apiv"=>"v2"}
2018-12-06T07:45:26 [I|app|d9aa8] Current user: USER1 (administrator)
2018-12-06T07:45:27 [I|aud|d9aa8] create event for JobInvocation with id 130
2018-12-06T07:45:27 [I|app|d9aa8] Current user: USER1 (administrator)
2018-12-06T07:45:27 [I|app|d9aa8] Proxying request to cert-api.access.redhat.com via 
2018-12-06T07:45:27 [I|app|d9aa8] Current user: USER1 (administrator)
2018-12-06T07:45:27 [E|bac|d9aa8] external method 'search_by_plan_id' failed with error: Failed to open TCP connection to cert-api.access.redhat.com:443 (getaddrinfo: Name or service not known) (ScopedSearch::QueryNotSupported)
/opt/theforeman/tfm/root/usr/share/gems/gems/scoped_search-4.1.3/lib/scoped_search/query_builder.rb:408:in `rescue in to_ext_method_sql'
/opt/theforeman/tfm/root/usr/share/gems/gems/scoped_search-4.1.3/lib/scoped_search/query_builder.rb:405:in `to_ext_method_sql'
/opt/theforeman/tfm/root/usr/share/gems/gems/scoped_search-4.1.3/lib/scoped_search/query_builder.rb:213:in `sql_test'
/opt/theforeman/tfm/root/usr/share/gems/gems/scoped_search-4.1.3/lib/scoped_search/query_builder.rb:550:in `sql_test'
/opt/theforeman/tfm/root/usr/share/gems/gems/scoped_search-4.1.3/lib/scoped_search/query_builder.rb:497:in `to_single_field_sql'
/opt/theforeman/tfm/root/usr/share/gems/gems/scoped_search-4.1.3/lib/scoped_search/query_builder.rb:509:in `to_sql'
/opt/theforeman/tfm/root/usr/share/gems/gems/scoped_search-4.1.3/lib/scoped_search/query_builder.rb:54:in `build_find_params'
/opt/theforeman/tfm/root/usr/share/gems/gems/scoped_search-4.1.3/lib/scoped_search/query_builder.rb:22:in `build_query'
/opt/theforeman/tfm/root/usr/share/gems/gems/scoped_search-4.1.3/lib/scoped_search/definition.rb:300:in `block in register_named_scope!'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activerecord-5.1.6/lib/active_record/relation/delegation.rb:66:in `block in search_for'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activerecord-5.1.6/lib/active_record/relation.rb:336:in `scoping'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activerecord-5.1.6/lib/active_record/relation/delegation.rb:66:in `search_for'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.5.6/app/models/targeting.rb:43:in `block in resolve_hosts!'
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:94:in `as'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.5.6/app/models/targeting.rb:43:in `resolve_hosts!'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.5.6/app/lib/actions/remote_execution/run_hosts_job.rb:30:in `plan'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.0.5.1/lib/dynflow/action.rb:493:in `block (3 levels) in execute_plan'


Here we can notice that the connection to cert-api.access.redhat.com is not going via HTTP Proxy server.

2018-12-06T07:45:26 [I|app|d9aa8] Current user: NTG022 (administrator)
2018-12-06T07:45:27 [I|aud|d9aa8] create event for JobInvocation with id 130
2018-12-06T07:45:27 [I|app|d9aa8] Current user: NTG022 (administrator)
2018-12-06T07:45:27 [I|app|d9aa8] Proxying request to cert-api.access.redhat.com via        <<<<<<<<<
2018-12-06T07:45:27 [I|app|d9aa8] Current user: NTG022 (administrator)
2018-12-06T07:45:27 [E|bac|d9aa8] external method 'search_by_plan_id' failed with error: Failed to open TCP connection to cert-api.access.redhat.com:443 (getaddrinfo: Name or service not known) (ScopedSearch::QueryNotSupported)


As per code in /usr/share/foreman/lib/foreman/http_proxy.rb there should be proxy server details at the end. 

 def log_proxied_request(current_proxy, requested_host)
      foreman_logger.info "Proxying request to #{requested_host} via #{current_proxy}"      <<<<<<<<<
    end



When we access Insights -> Manage page to check the connectivity the connection goes via proxy and the status is shown as connected. This issue is seen only while executing the ansible based playbooks.


Version-Release number of selected component (if applicable):
Red Hat Satellite 6.4  + BlueCoat proxy. 

How reproducible:
Always at the customer end

Steps to Reproduce:
1. Configure Satellite 6 to connect to the internet via BlueCoat proxy 
2. Try to execute ansible playbooks generated as part of Insights remediation script 
3.

Actual results:

Playbook execution fails to connect to cert-api.access.redhat.com via proxy configured on the satellite server.

Expected results:

Since all other actions work via HTTP Proxy, the playbook should also get executed successfully. 

Additional info:

Comment 4 Sebastian Gräßl 2019-01-11 11:58:45 UTC
Ashish, can you elaborate how the proxy is configured for Satillite? 
Is the "http_proxy" setting under "General" properly configured with the BlueCoat proxy?

Comment 5 Ashish Humbe 2019-01-16 14:41:14 UTC
(In reply to Sebastian Gräßl from comment #4)
> Ashish, can you elaborate how the proxy is configured for Satillite? 
> Is the "http_proxy" setting under "General" properly configured with the
> BlueCoat proxy?

Hi Sebastian, 

Proxy was configured on the satellite server using:

# satellite-installer --scenario satellite --katello-proxy-url http://XX.XX.XX.XX  --katello-proxy-port 3128   --katello-proxy-username 'username'   --katello-proxy-password  'password'

We confirmed proxy details in :  /etc/pulp/server/plugins.conf.d/yum_importer.json 

We have not confirmed the proxy details in satellite WebUI Settings -> "General" -> http_proxy 

One strange thing we had noticed was that : at the customer end, I can see log entries with "Proxying request to cert-api.access.redhat.com via       " but on local satellite with proxy configured, we do not see such logs. Any thoughts on this?

Comment 6 Sebastian Gräßl 2019-01-21 11:16:07 UTC
Hej Ashish,

the above command does only configure proxy for components used by katello, but not for requests of the foreman application itself. 
To configure the a HTTP proxy for all requests either append "--http-proxy"-flag and ensure that the correct value is set under Settings -> General in the WebUI.

Comment 7 Ashish Humbe 2019-01-22 13:42:56 UTC
(In reply to Sebastian Gräßl from comment #6)
> Hej Ashish,
> 
> the above command does only configure proxy for components used by katello,
> but not for requests of the foreman application itself. 
> To configure the a HTTP proxy for all requests either append
> "--http-proxy"-flag and ensure that the correct value is set under Settings
> -> General in the WebUI.


Okay, sure. We will ask the customer to try it.

For the authenticated proxy servers do we need to set the URL as:    http://user:password@proxyserver:proxyport   ?  or need to use any different syntax?

Comment 10 Marek Hulan 2019-02-26 07:07:43 UTC
Hi Ashish, did you hear back from customer about using generic http proxy setting? I think the format you suggested would work.  So my understanding is, there are two settings that need to be set, cdn_proxy and generic http_proxy. If that's the case and it works, what's left for resolving the issue? Thank you.

Comment 11 Ashish Humbe 2019-02-26 11:05:34 UTC
Hi Marek,

We do not have confirmation from the customer yet because the proxy username they are using also include " @ " in it, so still working with the customer to get simple username/password or get an unauthenticated connection for testing purpose. 

It seems that when the username also has @ in it, the satellite might not be able to read the proxy URL correctly. 

Thanks!

Comment 16 Lukas Pramuk 2019-04-29 20:53:47 UTC
VERIFIED.

@satellite-6.5.0-11.el7sat.noarch
tfm-rubygem-foreman_ansible-2.2.14-2.el7sat.noarch

by the following manual reproducer:

1) Have a host registered to Satellite and set up for remote execution
@HOST: # curl -k https://$SAT:9090/ssh/pubkey >> /root/.ssh/authorized_keys

2) Assign "RedHatInsights.insights-client" ansible role to the host and click "Run Ansible roles" for the host

3) Setup Satellite so that Katello plugin uses HTTP proxy
# satellite-installer --katello-proxy-url http://proxy.example.com --katello-proxy-port 3128

4) Pretend the host suffers from security vulnerability where its remediation rule has ansible support
@HOST: # sed -i 's/while read -r opt/while read opt/' /etc/NetworkManager/dispatcher.d/11-dhclient

5) Create new remediation plan for specific system (the host) with the rule "NetworkManager DHCP script vulnerable to remote code execution (CVE-2018-1111)"

6) Start watching active connections being made to the http proxy
# watch "netstat -pnt | grep :3128"

7) Click "Run Playbook" and check new connections being created to the http proxy

>>> remediation playbook is fetched thru (katello) http proxy

Comment 19 errata-xmlrpc 2019-05-14 12:39:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:1222


Note You need to log in before you can comment on or make changes to this bug.