Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1582197

Summary: Net::SSH::AuthenticationFailed error when using option remote_execution_without_proxy
Product: Red Hat Satellite Reporter: Stefan Meyer <smeyer>
Component: Remote ExecutionAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED WONTFIX QA Contact: Peter Ondrejka <pondrejk>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.3.1CC: aruzicka, bvassova, inecas, ktordeur, smeyer
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-01-15 20:30:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Stefan Meyer 2018-05-24 13:45:39 UTC
Description of problem:
The customer uses Vmware templates to deploy new hosts and the subnet option for new hosts are not set when they register to Satellite 6.3. 
The customer then enabled the option remote_execution_without_proxy in Administer => Settings => RemoteExecution

When trying to use Remote Execution he gets the error messages:
=>    Error initializing command: Net::SSH::AuthenticationFailed - Authentication failed for user root.com

The ssh keys were copied to /usr/share/foreman/.ssh and /usr/share/foreman-proxy/.ssh a manual test to connect to the 
client with the foreman user works fine.

On the client we see failed connect attempts with this message:

    client sshd[10763]: Connection closed by 192.168.1.199 port 57590 [preauth]

The 192.168.1.199 ip adress is the Satellite server.

This may be a documentation bug and it is just not clearly stated where and how to configure the ssh keys.


Version-Release number of selected component (if applicable):
- RHEL 7.5 on both Satellite and Client
- Satellite 6.3.1


How reproducible:
Everytime

Steps to Reproduce:
1. Enable remote_execution_without_proxy in Administer => Settings => RemoteExecution
2. Remove any subnet configuration from the client or remove the RemoteExecution option from the capsule that is managing the subnet
3. Run a job on the client

Actual results:
Erro message and failed job

Expected results:
It should be possible to run jobs without having a configured capsule for RemoteExecution

Additional info:

Task error is:
##################################################################
{"result"=>
  [{"output_type"=>"debug",
    "output"=>
     "Error initializing command: Net::SSH::AuthenticationFailed - Authentication failed for user root.com",
    "timestamp"=>1527167674.5693605}],
 "runner_id"=>"81b0a15f-72aa-4429-b595-4d378c4f9dff",
 "exit_status"=>"EXCEPTION"}
##################################################################


Backtrace:
##################################################################
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:74:in `block in assigns'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in `tap'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in `assigns'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:56:in `match_value'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:36:in `block in match?'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in `match?'
/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:23:in `match'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/execution_plan/steps/error.rb:12:in `new'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:460:in `set_error'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:423:in `error!'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-core-0.1.8/lib/foreman_tasks_core/runner/action.rb:27:in `finalize'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:540:in `block (2 levels) in execute_finalize'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action/progress.rb:30:in `with_progress_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action/progress.rb:22:in `finalize'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/middleware/keep_locale.rb:15:in `block in finalize'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/middleware/keep_locale.rb:22:in `with_locale'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.4.5.64.2/app/lib/actions/middleware/keep_locale.rb:15:in `finalize'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:39:in `finalize'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/world.rb:30:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:539:in `block in execute_finalize'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:431:in `block in with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:431:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:431:in `with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:538:in `execute_finalize'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/action.rb:268:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in `block (2 levels) in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/execution_plan/steps/abstract.rb:155:in `with_meta_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in `block in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in `open_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/director/sequential_manager.rb:68:in `run_step'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/director/sequential_manager.rb:53:in `dispatch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/director/sequential_manager.rb:60:in `block in run_in_sequence'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/director/sequential_manager.rb:60:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/director/sequential_manager.rb:60:in `all?'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/director/sequential_manager.rb:60:in `run_in_sequence'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/director/sequential_manager.rb:49:in `dispatch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/director/sequential_manager.rb:27:in `block in finalize'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:47:in `finalize_phase'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:47:in `finalize_phase'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/common/transaction.rb:16:in `block in rollback_on_error'
/opt/rh/rh-ror42/root/usr/share/gems/gems/activerecord-4.2.6/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `block in transaction'
/opt/rh/rh-ror42/root/usr/share/gems/gems/activerecord-4.2.6/lib/active_record/connection_adapters/abstract/transaction.rb:184:in `within_new_transaction'
/opt/rh/rh-ror42/root/usr/share/gems/gems/activerecord-4.2.6/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `transaction'
/opt/rh/rh-ror42/root/usr/share/gems/gems/activerecord-4.2.6/lib/active_record/transactions.rb:220:in `transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/transaction_adapters/active_record.rb:5:in `transaction'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/common/transaction.rb:15:in `rollback_on_error'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/common/transaction.rb:9:in `finalize_phase'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/middleware/world.rb:30:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/director/sequential_manager.rb:26:in `finalize'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/director.rb:66:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/executors/parallel/worker.rb:11:in `on_message'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/context.rb:46:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.34/lib/dynflow/actor.rb:26:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/awaits.rb:15:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/buffer.rb:38:in `process_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/buffer.rb:31:in `process_envelopes?'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/buffer.rb:20:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/termination.rb:55:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/removes_child.rb:10:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/core.rb:161:in `process_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/core.rb:95:in `block in on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/core.rb:118:in `block (2 levels) in schedule_execution'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `block in synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.3/lib/concurrent/actor/core.rb:115:in `block in schedule_execution'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/serialized_execution.rb:18:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/serialized_execution.rb:96:in `work'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/serialized_execution.rb:77:in `block in call_job'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `run_task'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in `block (3 levels) in create_worker'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `loop'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `block (2 levels) in create_worker'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-1.0.3/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `block in create_worker'
/opt/theforeman/tfm/root/usr/share/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'
##################################################################

Comment 1 Adam Ruzicka 2018-05-24 13:59:58 UTC
>> The ssh keys were copied to /usr/share/foreman/.ssh and /usr/share/foreman-proxy/.ssh a manual test to connect to the 
client with the foreman user works fine.

So if the rex_without_proxy option is set the keys have to be put into ~foreman-proxy/.ssh/

if the setting is turned on, they need to be in ~foreman/.ssh/

In any case, they have to be owned by the right user (and probably have the proper selinux context) and permissions set to 600. Another thing to watch out for is the keys have to be named id_rsa_foreman_proxy (the private key) and id_rsa_foreman_proxy.pub (the public key).

To try the keys are setup properly, run the following

sudo -u foreman ssh -i ~foreman/.ssh/id_rsa_foreman_proxy root.com

Comment 4 Ivan Necas 2018-05-28 15:53:18 UTC
What is the reason for turning remote_execution_without_proxy to false in the first place: As Adam mentioned, it's not recommended for production use.

I vote for opening a docs bug describing that this option is not recommended for production use.

Comment 8 Bryan Kearney 2019-12-03 16:34:49 UTC
The Satellite Team is attempting to provide an accurate backlog of bugzilla requests which we feel will be resolved in the next few releases. We do not believe this bugzilla will meet that criteria, and have plans to close it out in 1 month. This is not a reflection on the validity of the request, but a reflection of the many priorities for the product. If you have any concerns about this, feel free to contact Red Hat Technical Support or your account team. If we do not hear from you, we will close this bug out. Thank you.

Comment 9 Bryan Kearney 2020-01-15 20:30:36 UTC
Thank you for your interest in Satellite 6. We have evaluated this request, and while we recognize that it is a valid request, we do not expect this to be implemented in the product in the foreseeable future. This is due to other priorities for the product, and not a reflection on the request itself. We are therefore closing this out as WONTFIX. If you have any concerns about this, please do not reopen. Instead, feel free to contact Red Hat Technical Support. Thank you.