Bug 1186171

Summary: [routing-daemon] Improper scp command cause the failure of copying certificate and key for alias to BIGIP LTM host
Product: OpenShift Container Platform Reporter: Johnny Liu <jialiu>
Component: NodeAssignee: chris alfonso <calfonso>
Status: CLOSED ERRATA QA Contact: libra bugs <libra-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 2.2.0CC: calfonso, hbrock, jdetiber, jokerman, libra-onpremise-devel, mmccomas
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: rubygem-openshift-origin-routing-daemon-0.22.1.2-1.el6op Doc Type: Bug Fix
Doc Text:
Cause: A bug in the HA routing daemon prevented certificates from being copied to BIGIP LTM hosts. Consequence: Certificates were not properly copied to BIGIP LTM hosts. Fix: The routing daemon has been updated to properly copy certificates to BIGIP LTM hosts. Result: Certificates are now copied to BIGIP LTM hosts.
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-02-12 13:10:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Johnny Liu 2015-01-27 09:12:55 UTC
Description of problem:
In /opt/rh/ruby193/root/usr/share/gems/gems/openshift-origin-routing-daemon-0.22.1.1/lib/openshift/routing/models/f5-icontrol-rest.rb:
203         result = `scp -i #{@ssh_private_key} #{certfname.path} admin@    #{@host}:/var/tmp/#{alias_str}.crt`
204         result = `scp -i #{@ssh_private_key} #{keyfname.path} admin@#    {@host}:/var/tmp/#{alias_str}.key`

That would cause the failure of copying certificate and key for alias to BIGIP LTM host, because this is the first ssh connection to BIG LTM host, it has interactive behaviour.
# ssh -i /etc/openshift/bigip.key admin.21.25
The authenticity of host '54.175.21.25 (54.175.21.25)' can't be established.
RSA key fingerprint is 90:6f:08:b6:65:5c:ec:f9:ed:c3:7c:b1:5c:25:6c:83.
Are you sure you want to continue connecting (yes/no)?


should changed to the following to avoid such interaction:
203         result = `scp -o StrictHostKeyChecking=no -o PasswordAuthentication=no -o VerifyHostKeyDNS=no -o UserKnownHostsFile=/dev/null -i #{@ssh_private_key} #{certfname.path} admin@    #{@host}:/var/tmp/#{alias_str}.crt`
204         result = `scp -o StrictHostKeyChecking=no -o PasswordAuthentication=no -o VerifyHostKeyDNS=no -o UserKnownHostsFile=/dev/null -i #{@ssh_private_key} #{keyfname.path} admin@#    {@host}:/var/tmp/#{alias_str}.key`


Version-Release number of selected component (if applicable):
rubygem-openshift-origin-routing-daemon-0.22.1.1-1.el6op.noarch

How reproducible:
Always

Steps to Reproduce:
1.Set up BIG LTM env
2.Set up ose all-in-one env, and install routing-daemon, configure it to use BIGIP as external LB.
3.Create a scalable app, add alias, and update ssl cert for this alias

Actual results:
The error log in routing-daemon is shown when uploading alias ssl cert.
D, [2015-01-27T03:41:13.195436 #31234] DEBUG -- : Copying certificate and key for alias www.app1.com for pool pool_ose_myapp_jialiu_80 to LTM host

==> /var/log/openshift-routing-daemon.output <==
Host key verification failed.
lost connection
Host key verification failed.
lost connection

==> /var/log/openshift/routing-daemon.log <==
D, [2015-01-27T03:41:13.444515 #31234] DEBUG -- : LTM cert to be installed /var/tmp/www.app1.com.crt
W, [2015-01-27T03:41:13.704547 #31234]  WARN -- : Got an exception: 404 Resource Not Found
D, [2015-01-27T03:41:13.704769 #31234] DEBUG -- : Backtrace:


Expected results:
No error.

Additional info:

Comment 1 Johnny Liu 2015-01-27 09:19:24 UTC
The same issues also happened with the following lines:
238         # Requires LTM System->Users->admin terminal setting to be set to advanced (bash)
239         @logger.debug("LTM removing temporary alias certificate. rm -f /var/tmp/#{alias_str}.crt")
240         result = `ssh -i #{@ssh_private_key} admin@#{@host} 'rm -f /var/tmp/#{alias_str}.crt'`
241         @logger.debug("LTM removing temporary alias key. rm -f /var/tmp/#{alias_str}.key")
242         result = `ssh -i #{@ssh_private_key} admin@#{@host} 'rm -f /var/tmp/#{alias_str}.key'`


All the ssh/scp should be added with "-o StrictHostKeyChecking=no -o PasswordAuthentication=no -o VerifyHostKeyDNS=no -o UserKnownHostsFile=/dev/null" option.

Comment 2 chris alfonso 2015-01-27 16:54:36 UTC
PR opened upstream, https://github.com/openshift/origin-server/pull/6060

Comment 5 Johnny Liu 2015-02-03 08:58:25 UTC
Verified the bug with rubygem-openshift-origin-routing-daemon-0.22.1.2-1.el6op.noarch, and PASS.

Now all the scp/ssh commands are added with suggested options, when adding ssl cert via scp for the first time, it is working well.

I, [2015-02-03T03:42:10.078715 #8490]  INFO -- : Adding ssl configuration for www.app2.com in pool pool_ose_myapp_jialiu_80
D, [2015-02-03T03:42:10.079863 #8490] DEBUG -- : Copying certificate and key for alias www.app2.com for pool pool_ose_myapp_jialiu_80 to LTM host

==> /var/log/openshift-routing-daemon.output <==
Warning: Permanently added '54.175.21.25' (RSA) to the list of known hosts.
Warning: Permanently added '54.175.21.25' (RSA) to the list of known hosts.

Comment 7 errata-xmlrpc 2015-02-12 13:10:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0220.html