Description of problem: private keys are logged with debug loglevel set https://github.com/openstack/python-ironicclient/blob/11a3f12c3adb4835e92ec0e8aaff270e2c54ab29/ironicclient/common/http.py#L280 after speaking with bnemec, its recommended that the output be sanitized before logging to remove private key
Do you have an example of which private keys are being logged? What command where you running at the time?
Assuming the bug is talking about OSP8/Liberty, we should probably backport https://github.com/openstack/python-ironicclient/commit/eeb327b590b489a7f6b0e2176b288f7201de6aa9 (upstream as well, as we still can backport security issues to stable/liberty iirc). Derek, this might be SSH private keys we're using for power management (also IPMI passwords).
Hmm, I can confirm that the node creation request does not sanitize its debugging output even in OSP 10 (python-ironicclient-1.7.0): DEBUG (session:337) REQ: curl -g -i -X POST http://172.21.64.1:6385/v1/nodes -H "X-OpenStack-Ironic-API-Version: 1.9" -H "User-Agent: python-ironicclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}b91de03dcf2e8c00d1781f33a94977613f33ac1b" -d '{"driver": "fake", "driver_info": {"ipmi_password": 12345}}' I'm not exactly sure why the code above does not work. Probably a problem with mask_password from oslo_utils?
I've found the problem: it's in keystoneauth, not in ironicclient: https://github.com/openstack/keystoneauth/blob/ab39cf066784dc01869a7e22a9289e154410e4e5/keystoneauth1/session.py#L340. We should have the same masking there, then backport all patches to older releases.
UPD: the command I used: ironic --debug node-create -d fake -i ipmi_password=a12345
@tsuter Do you have an example of which private keys are being logged? What command where you running at the time? In the mean time I'll progress on the assumption that the command you ran was the one that Dmitry used $ ironic --debug node-create -d fake -i ipmi_password=a12345
Ok, so there may be the second part to this bug: in oslo.utils ssh_key_contents is not included in the list of secrets: https://github.com/openstack/oslo.utils/blob/master/oslo_utils/strutils.py#L56-L60. We may want to fix it as well.
There are a number of different instances of secrets being displayed in this bug, to keep the objective of this bug clear I'm going to concentrate on the the case that tsuter originally reported Problem: ssh private keys are being displayed when running "ironic node-show" secrets have been masked from being displayed by the client since Jan 2016, https://review.openstack.org/#/c/263822/ so they are masked in OSP 9 and 10 but not earlier versions but the ssh private key isn't include in the list of things to mask, it needs to be added the _SANITIZE_KEYS list in oslo_utils/strutils.py this will cover OSP 10 but will need to be backported to OSP 9 in order to cover earlier versions we'll need to backport both the patch to ironicclient/common/http.py and oslo_utils/strutils.py ironic also uses the _SANITIZE_KEYS list, adding ssh_key_contents to it will also prevent the key being sent to the client in the first place.
Fix merged into upstream newton https://review.openstack.org/#/c/401160/2/ironic/api/controllers/v1/node.py
According to our records, this should be resolved by openstack-ironic-6.2.4-1.el7ost. This build is available now.
installed latest osp10 undercloud with puddle 2017-10-30.3 on 11/1/2017 register node with debug and check output This bug has been verified environment openstack-ironic-api-6.2.4-2.el7ost.noarch openstack-ironic-conductor-6.2.4-2.el7ost.noarch openstack-ironic-common-6.2.4-2.el7ost.noarch openstack-ironic-inspector-4.2.2-3.el7ost.noarch cat node.yaml nodes: - name: overcloud-2-node5 driver: pxe_ssh driver_info: ssh_username: stack ssh_key_contents: | -----BEGIN RSA PRIVATE KEY----- MIIEogIBAAKCAQEAxc0a2u18EgTy5y9JvaExDXP2pWuE8Ebyo24AOo1iQoWR7D5n fNjkgCeKZRbABhsdoMBmbDMtn0PO3lzI2HnZQBB4BdBZprAiQ1NwKKotUv9puTeY [..] 7DsSKAL4EDqjufY3h+4fRwOcD+EFqlUTDG1sjsSDKjdiHyYMzjcrg8nbaj/M9kAs xXnSm9686KxUiCDXO5FWKun204B18mPH1UP20aYw098t6aAQwm4= -----END RSA PRIVATE KEY----- ssh_virt_type: virsh ssh_address: 10.0.0.1 properties: cpus: 4 memory_mb: 12288 local_gb: 60 #boot_option: local (it doesn't set 'capabilities') root_device: # vendor: "0x1af4" # model: "QEMU HARDDISK" # size: 64424509440 wwn: "0x0000000000000001" # serial: "0000000000000001" # vendor: QEMU # name: /dev/sda ports: - address: 52:54:00:a0:af:da ironic --debug node-create -d fake -i ipmi_password=a12345 [stack@undercloud ~]$ ironic --debug node-show overcloud-2-node5 DEBUG (extension:157) found extension EntryPoint.parse('v2token = keystoneauth1.loading._plugins.identity.v2:Token') DEBUG (extension:157) found extension EntryPoint.parse('v3oauth1 = keystoneauth1.extras.oauth1._loading:V3OAuth1') DEBUG (extension:157) found extension EntryPoint.parse('admin_token = keystoneauth1.loading._plugins.admin_token:AdminToken') DEBUG (extension:157) found extension EntryPoint.parse('v3oidcauthcode = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAuthorizationCode') DEBUG (extension:157) found extension EntryPoint.parse('v2password = keystoneauth1.loading._plugins.identity.v2:Password') DEBUG (extension:157) found extension EntryPoint.parse('v3samlpassword = keystoneauth1.extras._saml2._loading:Saml2Password') DEBUG (extension:157) found extension EntryPoint.parse('v3password = keystoneauth1.loading._plugins.identity.v3:Password') DEBUG (extension:157) found extension EntryPoint.parse('v3oidcaccesstoken = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectAccessToken') DEBUG (extension:157) found extension EntryPoint.parse('v3oidcpassword = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectPassword') DEBUG (extension:157) found extension EntryPoint.parse('v3kerberos = keystoneauth1.extras.kerberos._loading:Kerberos') DEBUG (extension:157) found extension EntryPoint.parse('token = keystoneauth1.loading._plugins.identity.generic:Token') DEBUG (extension:157) found extension EntryPoint.parse('v3oidcclientcredentials = keystoneauth1.loading._plugins.identity.v3:OpenIDConnectClientCredentials') DEBUG (extension:157) found extension EntryPoint.parse('v3tokenlessauth = keystoneauth1.loading._plugins.identity.v3:TokenlessAuth') DEBUG (extension:157) found extension EntryPoint.parse('v3token = keystoneauth1.loading._plugins.identity.v3:Token') DEBUG (extension:157) found extension EntryPoint.parse('v3totp = keystoneauth1.loading._plugins.identity.v3:TOTP') DEBUG (extension:157) found extension EntryPoint.parse('password = keystoneauth1.loading._plugins.identity.generic:Password') DEBUG (extension:157) found extension EntryPoint.parse('v3fedkerb = keystoneauth1.extras.kerberos._loading:MappedKerberos') DEBUG (extension:157) found extension EntryPoint.parse('token_endpoint = openstackclient.api.auth_plugin:TokenEndpoint') DEBUG (extension:157) found extension EntryPoint.parse('aodh-noauth = aodhclient.noauth:AodhNoAuthLoader') DEBUG (extension:157) found extension EntryPoint.parse('gnocchi-basic = gnocchiclient.auth:GnocchiBasicLoader') DEBUG (extension:157) found extension EntryPoint.parse('gnocchi-noauth = gnocchiclient.auth:GnocchiNoAuthLoader') DEBUG (session:342) REQ: curl -g -i -X GET http://192.168.0.1:5000/v2.0 -H "Accept: application/json" -H "User-Agent: ironic keystoneauth1/2.12.3 python-requests/2.11.1 CPython/2.7.5" INFO (connectionpool:214) Starting new HTTP connection (1): 192.168.0.1 DEBUG (connectionpool:401) "GET /v2.0 HTTP/1.1" 200 230 DEBUG (session:390) RESP: [200] Date: Sat, 21 Oct 2017 01:14:54 GMT Server: Apache/2.4.6 (Red Hat Enterprise Linux) Vary: X-Auth-Token,Accept-Encoding x-openstack-request-id: req-fd331491-f18e-4e01-b0d2-1deae86519ac Content-Encoding: gzip Content-Length: 230 Connection: close Content-Type: application/json RESP BODY: {"version": {"status": "deprecated", "updated": "2016-08-04T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "http://192.168.0.1:5000/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}]}} DEBUG (v2:63) Making authentication request to http://192.168.0.1:5000/v2.0/tokens INFO (connectionpool:249) Resetting dropped connection: 192.168.0.1 DEBUG (connectionpool:401) "POST /v2.0/tokens HTTP/1.1" 200 1097 DEBUG (session:342) REQ: curl -g -i -X GET http://192.168.0.1:6385/v1/nodes/overcloud-2-node5 -H "X-OpenStack-Ironic-API-Version: 1.9" -H "User-Agent: python-ironicclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}10d1283f542c7700b2a7f7b2e821f64f00edb890" INFO (connectionpool:214) Starting new HTTP connection (1): 192.168.0.1 DEBUG (connectionpool:401) "GET /v1/nodes/overcloud-2-node5 HTTP/1.1" 200 1246 DEBUG (session:390) RESP: [200] X-Openstack-Ironic-Api-Minimum-Version: 1.1 X-Openstack-Ironic-Api-Maximum-Version: 1.22 X-Openstack-Ironic-Api-Version: 1.9 Content-Length: 1246 Content-Type: application/json; charset=UTF-8 Openstack-Request-Id: req-6e4a3981-4918-4429-aa8d-133ea2004c29 Date: Sat, 21 Oct 2017 01:14:54 GMT Connection: keep-alive RESP BODY: {"target_power_state": null, "links": [{"href": "http://192.168.0.1:6385/v1/nodes/7de84d3f-22aa-4cd9-b482-50a80f971284", "rel": "self"}, {"href": "http://192.168.0.1:6385/nodes/7de84d3f-22aa-4cd9-b482-50a80f971284", "rel": "bookmark"}], "extra": {}, "last_error": null, "updated_at": null, "maintenance_reason": null, "provision_state": "enroll", "clean_step": {}, "uuid": "7de84d3f-22aa-4cd9-b482-50a80f971284", "console_enabled": false, "target_provision_state": null, "provision_updated_at": null, "power_state": null, "inspection_started_at": null, "inspection_finished_at": null, "maintenance": false, "driver": "pxe_ssh", "reservation": null, "properties": {"memory_mb": 12288, "root_device": {"wwn": "0x0000000000000001"}, "local_gb": 60, "cpus": 4}, "instance_uuid": null, "name": "overcloud-2-node5", "driver_info": {"ssh_username": "stack", "ssh_virt_type": "virsh", "ssh_address": "10.0.0.1", "ssh_key_contents": "******"}, "created_at": "2017-10-21T01:13:53+00:00", "ports": [{"href": "http://192.168.0.1:6385/v1/nodes/7de84d3f-22aa-4cd9-b482-50a80f971284/ports", "rel": "self"}, {"href": "http://192.168.0.1:6385/nodes/7de84d3f-22aa-4cd9-b482-50a80f971284/ports", "rel": "bookmark"}], "driver_internal_info": {}, "instance_info": {}} +------------------------+-------------------------------------------------------------------------+ | Property | Value | +------------------------+-------------------------------------------------------------------------+ | chassis_uuid | | | clean_step | {} | | console_enabled | False | | created_at | 2017-10-21T01:13:53+00:00 | | driver | pxe_ssh | | driver_info | {u'ssh_username': u'stack', u'ssh_virt_type': u'virsh', u'ssh_address': | | | u'10.0.0.1', u'ssh_key_contents': u'******'} | | driver_internal_info | {} | | extra | {} | | inspection_finished_at | None | | inspection_started_at | None | | instance_info | {} | | instance_uuid | None | | last_error | None | | maintenance | False | | maintenance_reason | None | | name | overcloud-2-node5 | | network_interface | | | power_state | None | | properties | {u'memory_mb': 12288, u'root_device': {u'wwn': u'0x0000000000000001'}, | | | u'local_gb': 60, u'cpus': 4} | | provision_state | enroll | | provision_updated_at | None | | raid_config | | | reservation | None | | resource_class | | | target_power_state | None | | target_provision_state | None | | target_raid_config | | | updated_at | None | | uuid | 7de84d3f-22aa-4cd9-b482-50a80f971284 | +------------------------+-------------------------------------------------------------------------+
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3235