Bug 1156000 - Resize of flavor and migrate fails due to insufficient SSH keys distribution
Summary: Resize of flavor and migrate fails due to insufficient SSH keys distribution
Keywords:
Status: CLOSED DUPLICATE of bug 1221776
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director
Version: 7.0 (Kilo)
Hardware: Unspecified
OS: Unspecified
high
urgent
Target Milestone: ---
: Director
Assignee: chris alfonso
QA Contact: yeylon@redhat.com
Mike Burns
URL:
Whiteboard:
Depends On:
Blocks: 1156010
TreeView+ depends on / blocked
 
Reported: 2014-10-23 11:27 UTC by Marian Krcmarik
Modified: 2016-04-18 07:01 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-08-28 16:43:05 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Marian Krcmarik 2014-10-23 11:27:40 UTC
Description of problem:
SSH keys distribution is needed for some Openstack features (such as resize of an instance to diffent flavor which uses SSH migration and There are some checks through SSH of existence of storage). SSh keys should be generated on the compute nodes for nova user as well as It should be added to authorized keys of other nodes and a node should has other nodes in known_hosts for nova user, so the ssh commands does not fail.
This is implemted in packstack in RHOS5.0 but not in staypuft

Version-Release number of selected component (if applicable):


How reproducible:
Always

Steps to Reproduce:
1. In RHOS setup deployed by staypuft try to resize running instance.

Actual results:
It fails due to missing ssh keys distributed.
Command: ssh 192.168.0.7 mkdir -p /var/lib/nova/instances/07ff020f-c1db-41d9-a55d-0dec2448d2cd
2014-10-22 12:34:33.738 22767 TRACE nova.compute.manager [instance: 07ff020f-c1db-41d9-a55d-0dec2448d2cd] Exit code: 255
2014-10-22 12:34:33.738 22767 TRACE nova.compute.manager [instance: 07ff020f-c1db-41d9-a55d-0dec2448d2cd] Stdout: u''
2014-10-22 12:34:33.738 22767 TRACE nova.compute.manager [instance: 07ff020f-c1db-41d9-a55d-0dec2448d2cd] Stderr: u'Host key verification failed.\r\n'


Expected results:
SSH keys should be generated and distributed correctly.

Additional info:

Comment 3 Marian Krcmarik 2015-01-22 14:04:29 UTC
To clarify more:
One of the other features which uses SSH is offline migration triggered by command "nova migrate instance". Staypuft does not generate any SSH keys for nodes (compute and controller) as well as It does not place such generated keys into authorized_keys on the other nodes - so There is no SSH key distribution. This cause all the feature which depend on SSH to fail - in the case of migration (nova migrate) instance ends up in ERROR state. There is a live migration triggered by command "nova live-migration" which does not use SSH and works smoothly but from user point of perspective I would say that users will find and use "nova migrate" command firstly which will place their instances into ERROR state.

Packstack does generate public SSH keys for this purpose and place it into authorized_keys on the nodes but It did not work for me as well unless I allowed login shell for nova user and places the ssh public keys of the nodes into known_hosts file of nova users on the other nodes. I have separated bug #1156010 for allowing the nova user to have a login shell which may have security impact.

I am raising severity since some of the features do not work out of the box. Possibly this feature "nova migrate" could be taken away since "nova live-migration" works without use of SSH (over TCP).

Comment 4 Ofer Blaut 2015-01-28 10:05:44 UTC
I have tested VM resize on latest rhel-osp 6 and it fails

to reproduce :
create VM
check its flavour 
resize it flavour to bigger one


[root@controller ~(keystone_admin_tenant1)]$nova resize 1781233a-75a8-478d-95f3-bc5149f5a3d2 3
[root@controller ~(keystone_admin_tenant1)]$nova show 1781233a-75a8-478d-95f3-bc5149f5a3d2
+--------------------------------------+--------------------------------------------------------------------------------------------------+
| Property                             | Value                                                                                            |
+--------------------------------------+--------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                                                                           |
| OS-EXT-AZ:availability_zone          | nova                                                                                             |
| OS-EXT-STS:power_state               | 1                                                                                                |
| OS-EXT-STS:task_state                | -                                                                                                |
| OS-EXT-STS:vm_state                  | error                                                                                            |
| OS-SRV-USG:launched_at               | 2015-01-20T06:15:30.000000                                                                       |
| OS-SRV-USG:terminated_at             | -                                                                                                |
| accessIPv4                           |                                                                                                  |
| accessIPv6                           |                                                                                                  |
| config_drive                         |                                                                                                  |
| created                              | 2015-01-20T06:14:10Z                                                                             |
| fault                                | {"message": "Unexpected error while running command.                                             |
|                                      | Command: ssh 10.35.186.231 mkdir -p /var/lib/nova/instances/1781233a-75a8-478d-95f3-bc5149f5a3d2 |
|                                      | Exit code: 255                                                                                   |
|                                      | Stdout: u''                                                                                      |
|                                      | Stderr: u'Host key verification failed.\\r\                                                      |
|                                      | '", "code": 500, "created": "2015-01-28T08:49:48Z"}                                              |
| flavor                               | m1.small (2)                                                                                     |
| hostId                               | 5d600347a8839a6ad387e64de85d88b9aa9318fab380d0e29daa7ef0                                         |
| id                                   | 1781233a-75a8-478d-95f3-bc5149f5a3d2                                                             |
| image                                | cirros (5b099a2c-a682-466f-82c1-73a229118342)                                                    |
| key_name                             | oskey1                                                                                           |
| metadata                             | {}                                                                                               |
| name                                 | C                                                                                                |
| net201 network                       | 3001::f816:3eff:fe52:d005, 21.0.0.5                                                              |
| os-extended-volumes:volumes_attached | []                                                                                               |
| security_groups                      | default                                                                                          |
| status                               | ERROR                                                                                            |
| tenant_id                            | caa7603588c14954b2f4f6616f87d4bc                                                                 |
| updated                              | 2015-01-28T08:49:48Z                                                                             |
| user_id                              | efa9b7baadc149bd8082eede59a3121c

Comment 6 Ivan Chavero 2015-02-09 22:44:55 UTC
This seems to be a Staypuft problem, changing component

Comment 19 Marian Krcmarik 2015-07-29 12:37:07 UTC
I am moving this bug to osp-d since the same problem occurs there as already said this was supported by packstack but it lacks of support by staypuft or osp-d so feature as resize flavor or offline migration does not work out of box on deployments deployed by osp-d.

Comment 20 Mike Burns 2015-08-28 16:43:05 UTC

*** This bug has been marked as a duplicate of bug 1221776 ***


Note You need to log in before you can comment on or make changes to this bug.