Bug 1381390

Summary: openstack-ironic: failed to start introspection, the nodes are in "enroll" state after registering.
Product: Red Hat OpenStack Reporter: Alexander Chuzhoy <sasha>
Component: openstack-ironicAssignee: Lucas Alvares Gomes <lmartins>
Status: CLOSED NOTABUG QA Contact: Raviv Bar-Tal <rbartal>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 10.0 (Newton)CC: mburns, rhel-osp-director-maint, sasha, srevivo
Target Milestone: ga   
Target Release: 10.0 (Newton)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-10-04 14:35:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Alexander Chuzhoy 2016-10-03 23:41:07 UTC
openstack-ironic:  failed to start introspection, the nodes are in "enroll" state after registering.

Environment:
python-ironic-inspector-client-1.9.0-0.20160916002438.6364bc9.el7ost.noarch
openstack-ironic-common-6.2.1-0.20160929033348.284005a.el7ost.noarch
openstack-ironic-api-6.2.1-0.20160929033348.284005a.el7ost.noarch
python-ironicclient-1.7.0-0.20160916002722.464044f.el7ost.noarch
puppet-ironic-9.4.0-0.20160928195647.1c94eb8.el7ost.noarch
openstack-ironic-conductor-6.2.1-0.20160929033348.284005a.el7ost.noarch
instack-undercloud-5.0.0-0.20160929132429.56e65af.el7ost.noarch
python-ironic-lib-2.1.0-0.20160915122034.52b2d2f.el7ost.noarch
openstack-ironic-inspector-4.2.1-0.20160922151040.36900fb.el7ost.noarc

Steps to reproduce:
1. import overcloud images.
2. Import ironic nodes.
3. Attempt to start introspection on all.
Result:

Exception registering nodes: The requested action "provide" can not be performed on node "d583c570-41ad-40b8-8347-a66fb49ebdcf" while it is in state "enroll".

Expected result:
The registered nodes should have 'provisionig state' available and not enroll.



The instackenv.json is below:        
{                                                                                                                                                                           
  "nodes": [                                                                                                                                                                
    {                                                                                                                                                                       
      "pm_password": "password",                                                                                                                                            
      "memory": 6144,                                                                                                                                                       
      "pm_addr": "192.168.102.168",                                                                                                                                         
      "mac": [                                                                                                                                                              
        "fa:16:3e:68:7d:16"                                                                                                                                                 
      ],                                                                                                                                                                    
      "capabilities": "boot_option:local",                                                                                                                                  
      "pm_type": "pxe_ipmitool",                                                                                                                                            
      "disk": 41,                                                                                                                                                           
      "arch": "x86_64",                                                                                                                                                     
      "cpu": 1,                                                                                                                                                             
      "pm_user": "admin"                                                                                                                                                    
    },                                                                                                                                                                      
    {                                                                                                                                                                       
      "pm_password": "password",                                                                                                                                            
      "memory": 6144,                                                                                                                                                       
      "pm_addr": "192.168.102.174",                                                                                                                                         
      "mac": [                                                                                                                                                              
        "fa:16:3e:3f:49:37"                                                                                                                                                 
      ],                                                                                                                                                                    
      "capabilities": "boot_option:local",                                                                                                                                  
      "pm_type": "pxe_ipmitool",                                                                                                                                            
      "disk": 41,                                                                                                                                                           
      "arch": "x86_64",                                                                                                                                                     
      "cpu": 1,                                                                                                                                                             
      "pm_user": "admin"                                                                                                                                                    
    },                                                                                                                                                                      
    {                                                                                                                                                                       
      "pm_password": "password",                                                                                                                                            
      "memory": 6144,                                                                                                                                                       
      "pm_addr": "192.168.102.182",                                                                                                                                         
      "mac": [                                                                                                                                                              
        "fa:16:3e:bb:a8:61"                                                                                                                                                 
      ],                                                                                                                                                                    
      "capabilities": "boot_option:local",                                                                                                                                  
      "pm_type": "pxe_ipmitool",                                                                                                                                            
      "disk": 41,                                                                                                                                                           
      "arch": "x86_64",                                                                                                                                                     
      "cpu": 1,                                                                                                                                                             
      "pm_user": "admin"                                                                                                                                                    
    },                                                                                                                                                                      
    {                                                                                                                                                                       
      "pm_password": "password",                                                                                                                                            
      "memory": 6144,                                                                                                                                                       
      "pm_addr": "192.168.102.177",                                                                                                                                         
      "mac": [                                                                                                                                                              
        "fa:16:3e:5e:aa:41"                                                                                                                                                 
      ],                                                                                                                                                                    
      "capabilities": "boot_option:local",                                                                                                                                  
      "pm_type": "pxe_ipmitool",                                                                                                                                            
      "disk": 41,                                                                                                                                                           
      "arch": "x86_64",                                                                                                                                                     
      "cpu": 1,                                                                                                                                                             
      "pm_user": "admin"                                                                                                                                                    
    },                                                                                                                                                                      
    {                                                                                                                                                                       
      "pm_password": "password",                                                                                                                                            
      "memory": 6144,                                                                                                                                                       
      "pm_addr": "192.168.102.171",                                                                                                                                         
      "mac": [                                                                                                                                                              
        "fa:16:3e:f3:55:72"                                                                                                                                                 
      ],                                                                                                                                                                    
      "capabilities": "boot_option:local",                                                                                                                                  
      "pm_type": "pxe_ipmitool",                                                                                                                                            
      "disk": 41,                                                                                                                                                           
      "arch": "x86_64",                                                                                                                                                     
      "cpu": 1,                                                                                                                                                             
      "pm_user": "admin"                                                                                                                                                    
    },                                                                                                                                                                      
    {                                                                                                                                                                       
      "pm_password": "password",                                                                                                                                            
      "memory": 6144,                                                                                                                                                       
      "pm_addr": "192.168.102.176",                                                                                                                                         
      "mac": [                                                                                                                                                              
        "fa:16:3e:3c:4e:de"                                                                                                                                                 
      ],                                                                                                                                                                    
      "capabilities": "boot_option:local",                                                                                                                                  
      "pm_type": "pxe_ipmitool",                                                                                                                                            
      "disk": 41,                                                                                                                                                           
      "arch": "x86_64",                                                                                                                                                     
      "cpu": 1,                                                                                                                                                             
      "pm_user": "admin"                                                                                                                                                    
    },                                                                                                                                                                      
    {                                                                                                                                                                       
      "pm_password": "password",                                                                                                                                            
      "memory": 6144,                                                                                                                                                       
      "pm_addr": "192.168.102.178",                                                                                                                                         
      "mac": [                                                                                                                                                              
        "fa:16:3e:89:33:16"                                                                                                                                                 
      ],                                                                                                                                                                    
      "capabilities": "boot_option:local",                                                                                                                                  
      "pm_type": "pxe_ipmitool",                                                                                                                                            
      "disk": 41,                                                                                                                                                           
      "arch": "x86_64",                                                                                                                                                     
      "cpu": 1,                                                                                                                                                             
      "pm_user": "admin"                                                                                                                                                    
    },                                                                                                                                                                      
    {                                                                                                                                                                       
      "pm_password": "password",                                                                                                                                            
      "memory": 6144,                                                                                                                                                       
      "pm_addr": "192.168.102.179",                                                                                                                                         
      "mac": [                                                                                                                                                              
        "fa:16:3e:86:33:41"                                                                                                                                                 
      ],                                                                                                                                                                    
      "capabilities": "boot_option:local",                                                                                                                                  
      "pm_type": "pxe_ipmitool",                                                                                                                                            
      "disk": 41,                                                                                                                                                           
      "arch": "x86_64",                                                                                                                                                     
      "cpu": 1,                                                                                                                                                             
      "pm_user": "admin"                                                                                                                                                    
    },                                                                                                                                                                      
    {                                                                                                                                                                       
      "pm_password": "password",                                                                                                                                            
      "memory": 6144,                                                                                                                                                       
      "pm_addr": "192.168.102.167",                                                                                                                                         
      "mac": [                                                                                                                                                              
        "fa:16:3e:45:10:b0"                                                                                                                                                 
      ],                                                                                                                                                                    
      "capabilities": "boot_option:local",                                                                                                                                  
      "pm_type": "pxe_ipmitool",                                                                                                                                            
      "disk": 41,                                                                                                                                                           
      "arch": "x86_64",                                                                                                                                                     
      "cpu": 1,                                                                                                                                                             
      "pm_user": "admin"                                                                                                                                                    
    },                                                                                                                                                                      
    {                                                                                                                                                                       
      "pm_password": "password",                                                                                                                                            
      "memory": 6144,                                                                                                                                                       
      "pm_addr": "192.168.102.172",                                                                                                                                         
      "mac": [                                                                                                                                                              
        "fa:16:3e:74:d3:10"                                                                                                                                                 
      ],                                                                                                                                                                    
      "capabilities": "boot_option:local",                                                                                                                                  
      "pm_type": "pxe_ipmitool",                                                                                                                                            
      "disk": 41,                                                                                                                                                           
      "arch": "x86_64",                                                                                                                                                     
      "cpu": 1,                                                                                                                                                             
      "pm_user": "admin"                                                                                                                                                    
    },                                                                                                                                                                      
    {                                                                                                                                                                       
      "pm_password": "password",                                                                                                                                            
      "memory": 6144,                                                                                                                                                       
      "pm_addr": "192.168.102.169",                                                                                                                                         
      "mac": [                                                                                                                                                              
        "fa:16:3e:b2:ff:d4"                                                                                                                                                 
      ],                                                                                                                                                                    
      "capabilities": "boot_option:local",                                                                                                                                  
      "pm_type": "pxe_ipmitool",                                                                                                                                            
      "disk": 41,                                                                                                                                                           
      "arch": "x86_64",                                                                                                                                                     
      "cpu": 1,                                                                                                                                                             
      "pm_user": "admin"                                                                                                                                                    
    },                                                                                                                                                                      
    {
      "pm_password": "password",
      "memory": 6144,
      "pm_addr": "192.168.102.180",
      "mac": [
        "fa:16:3e:6c:98:b5"
      ],
      "capabilities": "boot_option:local",
      "pm_type": "pxe_ipmitool",
      "disk": 41,
      "arch": "x86_64",
      "cpu": 1,
      "pm_user": "admin"
    },
    {
      "pm_password": "password",
      "memory": 6144,
      "pm_addr": "192.168.102.175",
      "mac": [
        "fa:16:3e:08:05:a7"
      ],
      "capabilities": "boot_option:local",
      "pm_type": "pxe_ipmitool",
      "disk": 41,
      "arch": "x86_64",
      "cpu": 1,
      "pm_user": "admin"
    },
    {
      "pm_password": "password",
      "memory": 6144,
      "pm_addr": "192.168.102.173",
      "mac": [
        "fa:16:3e:4f:19:c7"
      ],
      "capabilities": "boot_option:local",
      "pm_type": "pxe_ipmitool",
      "disk": 41,
      "arch": "x86_64",
      "cpu": 1,
      "pm_user": "admin"
    },
    {
      "pm_password": "password",
      "memory": 6144,
      "pm_addr": "192.168.102.181",
      "mac": [
        "fa:16:3e:31:be:31"
      ],
      "capabilities": "boot_option:local",
      "pm_type": "pxe_ipmitool",
      "disk": 41,
      "arch": "x86_64",
      "cpu": 1,
      "pm_user": "admin"
    }
  ]
}

Comment 1 Lucas Alvares Gomes 2016-10-04 13:44:10 UTC
Hi Alexander,

Can you please upload the logs from ironic-conductor and the output of "ironic node-show <node uuid/name>" ?

Enroll is a new state in Ironic and in order to it to progress to "manageable" and later "available" the nodes need to be created with valid credentials (otherwise they are not "manageable").

Comment 2 Alexander Chuzhoy 2016-10-04 14:10:38 UTC
Looking at the output from ironic node-show <id>
last_error: Failed to get power state for node f265e343-2794-46cd-88ab-96a554a58d87.  Error: IPMI call failed: power status.

Comment 3 Alexander Chuzhoy 2016-10-04 14:35:13 UTC
This was from ovb.
Bad instackenv.json was used.