Bug 754250 - instance w/ audrey bits fails to launch, stuck in "New" state
Summary: instance w/ audrey bits fails to launch, stuck in "New" state
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: CloudForms Cloud Engine
Classification: Retired
Component: aeolus-audrey-agent
Version: 1.0.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: rc
Assignee: Dan Radez
QA Contact: dgao
URL:
Whiteboard:
Depends On: 754744 754810
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-11-15 20:42 UTC by dgao
Modified: 2012-08-30 17:18 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed:
Embargoed:


Attachments (Terms of Use)

Description dgao 2011-11-15 20:42:26 UTC
Looks like launching an instance w/ audrey client rpms in conductor is stuck in the "New" state. This was also reproduced in automation using mock.

Comment 1 wes hayutin 2011-11-15 20:45:38 UTC
http://hudson.rhq.lab.eng.bos.redhat.com:8080/hudson/job/nightly-cloudEngine-smokeTest-Fedora/49/TestNG_Report/

Click on link: 'Administer'. (//a[normalize-space(.)='Administer'])
Navigate to Provider Accounts
Administer already selected
Click on link: 'Cloud Providers'. (//a[normalize-space(.)='Cloud Providers'])
Wait for element '//span[@Class='label light' and normalize-space(.)='Choose a provider:']', with timeout of 60000.
Provider already selected
Click on link: element with id 'details_accounts'. (//*[@id='details_accounts'])
Wait for element 'element with id 'new_provider_account'. (//*[normalize-space(@id)='new_provider_account'])', with timeout of 60000.
Click on link: 'mock_mockuser'. (//a[normalize-space(.)='mock_mockuser'])
Click on link: 'Add'. (//a[normalize-space(.)='Add'])
Type ' into password textbox: element with id 'config_server_secret'. (//*[normalize-space(@id)='config_server_secret'])
Type 'https://https://ec2-50-17-8-200.compute-1.amazonaws.com' into textbox: element with id 'config_server_endpoint'. (//*[normalize-space(@id)='config_server_endpoint'])
Type '' into textbox: element with id 'config_server_key'. (//*[normalize-space(@id)='config_server_key'])
Click on submit button: 'Save'. (//*[@value='Save'])
SCP: Copying /home/hudson/.hudson/jobs/nightly-cloudEngine-smokeTest-Fedora/workspace/automatjon/cloudEngine/scripts/audrey-client-template.tpl to qeblade30.rhq.lab.eng.bos.redhat.com:/root/
SCP: Transfer succeeded
DEBUG/root/audrey-client-template.tpl
ssh root.lab.eng.bos.redhat.com aeolus-cli build --target ec2 --template=/root/audrey-client-template.tpl
Stdout: 
Image: 94c9cb8c-6f07-4715-b074-f717437b7e4a
Build: c88309b1-faa6-4f6f-87f7-27e5e50c7c73
Target Image: 9991bc9b-4df7-4104-88ec-0d72318431b4	 :Status New
Stderr:
ExitCode: 0
Image: 94c9cb8c-6f07-4715-b074-f717437b7e4a
Build: c88309b1-faa6-4f6f-87f7-27e5e50c7c73
Target Image: 9991bc9b-4df7-4104-88ec-0d72318431b4	 :Status New
Image ID=94c9cb8c-6f07-4715-b074-f717437b7e4a
Build ID=c88309b1-faa6-4f6f-87f7-27e5e50c7c73
Target ID=9991bc9b-4df7-4104-88ec-0d72318431b4
ssh root.lab.eng.bos.redhat.com ls /root/getTemplateStatus.py
Stdout: /root/getTemplateStatus.py
Stderr:
ExitCode: 0
testing template build status
ssh root.lab.eng.bos.redhat.com /root/getTemplateStatus.py -i 9991bc9b-4df7-4104-88ec-0d72318431b4
Stdout: COMPLETED
Stderr:
ExitCode: 0
STATUS=COMPLETED
Sleep for 2 minutes
Template Fedora 15 Audrey Client has COMPLETED building
Asserted: Asserted template Fedora 15 Audrey Client has been built successfully
ssh root.lab.eng.bos.redhat.com aeolus-cli push --account ec2-us-east-1_AKIAJ557U7P7OIHRV2EQ --targetimage 9991bc9b-4df7-4104-88ec-0d72318431b4
Stdout: 
Target Image: 9991bc9b-4df7-4104-88ec-0d72318431b4
Provider Image: f7c20935-a695-42d8-b18a-c868f05c9428	 Status: New
Stderr:
ExitCode: 0
Provider ID=f7c20935-a695-42d8-b18a-c868f05c9428	 Status
ssh root.lab.eng.bos.redhat.com ls /root/getTemplateStatus.py
Stdout: /root/getTemplateStatus.py
Stderr:
ExitCode: 0
testing template push status
ssh root.lab.eng.bos.redhat.com /root/getTemplateStatus.py -i f7c20935-a695-42d8-b18a-c868f05c9428	 Status
Stdout: New
Stderr:
ExitCode: 0
STATUS=New
Sleep for 2 minutes
Template Fedora 15 Audrey Client is PUSHING
Attempts = 1/20
Sleep for 60000ms.
STATUS =PUSHING
Fedora 15 Audrey Client is PUSHING
ssh root.lab.eng.bos.redhat.com ls /root/getTemplateStatus.py
Stdout: /root/getTemplateStatus.py
Stderr:
ExitCode: 0
testing template push status
ssh root.lab.eng.bos.redhat.com /root/getTemplateStatus.py -i f7c20935-a695-42d8-b18a-c868f05c9428	 Status
Stdout: PUSHING
Stderr:
ExitCode: 0
STATUS=PUSHING
Sleep for 2 minutes
Template Fedora 15 Audrey Client is PUSHING
Attempts = 2/20
Sleep for 60000ms.
STATUS =PUSHING
Fedora 15 Audrey Client is PUSHING
ssh root.lab.eng.bos.redhat.com ls /root/getTemplateStatus.py
Stdout: /root/getTemplateStatus.py
Stderr:
ExitCode: 0
testing template push status
ssh root.lab.eng.bos.redhat.com /root/getTemplateStatus.py -i f7c20935-a695-42d8-b18a-c868f05c9428	 Status
Stdout: PUSHING
Stderr:
ExitCode: 0
STATUS=PUSHING
Sleep for 2 minutes
Template Fedora 15 Audrey Client is PUSHING
Attempts = 3/20
Sleep for 60000ms.
STATUS =PUSHING
Fedora 15 Audrey Client is PUSHING
ssh root.lab.eng.bos.redhat.com ls /root/getTemplateStatus.py
Stdout: /root/getTemplateStatus.py
Stderr:
ExitCode: 0
testing template push status
ssh root.lab.eng.bos.redhat.com /root/getTemplateStatus.py -i f7c20935-a695-42d8-b18a-c868f05c9428	 Status
Stdout: PUSHING
Stderr:
ExitCode: 0
STATUS=PUSHING
Sleep for 2 minutes
Template Fedora 15 Audrey Client is PUSHING
Attempts = 4/20
Sleep for 60000ms.
STATUS =PUSHING
Fedora 15 Audrey Client is PUSHING
ssh root.lab.eng.bos.redhat.com ls /root/getTemplateStatus.py
Stdout: /root/getTemplateStatus.py
Stderr:
ExitCode: 0
testing template push status
ssh root.lab.eng.bos.redhat.com /root/getTemplateStatus.py -i f7c20935-a695-42d8-b18a-c868f05c9428	 Status
Stdout: PUSHING
Stderr:
ExitCode: 0
STATUS=PUSHING
Sleep for 2 minutes
Template Fedora 15 Audrey Client is PUSHING
Attempts = 5/20
Sleep for 60000ms.
STATUS =PUSHING
Fedora 15 Audrey Client is PUSHING
ssh root.lab.eng.bos.redhat.com ls /root/getTemplateStatus.py
Stdout: /root/getTemplateStatus.py
Stderr:
ExitCode: 0
testing template push status
ssh root.lab.eng.bos.redhat.com /root/getTemplateStatus.py -i f7c20935-a695-42d8-b18a-c868f05c9428	 Status
Stdout: PUSHING
Stderr:
ExitCode: 0
STATUS=PUSHING
Sleep for 2 minutes
Template Fedora 15 Audrey Client is PUSHING
Attempts = 6/20
Sleep for 60000ms.
STATUS =PUSHING
Fedora 15 Audrey Client is PUSHING
ssh root.lab.eng.bos.redhat.com ls /root/getTemplateStatus.py
Stdout: /root/getTemplateStatus.py
Stderr:
ExitCode: 0
testing template push status
ssh root.lab.eng.bos.redhat.com /root/getTemplateStatus.py -i f7c20935-a695-42d8-b18a-c868f05c9428	 Status
Stdout: PUSHING
Stderr:
ExitCode: 0
STATUS=PUSHING
Sleep for 2 minutes
Template Fedora 15 Audrey Client is PUSHING
Attempts = 7/20
Sleep for 60000ms.
STATUS =PUSHING
Fedora 15 Audrey Client is PUSHING
ssh root.lab.eng.bos.redhat.com ls /root/getTemplateStatus.py
Stdout: /root/getTemplateStatus.py
Stderr:
ExitCode: 0
testing template push status
ssh root.lab.eng.bos.redhat.com /root/getTemplateStatus.py -i f7c20935-a695-42d8-b18a-c868f05c9428	 Status
Stdout: PUSHING
Stderr:
ExitCode: 0
STATUS=PUSHING
Sleep for 2 minutes
Template Fedora 15 Audrey Client is PUSHING
Attempts = 8/20
Sleep for 60000ms.
STATUS =PUSHING
Fedora 15 Audrey Client is PUSHING
ssh root.lab.eng.bos.redhat.com ls /root/getTemplateStatus.py
Stdout: /root/getTemplateStatus.py
Stderr:
ExitCode: 0
testing template push status
ssh root.lab.eng.bos.redhat.com /root/getTemplateStatus.py -i f7c20935-a695-42d8-b18a-c868f05c9428	 Status
Stdout: COMPLETED
Stderr:
ExitCode: 0
STATUS=COMPLETED
Sleep for 2 minutes
Template Fedora 15 Audrey Client has COMPLETED uploading
STATUS =COMPLETED
Asserted: Asserted template Fedora 15 Audrey Client has been uploaded successfully
<deployable name='ce-5clvz'>
<description>auto description</description>
<assemblies>
<assembly name='testMock' hwp='i386SMALL'>
<image id='94c9cb8c-6f07-4715-b074-f717437b7e4a' build='c88309b1-faa6-4f6f-87f7-27e5e50c7c73' />
<services>
<service name='auto_Mock_service'>
<executable url='https://www.aeolusproject.org/redmine/attachments/download/169/start_simple' />
<parameters>
<parameter name='auto_service_param1' type='scalar'>
<value>![CDATA[value 2]]</value>
</parameter>
</parameters>
</service>
</services>
</assembly>
</assemblies>
</deployable>
ssh root.lab.eng.bos.redhat.com mkdir /var/www/html/ce/
Stdout:
Stderr: mkdir: cannot create directory `/var/www/html/ce/': File exists
ExitCode: 1
SCP: Copying automatjon/cloudEngine/yum/ce-5clvz to qeblade30.rhq.lab.eng.bos.redhat.com:/var/www/html/ce/
SCP: Transfer succeeded
ssh root.lab.eng.bos.redhat.com chmod 777 /var/www/html/ce/ce-5clvz
Stdout:
Stderr:
ExitCode: 0
Click on link: 'Administer'. (//a[normalize-space(.)='Administer'])
Administer already selected
Click on link: 'Content'. (//a[normalize-space(.)='Content'])
Click on link: element with id 'details_catalogs'. (//*[normalize-space(@id)='details_catalogs'])
Wait for element 'element with id 'new_catalog_button'. (//*[normalize-space(@id)='new_catalog_button'])', with timeout of 60000.
Click on link: 'test_catalog_4ae5j'. (//a[normalize-space(.)='test_catalog_4ae5j'])
Click on link: 'New Catalog Entry'. (//a[normalize-space(.)='New Catalog Entry'])
Type 'ce-5clvz' into textbox: element with id 'catalog_entry_name'. (//*[@id='catalog_entry_name'])
Type 'ce-5clvz' into textarea: element with id 'catalog_entry_description'. (//*[@id='catalog_entry_description'])
Select option 'test_catalog_4ae5j' in list element with id 'catalog_entry_catalog_id'. (//*[normalize-space(@id)='catalog_entry_catalog_id'])
Type 'http://qeblade30.rhq.lab.eng.bos.redhat.com/ce/ce-5clvz' into textbox: element with id 'catalog_entry_url'. (//*[@id='catalog_entry_url'])
Click on submit button: 'Save'. (//*[@value='Save'])
Click on link: 'Monitor'. (//a[normalize-space(.)='Monitor'])
Click on link: 'Default'. (//a[normalize-space(.)='Default'])
Click on link: 'New Deployment'. (//a[normalize-space(.)='New Deployment'])
Type 'Deployment-ConfigSrv-dd8h3' into textbox: element with id 'deployment_name'. (//*[normalize-space(@id)='deployment_name'])
Select option 'ce-5clvz' in list element with id 'catalog_entry_id'. (//*[normalize-space(@id)='catalog_entry_id'])
Select option 'realm-mock' in list element with id 'deployment_frontend_realm_id'. (//*[normalize-space(@id)='deployment_frontend_realm_id'])
Click on submit button: value 'Next'. (//input[@value='Next'])
Asserted: Found launch time param page
Click on submit button: element with id 'submit_params'. (//*[normalize-space(@id)='submit_params'])
Click on submit button: element with id 'launch_deployment'. (//*[normalize-space(@id)='launch_deployment'])
FULL NAME OF INSTANCE SHOULD BE: Deployment-ConfigSrv-dd8h3/testMock
instance name=Deployment-ConfigSrv-dd8h3/testMock
Click on link: 'Monitor'. (//a[normalize-space(.)='Monitor'])
Click on link: element with id 'filter_view'. (//*[normalize-space(@id)='filter_view'])
Wait for element 'element with id 'details_instances'. (//*[normalize-space(@id)='details_instances'])', with timeout of 60000.
Click on link: element with id 'details_instances'. (//*[normalize-space(@id)='details_instances'])
Wait for element 'element with id 'stop_selected_instances'. (//*[normalize-space(@id)='stop_selected_instances'])', with timeout of 60000.
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 20
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 19
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 18
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 17
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 16
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 15
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 14
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 13
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 12
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 11
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 10
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 9
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 8
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 7
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 6
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 5
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 4
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 3
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 2
Sleep for 30000ms.
already on the correct page
Deployment-ConfigSrv-dd8h3/testMock is in a NEW state
Remaining available checks = 1

[root@qeblade31 ~]# rpm -qa | grep aeolus
aeolus-conductor-daemons-0.6.0-0.20111114172747gitde62906.fc15.noarch
aeolus-configure-2.3.0-0.20111111160331git9bde372.fc15.noarch
aeolus-all-0.6.0-0.20111114172747gitde62906.fc15.noarch
aeolus-conductor-0.6.0-0.20111114172747gitde62906.fc15.noarch
aeolus-conductor-devel-0.6.0-0.20111114172747gitde62906.fc15.noarch
rubygem-aeolus-cli-0.1.0-3.20111115145311git5ec94c5.fc15.noarch
rubygem-aeolus-image-0.1.0-4.20111024205454git6b2b696.fc15.noarch
aeolus-conductor-doc-0.6.0-0.20111114172747gitde62906.fc15.noarch
[root@qeblade31 ~]#

Comment 2 Greg Blomquist 2011-11-16 20:23:45 UTC
I cannot reproduce this exact situation.

I can reproduce this under these specific circumstances:

  1) my deployable has more than one assembly, AND
  2) each assembly in the deployable has a unique image ID.

If I attempt to deploy something where either of those conditions are _not_ true, then it works as expected.

Can you retest from a clean start with the latest conductor RPMs from the conductor testing yum repo?

I'll try to keep testing as well to see if there's anything I'm missing.

Comment 3 dgao 2011-11-18 18:58:22 UTC
No longer seeing this issue w/ the latest build. Mark as verified.

Comment 4 wes hayutin 2011-11-28 01:17:13 UTC
removing bugs from ce-sprint from the tracker.. you can find these bugs by querying the "qa whiteboard" for ce-sprint-60

Comment 5 James Laska 2012-02-10 14:09:49 UTC
Reassigning to proper 'aeolus-audrey-agent' component.


Note You need to log in before you can comment on or make changes to this bug.