Bug 785028

Summary: package group installs need to handle whitespaces
Product: [Retired] CloudForms Cloud Engine Reporter: Dave Johnson <dajohnso>
Component: imagefactoryAssignee: Ian McLeod <imcleod>
Status: CLOSED CURRENTRELEASE QA Contact: Martin Kočí <mkoci>
Severity: high Docs Contact:
Priority: medium    
Version: 1.0.0CC: akarol, brad, dajohnso, deltacloud-maint, dgao, ssachdev, whayutin
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-12-13 19:50:10 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Dave Johnson 2012-01-27 00:11:49 UTC
Description of problem:

Tried to build a image with a bunch of groups and found that group names with spaces are not quoted when used by the imagefactory code.

  <packages>
    <package name="@Eclipse"/>
    <package name="@Office Suite and Productivity"/>
    <package name="@X Window System"/>
    <package name="@MySQL Database server"/>
  </packages>

Translates to:
    'yum -y install @Eclipse @Office Suite and Productivity @X Window System @MySQL Database server '


imagefactory.log snippet:
--------------------------------
2012-01-26 16:40:19,214 DEBUG imgfac.builders.BaseBuilder.RHEL6_rhevm_Builder pid(10444) Message: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_rhevm_Builder.py", line 65, in build_image
    self.build_upload(build_id)
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_rhevm_Builder.py", line 116, in build_upload
    self.output_descriptor = guest.customize_and_generate_icicle(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1177, in customize_and_generate_icicle
    return self._internal_customize(libvirt_xml, True)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1153, in _internal_customize
    self.do_customize(guestaddr)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1109, in do_customize
    tunnels=self.tunnels)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 476, in guest_execute_command
    command, timeout, tunnels)
  File "/usr/lib/python2.6/site-packages/oz/ozutil.py", line 355, in ssh_execute_command
    return subprocess_check_output(cmd)
  File "/usr/lib/python2.6/site-packages/oz/ozutil.py", line 323, in subprocess_check_output
    raise SubprocessException("'%s' failed(%d): %s" % (cmd, retcode, stderr), retcode)
SubprocessException: 'ssh -i /etc/oz/id_rsa-icicle-gen -F /dev/null -o ServerAliveInterval=30 -o StrictHostKeyChecking=no -o ConnectTimeout=10 -o UserKnownHostsFile=/dev/null -o PasswordAuthentication=no root.122.130 yum -y install @Eclipse @Office Suite and Productivity @X Window System @MySQL Database server ' failed(1): Warning: Permanently added '192.168.122.130' (RSA) to the list of known hosts.
Error: No Groups Available in any repository

2012-01-26 16:40:19,214 DEBUG imgfac.BuildJob.BuildJob pid(10444) Message: Builder (f5e37f2a-135a-44f9-985c-9b8ac3fe9c36) changed status from BUILDING to FAILED





Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 1 Dave Johnson 2012-01-27 01:45:23 UTC
Okay, so the stack trace is not related...  Apparently the repo locations changed. I updated them and it seems to be working however....

This does not work:
  [root@hp-xw9300-01 ~]# yum install @Office Suite and Productivity
  Loaded plugins: product-id, rhnplugin, security, subscription-manager
  Updating certificate-based repositories.
  Setting up Install Process
  No package Suite available.
  No package and available.
  No package Productivity available.
  Nothing to do

So I doubt this is going to which was shown in the log:
    'yum -y install @Eclipse @Office Suite and Productivity @X Window System
@MySQL Database server '

This does work so just thinking that imagefactory probably needs to injects some single quotes or something....

  [root@hp-xw9300-01 ~]# yum install @'Office Suite and Productivity'
  Loaded plugins: product-id, rhnplugin, security, subscription-manager
  Updating certificate-based repositories.
  Setting up Install Process
  Resolving Dependencies
  --> Running transaction check
  ---> Package openoffice.org-calc.x86_64 1:3.2.1-19.6.el6_0.5 will be installed

Comment 2 jrd 2012-01-27 15:35:07 UTC
I'm this is sufficiently serious to warrant fixing for 1.0.  Wes, Mitch, Hugh, dissent?

Assigning to Ian for difficulty/risk assessment.

Comment 3 wes hayutin 2012-01-27 16:12:28 UTC
This is in fact a regression..  when the first beta went out w/ .3 installing package groups worked..

This is a must have

Comment 4 Ian McLeod 2012-01-27 18:23:11 UTC
For the record, package groups with spaces in them have never worked.  Package groups always have an <id> handle that is space free.  For example:

<group>
<id>mysql</id>
<name>MySQL Database server</name>

If you use this name with the "@" notation, it will work.

Having said this, there's no reason to deny users the convenience of using the space-allowed <name> field.  A fix for this is here:

https://github.com/aeolusproject/oz/commit/043d58252ee2315d14074b27f6927343e142ea22

I've tested it against dajo's original template and it works.

It has been brewed as oz-0.8.0-4.

Comment 5 Martin Kočí 2012-01-30 18:57:00 UTC
Hi, 
test didn't passed for me with package 

#rpm -qa |grep 'aeolus\|imagefactory-\|oz-\|iwhd'
iwhd-1.1-2.el6.x86_64
oz-0.9.0-0.20120127190146git043d582.el6.noarch
aeolus-conductor-doc-0.9.0-0.20120118181603git71cd8bc.el6.noarch
aeolus-all-0.9.0-0.20120118181603git71cd8bc.el6.noarch
rubygem-imagefactory-console-0.5.0-4.20110824113238gitd9debef.el6.noarch
aeolus-conductor-daemons-0.9.0-0.20120118181603git71cd8bc.el6.noarch
imagefactory-jeosconf-ec2-fedora-1.0.0rc3_5_g1041f75-1.el6.noarch
rubygem-arel-2.0.10-0.aeolus.el6.noarch
rubygem-rack-mount-0.7.1-3.aeolus.el6.noarch
rubygem-ZenTest-4.3.3-2.aeolus.el6.noarch
rubygem-aeolus-image-0.4.0-0.20120118121635git0d31a37.el6.noarch
rubygem-aeolus-cli-0.4.0-0.20120118121626git6fddd65.el6.noarch
aeolus-configure-2.6.0-0.20120118121620gita996371.el6.noarch
imagefactory-jeosconf-ec2-rhel-1.0.0rc3_5_g1041f75-1.el6.noarch
aeolus-conductor-0.9.0-0.20120118181603git71cd8bc.el6.noarch
imagefactory-1.0.0rc3_5_g1041f75-1.el6.noarch

with error:
2012-01-30 13:39:23,422 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(12298) Message: Exception caught in ImageFactory
2012-01-30 13:39:23,422 DEBUG imgfac.builders.BaseBuilder.RHEL6_vsphere_Builder pid(12298) Message: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_vsphere_Builder.py", line 49, in build_image
    self.build_upload(build_id)
  File "/usr/lib/python2.6/site-packages/imgfac/builders/Fedora_vsphere_Builder.py", line 115, in build_upload
    self.output_descriptor = self.guest.customize_and_generate_icicle(libvirt_xml)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1178, in customize_and_generate_icicle
    return self._internal_customize(libvirt_xml, True)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1154, in _internal_customize
    self.do_customize(guestaddr)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 1110, in do_customize
    tunnels=self.tunnels)
  File "/usr/lib/python2.6/site-packages/oz/RedHat.py", line 476, in guest_execute_command
    command, timeout, tunnels)
  File "/usr/lib/python2.6/site-packages/oz/ozutil.py", line 355, in ssh_execute_command
    return subprocess_check_output(cmd)
  File "/usr/lib/python2.6/site-packages/oz/ozutil.py", line 323, in subprocess_check_output
    raise SubprocessException("'%s' failed(%d): %s" % (cmd, retcode, stderr), retcode)
SubprocessException: 'ssh -i /etc/oz/id_rsa-icicle-gen -F /dev/null -o ServerAliveInterval=30 -o StrictHostKeyChecking=no -o ConnectTimeout=10 -o UserKnownHostsFile=/dev/null -o PasswordAuthentication=no root.122.4 yum -y install "@Eclipse" "@Office Suite and Productivity" "@X Window System" "@MySQL Database server" ' failed(1): Warning: Permanently added '192.168.122.4' (RSA) to the list of known hosts.
Error: No Groups Available in any repository

If I try the command below in the command line on the server it installs everything without issues. 
#yum -y install "@Eclipse" "@Office Suite and Productivity" "@X Window System" "@MySQL Database server"


For further info see :
http://hudson.rhq.lab.eng.bos.redhat.com:8080/hudson/view/DEV-CloudForms/job/ImageFactory-KocaTesting2/244/console

Comment 6 Ian McLeod 2012-01-31 15:28:18 UTC
Martin,

This error:

"Error: No Groups Available in any repository"

Suggests that the repos listed in the TDL file do not actually contain package groups.  Can you share the full TDL you used to generate this error?

Comment 7 Martin Kočí 2012-01-31 17:40:06 UTC
Ian, you right! I'm sorry, shame on me, I have had typo in the repository path.. After I have fixed that, it builds without issues.

Moving bug to VERIFIED with packages: 
rpm -qa | grep 'aeolus\|imagefactory-\|oz-\|iwhd'
iwhd-1.1-2.el6.x86_64
oz-0.9.0-0.20120127190146git043d582.el6.noarch
aeolus-conductor-doc-0.9.0-0.20120118181603git71cd8bc.el6.noarch
aeolus-all-0.9.0-0.20120118181603git71cd8bc.el6.noarch
rubygem-imagefactory-console-0.5.0-4.20110824113238gitd9debef.el6.noarch
aeolus-conductor-daemons-0.9.0-0.20120118181603git71cd8bc.el6.noarch
imagefactory-jeosconf-ec2-fedora-1.0.0rc3_5_g1041f75-1.el6.noarch
rubygem-arel-2.0.10-0.aeolus.el6.noarch
rubygem-rack-mount-0.7.1-3.aeolus.el6.noarch
rubygem-ZenTest-4.3.3-2.aeolus.el6.noarch
rubygem-aeolus-image-0.4.0-0.20120118121635git0d31a37.el6.noarch
rubygem-aeolus-cli-0.4.0-0.20120118121626git6fddd65.el6.noarch
aeolus-configure-2.6.0-0.20120118121620gita996371.el6.noarch
imagefactory-jeosconf-ec2-rhel-1.0.0rc3_5_g1041f75-1.el6.noarch
aeolus-conductor-0.9.0-0.20120118181603git71cd8bc.el6.noarch
imagefactory-1.0.0rc3_5_g1041f75-1.el6.noarch