+++ This bug was initially created as a clone of Bug #1109079 +++

Description of problem:

I hit another installation issue while retesting bug 1108005:

10.16.67.27_swift.pp:                             [ ERROR ]         
Applying Puppet manifests                         [ ERROR ]

ERROR : Error appeared during Puppet run: 10.16.67.27_swift.pp
Error: /Stage[main]//Swift::Storage::Loopback[swift_loopback]/Swift::Storage::Ext4[swift_loopback]/Swift::Storage::Mount[swift_loopback]/Exec[restorecon_mount_swift_loopback]: Failed to call refresh: Could not find command 'restorecon'


Note: restorecon path on el6 (/sbin/restorecon) is different from el7 (/usr/sbin/restorecon).

Version-Release number of selected component (if applicable):
openstack-packstack-2014.1.1-0.23.dev1120.el6ost.noarch
openstack-puppet-modules-2014.1-14.1.el6ost.noarch
openstack-packstack-puppet-2014.1.1-0.23.dev1120.el6ost.noarch

How reproducible:
100%

Steps to Reproduce:
1. yum -y install http://team.virt.bos.redhat.com/repos/rhos-release/rhos-release-latest.noarch.rpm
2. rhos-release 5
3. yum -y install openstack-packstack
4. upgrade packstack packages to -2014.1.1-0.24.dev1120.el6ost
5. setenforce 0
6. packstack --allinone

Actual results:
Packstack installation fail

Expected results:
Packstack installation succeeds

Additional info:

# tail -n 25 /var/tmp/packstack/20140612-113454-1e5A5F/openstack-setup.log
cd /usr/share/openstack-puppet/modules
tar --dereference -cpzf - apache ceilometer certmonger cinder concat firewall glance heat horizon inifile keystone memcached mongodb mysql neutron nova nssdb openstack packstack qpid rabbitmq rsync ssh stdlib swift sysctl tempest vcsrepo vlan vswitch xinetd | ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root.67.27 tar -C /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules -xpzf -
2014-06-12 11:50:50::ERROR::run_setup::920::root:: Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/packstack/installer/run_setup.py", line 901, in main
    single_step_aio_install(options)
  File "/usr/lib/python2.6/site-packages/packstack/installer/run_setup.py", line 688, in single_step_aio_install
    single_step_install(options)
  File "/usr/lib/python2.6/site-packages/packstack/installer/run_setup.py", line 718, in single_step_install
    _main(answerfilepath)
  File "/usr/lib/python2.6/site-packages/packstack/installer/run_setup.py", line 605, in _main
    runSequences()
  File "/usr/lib/python2.6/site-packages/packstack/installer/run_setup.py", line 584, in runSequences
    controller.runAllSequences()
  File "/usr/lib/python2.6/site-packages/packstack/installer/setup_controller.py", line 68, in runAllSequences
    sequence.run(config=self.CONF, messages=self.MESSAGES)
  File "/usr/lib/python2.6/site-packages/packstack/installer/core/sequences.py", line 98, in run
    step.run(config=config, messages=messages)
  File "/usr/lib/python2.6/site-packages/packstack/installer/core/sequences.py", line 44, in run
    raise SequenceError(str(ex))
SequenceError: Error appeared during Puppet run: 10.16.67.27_swift.pp
Error: /Stage[main]//Swift::Storage::Loopback[swift_loopback]/Swift::Storage::Ext4[swift_loopback]/Swift::Storage::Mount[swift_loopback]/Exec[restorecon_mount_swift_loopback]: Failed to call refresh: Could not find command 'restorecon'
You will find full trace in log /var/tmp/packstack/20140612-113454-1e5A5F/manifests/10.16.67.27_swift.pp.log

2014-06-12 11:50:50::INFO::shell::81::root:: [10.16.67.27] Executing script:
rm -rf /var/tmp/packstack/7cf239cda58148b5b72301b25600509c


# cat /var/tmp/packstack/20140612-113454-1e5A5F/manifests/10.16.67.27_swift.pp.log
Warning: Config file /etc/puppet/hiera.yaml not found, using Hiera defaults
Warning: Variable access via 'tcp_port' is deprecated. Use '@tcp_port' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/memcached/templates/memcached_sysconfig.erb]:1
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/memcached/templates/memcached_sysconfig.erb:1:in `result')
Warning: Variable access via 'user' is deprecated. Use '@user' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/memcached/templates/memcached_sysconfig.erb]:2
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/memcached/templates/memcached_sysconfig.erb:2:in `result')
Warning: Variable access via 'max_connections' is deprecated. Use '@max_connections' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/memcached/templates/memcached_sysconfig.erb]:3
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/memcached/templates/memcached_sysconfig.erb:3:in `result')
Warning: Variable access via 'max_memory' is deprecated. Use '@max_memory' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/memcached/templates/memcached_sysconfig.erb]:5
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/memcached/templates/memcached_sysconfig.erb:5:in `result')
Warning: Variable access via 'listen_ip' is deprecated. Use '@listen_ip' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/memcached/templates/memcached_sysconfig.erb]:15
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/memcached/templates/memcached_sysconfig.erb:15:in `result')
Warning: Variable access via 'udp_port' is deprecated. Use '@udp_port' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/memcached/templates/memcached_sysconfig.erb]:18
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/memcached/templates/memcached_sysconfig.erb:18:in `result')
Warning: Variable access via 'processorcount' is deprecated. Use '@processorcount' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/memcached/templates/memcached_sysconfig.erb]:20
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/memcached/templates/memcached_sysconfig.erb:20:in `result')
Warning: Variable access via 'use_chroot' is deprecated. Use '@use_chroot' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/header.erb]:7
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/header.erb:7:in `result')
Warning: Variable access via 'address' is deprecated. Use '@address' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/header.erb]:11
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/header.erb:11:in `result')
Warning: Variable access via 'motd_file' is deprecated. Use '@motd_file' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/header.erb]:12
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/header.erb:12:in `result')
Warning: Variable access via 'name' is deprecated. Use '@name' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:4
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:4:in `result')
Warning: Variable access via 'port' is deprecated. Use '@port' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:6
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:6:in `result')
Warning: Variable access via 'disable' is deprecated. Use '@disable' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:7
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:7:in `result')
Warning: Variable access via 'socket_type' is deprecated. Use '@socket_type' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:8
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:8:in `result')
Warning: Variable access via 'protocol' is deprecated. Use '@protocol' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:9
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:9:in `result')
Warning: Variable access via 'mywait' is deprecated. Use '@mywait' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:10
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:10:in `result')
Warning: Variable access via 'user' is deprecated. Use '@user' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:11
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:11:in `result')
Warning: Variable access via 'group' is deprecated. Use '@group' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:12
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:12:in `result')
Warning: Variable access via 'server' is deprecated. Use '@server' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:13
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:13:in `result')
Warning: Variable access via 'bind' is deprecated. Use '@bind' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:14
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:14:in `result')
Warning: Variable access via 'server_args' is deprecated. Use '@server_args' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:15
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:15:in `result')
Warning: Variable access via 'per_source' is deprecated. Use '@per_source' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:16
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:16:in `result')
Warning: Variable access via 'log_on_failure' is deprecated. Use '@log_on_failure' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:17
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:17:in `result')
Warning: Variable access via 'cps' is deprecated. Use '@cps' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:18
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:18:in `result')
Warning: Variable access via 'flags' is deprecated. Use '@flags' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:19
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:19:in `result')
Warning: Variable access via 'service_type' is deprecated. Use '@service_type' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb]:20
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/xinetd/templates/service.erb:20:in `result')
Warning: Scope(Swift::Storage::Server[6002]): swift storage server account must specify account-server
Warning: Scope(Swift::Storage::Server[6001]): swift storage server container must specify container-server
Warning: Scope(Swift::Storage::Server[6000]): swift storage server object must specify object-server
Warning: Variable access via 'name' is deprecated. Use '@name' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:4
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:4:in `result')
Warning: Variable access via 'path' is deprecated. Use '@path' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:5
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:5:in `result')
Warning: Variable access via 'read_only' is deprecated. Use '@read_only' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:6
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:6:in `result')
Warning: Variable access via 'write_only' is deprecated. Use '@write_only' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:7
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:7:in `result')
Warning: Variable access via 'list' is deprecated. Use '@list' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:8
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:8:in `result')
Warning: Variable access via 'uid' is deprecated. Use '@uid' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:9
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:9:in `result')
Warning: Variable access via 'gid' is deprecated. Use '@gid' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:10
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:10:in `result')
Warning: Variable access via 'incoming_chmod' is deprecated. Use '@incoming_chmod' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:11
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:11:in `result')
Warning: Variable access via 'outgoing_chmod' is deprecated. Use '@outgoing_chmod' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:12
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:12:in `result')
Warning: Variable access via 'max_connections' is deprecated. Use '@max_connections' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:13
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:13:in `result')
Warning: Variable access via 'max_connections' is deprecated. Use '@max_connections' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:14
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:14:in `result')
Warning: Variable access via 'comment' is deprecated. Use '@comment' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:15
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:15:in `result')
Warning: Variable access via 'secrets_file' is deprecated. Use '@secrets_file' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:16
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:16:in `result')
Warning: Variable access via 'auth_users' is deprecated. Use '@auth_users' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:17
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:17:in `result')
Warning: Variable access via 'hosts_allow' is deprecated. Use '@hosts_allow' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:18
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:18:in `result')
Warning: Variable access via 'hosts_deny' is deprecated. Use '@hosts_deny' instead. template[/var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb]:19
   (at /var/tmp/packstack/7cf239cda58148b5b72301b25600509c/modules/rsync/templates/module.erb:19:in `result')
Notice: Compiled catalog for hp-bl420cgen8-01.rhts.eng.bos.redhat.com in environment production in 1.59 seconds
Notice: /Firewall[001 swift proxy incoming]/ensure: created
Notice: /Firewall[001 swift storage and rsync incoming swift_storage_and_rsync_10.16.67.27]/ensure: created
Notice: /File[/srv/node]/ensure: created
Notice: /File[/srv/loopback-device]/ensure: created
Notice: /Stage[main]//Swift::Storage::Loopback[swift_loopback]/Exec[create_partition-swift_loopback]/returns: executed successfully
Notice: /Stage[main]//Swift::Storage::Loopback[swift_loopback]/Swift::Storage::Ext4[swift_loopback]/Exec[mkfs-swift_loopback]: Triggered 'refresh' from 1 events
Notice: /File[/srv/node/swift_loopback]/ensure: created
Notice: /Stage[main]//Swift::Storage::Loopback[swift_loopback]/Swift::Storage::Ext4[swift_loopback]/Swift::Storage::Mount[swift_loopback]/Mount[/srv/node/swift_loopback]/ensure: defined 'ensure' as 'defined'
Notice: /Stage[main]//Swift::Storage::Loopback[swift_loopback]/Swift::Storage::Ext4[swift_loopback]/Swift::Storage::Mount[swift_loopback]/Mount[/srv/node/swift_loopback]: Triggered 'refresh' from 2 events
Notice: /Stage[main]//Swift::Storage::Loopback[swift_loopback]/Swift::Storage::Ext4[swift_loopback]/Swift::Storage::Mount[swift_loopback]/Exec[mount_swift_loopback]/returns: executed successfully
Notice: /Stage[main]//Swift::Storage::Loopback[swift_loopback]/Swift::Storage::Ext4[swift_loopback]/Swift::Storage::Mount[swift_loopback]/Exec[mount_swift_loopback]: Triggered 'refresh' from 1 events
Error: /Stage[main]//Swift::Storage::Loopback[swift_loopback]/Swift::Storage::Ext4[swift_loopback]/Swift::Storage::Mount[swift_loopback]/Exec[restorecon_mount_swift_loopback]: Failed to call refresh: Could not find command 'restorecon'
Error: /Stage[main]//Swift::Storage::Loopback[swift_loopback]/Swift::Storage::Ext4[swift_loopback]/Swift::Storage::Mount[swift_loopback]/Exec[restorecon_mount_swift_loopback]: Could not find command 'restorecon'

--- Additional comment from Petr Matousek on 2014-06-16 08:13:32 EDT ---

Note: The issue is fixed, installation completed successfully, suggesting to move to verified.

packages under tests:
openstack-packstack-puppet-2014.1.1-0.24.dev1120.el6ost.noarch
openstack-packstack-2014.1.1-0.24.dev1120.el6ost.noarch
openstack-puppet-modules-2014.1-14.4.el6ost.noarch

--- Additional comment from  on 2014-06-20 09:03:54 EDT ---

I am getting a similar error using packstack --allinone, how were you able to resolve this? Have all of the latest updates.

Thanks!

10.0.0.213_swift.pp:                              [ ERROR ]
Applying Puppet manifests                         [ ERROR ]

ERROR : Error appeared during Puppet run: 10.0.0.213_swift.pp
Error: /Stage[main]/Main/Swift::Storage::Loopback[swift_loopback]/Swift::Storage::Ext4[swift_loopback]/Swift::Storage::Mount[swift_loopback]/Exec[restorecon_mount_swift_loopback]: Failed to call refresh: Could not find command 'restorecon'
You will find full trace in log /var/tmp/packstack/20140619-164551-2eLjEN/manifests/10.0.0.213_swift.pp.log
Please check log file /var/tmp/packstack/20140619-164551-2eLjEN/openstack-setup.log for more information

--- Additional comment from Lukas Bezdicka on 2014-06-20 09:20:38 EDT ---

Hi, I guess you are reporting RDO issue, if this is only thing broken for you, you might workaround it by changing line in /usr/share/openstack-puppet/modules/swift/manifests/storage/mount.pp:68

      path        => ['/usr/sbin', '/bin'],

to
      path        => ['/usr/sbin', '/sbin'],

And as for the package it should be hopefully out soon.


(In reply to termina from comment #2)
> I am getting a similar error using packstack --allinone, how were you able
> to resolve this? Have all of the latest updates.
> 
> Thanks!
> 
> 10.0.0.213_swift.pp:                              [ ERROR ]
> Applying Puppet manifests                         [ ERROR ]
> 
> ERROR : Error appeared during Puppet run: 10.0.0.213_swift.pp
> Error:
> /Stage[main]/Main/Swift::Storage::Loopback[swift_loopback]/Swift::Storage::
> Ext4[swift_loopback]/Swift::Storage::Mount[swift_loopback]/
> Exec[restorecon_mount_swift_loopback]: Failed to call refresh: Could not
> find command 'restorecon'
> You will find full trace in log
> /var/tmp/packstack/20140619-164551-2eLjEN/manifests/10.0.0.213_swift.pp.log
> Please check log file
> /var/tmp/packstack/20140619-164551-2eLjEN/openstack-setup.log for more
> information