rhel-osp-director: The time isn't synced on all overcloud nodes, deployed with the --ntp-server argument. Environment: instack-undercloud-2.1.2-29.el7ost.noarch openstack-heat-common-2015.1.1-5.el7ost.noarch Steps to reproduce: Attempt to deploy overcloud on nodes with skewed time, using the --ntp-server arg. Result: The time isn't synced with the ntp server. Expected result: The time on all nodes should be synced with the ntp server specified with --ntp-server. Notes: Runing ntpdate <server> manually on the nodes adjusts the time properly, so it seems like the command isn't triggered. Also, Looking at /usr/libexec/os-refresh-config/configure.d/50-ntp that's mentioned in the os-collect-config logs: NTP_SERVER="$(os-apply-config --key 'ntp.servers.0.server' --type raw --key-default '')" if [ -n "${NTP_SERVER}" ]; then Running "os-apply-config --key 'ntp.servers.0.server' --type raw --key-default ''" results in empty string, so the condition isn't true.
Sasha, can you please provide puppet logs? Thanks
Closing for now, if you reproduce, please reopen.
I have the same issue on an OSP-d 7.1 deployment Environment: instack-undercloud-2.1.2-29.el7ost.noarch openstack-puppet-modules-2015.1.8-21.el7ost.noarch openstack overcloud deploy --templates ~/templates/ -e ~/env.yaml -e ~/templates/environments/storage-environment.yaml -e ~/templates/environments/network-isolation.yaml --control-scale 3 --control-flavor control --compute-scale 1 --compute-flavor compute --ceph-storage-scale 1 --ceph-storage-flavor ceph-storage --swift-storage-scale 0 --swift-storage-flavor swift-storage --block-storage-scale 0 --neutron-network-type vxlan --neutron-tunnel-types vxlan --neutron-vni-ranges 1:10000 --neutron-bridge-mappings datacentre:br-bond --libvirt-type kvm --ntp-server pool.ntp.org -t 120 On overcloud side the value in hiera is right : $ sudo hiera ntp::servers ["pool.ntp.org"] But the ntp configuration is not apply with this value. $ sudo grep ^server /etc/ntp.conf server 127.127.1.0 # local clock address In the puppet logs we can see the changes but there are not applied on each puppet run (or maybe something rollback the changes after) $ heat deployment-output-show <physical_resource_id> deploy_stdout|grep -i ntp Notice: /Stage[main]/Ntp::Config/File[/etc/ntp.conf]/content: content changed '{md5}c07b9a377faea45b96b7d3bf8976004b' to '{md5}7044a259392968d8dbe468670fd9c98c' Notice: /File[/etc/ntp.conf]/seltype: seltype changed 'etc_t' to 'net_conf_t' Notice: /Stage[main]/Ntp::Service/Service[ntp]/ensure: ensure changed 'stopped' to 'running' Running the puppet command by hand only for ntp : $ sudo FACTER_heat_outputs_path="/var/run/heat-config/heat-config-puppet/1ac2a3a5-0b2a-4ab7-aea2-13379c20b3d0" FACTER_fqdn="ctrl00-prv.localdomain" FACTER_deploy_config_name="overcloud-ControllerNodesPostDeployment-l2fejf76ersk-ControllerPuppetConfig-bcgn5k5qg3vt-ControllerPuppetConfigImpl-4p6ejshk2opq" puppet apply --detailed-exitcodes /var/lib/heat-config/heat-config-puppet/1ac2a3a5-0b2a-4ab7-aea2-13379c20b3d0.pp --tags Ntp Warning: Scope(Class[Keystone]): Execution of db_sync does not depend on $enabled anymore. Please use sync_db instead. Warning: Scope(Class[Glance::Registry]): Execution of db_sync does not depend on $manage_service or $enabled anymore. Please use sync_db instead. Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_host'; class ::nova::compute has not been evaluated Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_protocol'; class ::nova::compute has not been evaluated Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_port'; class ::nova::compute has not been evaluated Warning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_path'; class ::nova::compute has not been evaluated Warning: Scope(Class[Concat::Setup]): concat::setup is deprecated as a public API of the concat module and should no longer be directly included in the manifest. Notice: Compiled catalog for ctrl00-prv.localdomain in environment production in 13.35 seconds Notice: /Stage[main]/Ntp::Config/File[/etc/ntp.conf]/content: content changed '{md5}c07b9a377faea45b96b7d3bf8976004b' to '{md5}7044a259392968d8dbe468670fd9c98c' Notice: /File[/etc/ntp.conf]/seltype: seltype changed 'etc_t' to 'net_conf_t' Notice: /Stage[main]/Ntp::Service/Service[ntp]: Triggered 'refresh' from 1 events Notice: Finished catalog run in 7.80 seconds $ sudo grep ^server /etc/ntp.conf server pool.ntp.org
Please retest this on OSP 7.2 which is now released.
Exactly the same issue with OSP-d 7.2 Environment: instack-undercloud-2.1.2-36.el7ost.noarch openstack-tripleo-heat-templates-0.8.6-94.el7ost.noarch openstack-puppet-modules-2015.1.8-32.el7ost.noarch
Verified: Environment: instack-undercloud-2.2.0-1.el7ost.noarch openstack-heat-common-5.0.0-1.el7ost.noarch The reported issue doesn't reproduce.
The bug doesn't seem to be present in 7.3 : # grep ^server /etc/ntp.conf server pool.ntp.org The big difference between 7.{0,1,2} and 7.3 is the file 50-ntp in /usr/libexec/os-refresh-config/configure.d/ # in 7.0/7.1/7.2 # ls -lh /usr/libexec/os-refresh-config/configure.d/ total 36K -rwxr-xr-x. 1 root root 396 Dec 16 15:08 10-sysctl-apply-config -rwxr-xr-x. 1 root root 42 Dec 16 15:08 20-os-apply-config -rwxr-xr-x. 1 root root 1.6K Dec 16 15:08 20-os-net-config -rwxr-xr-x. 1 root root 629 Dec 16 15:08 25-set-network-gateway -rwxr-xr-x. 1 root root 2.8K Dec 16 15:08 40-hiera-datafiles -rwxr-xr-x. 1 root root 969 Dec 16 15:08 50-ntp -rwxr-xr-x. 1 root root 1.4K Dec 16 15:08 51-hosts -rwxr-xr-x. 1 root root 6.1K Dec 16 15:08 55-heat-config # in 7.3 # ls -lh /usr/libexec/os-refresh-config/configure.d/ total 32K -rwxr-xr-x. 1 root root 396 Feb 16 22:20 10-sysctl-apply-config -rwxr-xr-x. 1 root root 42 Feb 16 22:20 20-os-apply-config -rwxr-xr-x. 1 root root 1.6K Feb 16 22:20 20-os-net-config -rwxr-xr-x. 1 root root 629 Feb 16 22:20 25-set-network-gateway -rwxr-xr-x. 1 root root 2.8K Feb 16 22:20 40-hiera-datafiles -rwxr-xr-x. 1 root root 1.4K Feb 16 22:20 51-hosts -rwxr-xr-x. 1 root root 6.1K Feb 16 22:20 55-heat-config But the problem is still present if we upgrade from 7.x to 7.3 because the file 50-ntp is not deleted. The ntp server is configured during the puppet part then erased by the resource 50-ntp Mar 04 11:56:37 sc-ctrl00.localdomain os-collect-config[3696]: Notice: /Stage[main]/Ntp::Config/File[/etc/ntp.conf]/content: content changed '{md5}c07b9a377faea45b96b7d3bf8976004b' to '{md5}7044a259392968d8dbe468670fd9c98c' Mar 04 11:56:37 sc-ctrl00.localdomain os-collect-config[3696]: Notice: /File[/etc/ntp.conf]/seltype: seltype changed 'etc_t' to 'net_conf_t' Mar 04 11:56:37 sc-ctrl00.localdomain os-collect-config[3696]: Notice: /Stage[main]/Ntp::Service/Service[ntp]: Triggered 'refresh' from 1 events # grep ^server /etc/ntp.conf server pool.ntp.org Mar 04 16:57:23 sc-ctrl00.localdomain os-collect-config[3696]: [2016/03/04 04:57:23 PM] [INFO] writing /etc/ntp.conf Mar 04 16:57:47 sc-ctrl00.localdomain os-collect-config[3696]: dib-run-parts Fri Mar 4 16:57:47 UTC 2016 Running /usr/libexec/os-refresh-config/configure.d/50-ntp Mar 04 16:57:47 sc-ctrl00.localdomain os-collect-config[3696]: dib-run-parts Fri Mar 4 16:57:47 UTC 2016 50-ntp completed # grep ^server /etc/ntp.conf server 127.127.1.0 # local clock address
Any update for a fix ? I have the same issue than Dimitri when I'm upgrading from 7.1 to 7.3. Thanks.
There is additional discusson on this issue in https://bugzilla.redhat.com/show_bug.cgi?id=1296365 Since the goal is to have Puppet configure and maintain ntp.conf, not os-config (and certainly not both as was the case), the configuration script was removed in the OSP 7.3 overcloud-full image. The issue is that the images are only used in new deployments, or scale ups, so right now they are saying the solution may just be manually removing /usr/libexec/os-refresh-config/configure.d/50-ntp
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-0604.html