Bug 1459976
| Summary: | openstack-nova: repeating errors on compute: Jun 8 05:33:38 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Alexander Chuzhoy <sasha> | ||||
| Component: | openstack-tripleo-heat-templates | Assignee: | Pradeep Kilambi <pkilambi> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Sasha Smolyak <ssmolyak> | ||||
| Severity: | medium | Docs Contact: | |||||
| Priority: | medium | ||||||
| Version: | 12.0 (Pike) | CC: | berrange, dasmith, eglynn, jdanjou, jruzicka, kchamart, mburns, pkilambi, rhel-osp-director-maint, sasha, sbauza, scorcora, sferdjao, sgordon, srevivo, svanders, tvignaud, vromanso | ||||
| Target Milestone: | Upstream M2 | Keywords: | Triaged | ||||
| Target Release: | 12.0 (Pike) | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | openstack-tripleo-heat-templates-7.0.0-0.20170624014919.el7ost | Doc Type: | If docs needed, set a value | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2017-12-13 21:31:55 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
I just want to clarify, does the overcloud deploy fail? Or does everything work as expected, except for the presence of those errors in the logs? The overcloud deploy succeeded, but not everything works, here's a list of issues reported upon successful deployment: https://bugzilla.redhat.com/show_bug.cgi?id=1457358 https://bugzilla.redhat.com/show_bug.cgi?id=1457421 https://bugzilla.redhat.com/show_bug.cgi?id=1459592 What are the used environment files in /home/stack/virt/ ? Created attachment 1287092 [details]
env files used in deployment
seems to be a ceilometer-agent problem. should be fixed upstream by https://review.openstack.org/#/c/469935/ moving to openstack-ceilometer Failed_QA Environment: openstack-tripleo-heat-templates-7.0.0-0.20170628002128.el7ost.noarch The reported issue still reproduces. Is this the same ceilometer-compute-agent container issue thats failing? Yes, logs on ceilometer_agent_compute: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': Connection refused I just did a fresh deploy of tripleo with containers and on compute nodes i see compute agent running without errors: 2017-07-06 19:56:18Z [overcloud]: CREATE_COMPLETE Stack CREATE completed successfully Stack overcloud CREATE_COMPLETE Overcloud Endpoint: http://192.168.24.7:5000/v2.0 Overcloud Deployed (undercloud) [stack@undercloud ~]$ rpm -qa|grep heat-templates openstack-tripleo-heat-templates-7.0.0-0.20170706121721.25caba5.el7.centos.noarch [root@overcloud-novacompute-0 ~]# docker ps -a |grep ceilometer b91b783b48b0 192.168.24.1:8787/tripleoupstream/centos-binary-ceilometer-compute:latest "kolla_start" 29 minutes ago Up 29 minutes ceilometer_agent_compute [root@overcloud-novacompute-0 ~]# docker logs ceilometer_agent_compute INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json INFO:__main__:Validating config file INFO:__main__:Kolla config strategy set to: COPY_ALWAYS INFO:__main__:Writing out command to execute Running command: '/usr/bin/ceilometer-polling --polling-namespaces compute' so something is definitely missing downstream. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:3462 |
openstack-nova: repeating errors on compute: Jun 8 05:33:38 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Environment: libvirt-daemon-driver-nwfilter-3.2.0-6.el7.x86_64 libvirt-client-3.2.0-6.el7.x86_64 openstack-nova-novncproxy-16.0.0-0.20170521033533.99bd334.el7ost.noarch libvirt-daemon-driver-storage-logical-3.2.0-6.el7.x86_64 libvirt-libs-3.2.0-6.el7.x86_64 libvirt-daemon-config-nwfilter-3.2.0-6.el7.x86_64 python-nova-16.0.0-0.20170521033533.99bd334.el7ost.noarch openstack-nova-conductor-16.0.0-0.20170521033533.99bd334.el7ost.noarch libvirt-daemon-driver-storage-mpath-3.2.0-6.el7.x86_64 puppet-nova-11.1.0-0.20170520132314.882caca.el7ost.noarch libvirt-daemon-3.2.0-6.el7.x86_64 libvirt-daemon-driver-nodedev-3.2.0-6.el7.x86_64 libvirt-daemon-driver-lxc-3.2.0-6.el7.x86_64 libvirt-daemon-kvm-3.2.0-6.el7.x86_64 openstack-nova-migration-16.0.0-0.20170521033533.99bd334.el7ost.noarch openstack-nova-api-16.0.0-0.20170521033533.99bd334.el7ost.noarch libvirt-3.2.0-6.el7.x86_64 libvirt-daemon-driver-storage-core-3.2.0-6.el7.x86_64 libvirt-daemon-driver-storage-rbd-3.2.0-6.el7.x86_64 libvirt-daemon-driver-storage-3.2.0-6.el7.x86_64 python-novaclient-8.0.0-0.20170517113627.e1b9e76.el7ost.noarch libvirt-daemon-driver-network-3.2.0-6.el7.x86_64 libvirt-daemon-driver-interface-3.2.0-6.el7.x86_64 libvirt-daemon-config-network-3.2.0-6.el7.x86_64 openstack-nova-compute-16.0.0-0.20170521033533.99bd334.el7ost.noarch openstack-nova-placement-api-16.0.0-0.20170521033533.99bd334.el7ost.noarch openstack-nova-console-16.0.0-0.20170521033533.99bd334.el7ost.noarch libvirt-daemon-driver-storage-scsi-3.2.0-6.el7.x86_64 libvirt-daemon-driver-storage-iscsi-3.2.0-6.el7.x86_64 libvirt-daemon-driver-secret-3.2.0-6.el7.x86_64 openstack-nova-common-16.0.0-0.20170521033533.99bd334.el7ost.noarch openstack-nova-scheduler-16.0.0-0.20170521033533.99bd334.el7ost.noarch libvirt-daemon-driver-storage-gluster-3.2.0-6.el7.x86_64 libvirt-python-3.2.0-2.el7.x86_64 libvirt-daemon-driver-qemu-3.2.0-6.el7.x86_64 libvirt-daemon-driver-storage-disk-3.2.0-6.el7.x86_64 instack-undercloud-7.0.0-0.20170521005009.el7ost.noarch openstack-tripleo-heat-templates-7.0.0-0.20170521033302.el7ost.noarch openstack-puppet-modules-10.0.0-0.20170315222135.0333c73.el7.1.noarch Steps to reproduce: deploy overcloud with containers: openstack overcloud deploy --templates \ --libvirt-type kvm \ -e /usr/share/openstack-tripleo-heat-templates/environments/docker.yaml \ -e /home/stack/virt/network/network-environment.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/virt/hostnames.yml \ -e /home/stack/virt/docker-osp12.yaml \ -e /home/stack/virt/debug.yaml \ -e /home/stack/virt/nodes_data.yaml Check the /var/log/messages on compute. Result: repeating error: Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory Jun 8 05:26:23 localhost journal: libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock-ro': No such file or directory