Attachment 686623 Details for Bug 903568 – Failure logs on controller and compute node

Failure logs on controller and compute node

file_903568.txt (text/plain), 5.37 KB, created by Ofer Blaut on 2013-01-24 10:12:15 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Ofer Blaut

Created: 2013-01-24 10:12:15 UTC

Size: 5.37 KB

patch

obsolete

>
>
>
>Debug level of nova.scheduler ( controller )
>Puma05  is the compute node that failed 
>
>2013-01-24 09:54:08 DEBUG nova.scheduler.filters.retry_filter [req-0c8c98e9-65a0-4d49-a624-c2ddf636c669 354ec8ca733a44dc8bf44708d977d147 af733fdee33d4c71bd739c3736c3057d] Previously tried hosts: [].  (host=puma05.scl.lab.tlv.redhat.com) host_passes /usr/lib/python2.6/site-packages/nova/scheduler/filters/retry_filter.py:39
>2013-01-24 09:54:08 DEBUG nova.scheduler.filters.compute_filter [req-0c8c98e9-65a0-4d49-a624-c2ddf636c669 354ec8ca733a44dc8bf44708d977d147 af733fdee33d4c71bd739c3736c3057d] host 'puma05.scl.lab.tlv.redhat.com': free_ram_mb:30051 free_disk_mb:419840 is disabled or has not been heard from in a while host_passes /usr/lib/python2.6/site-packages/nova/scheduler/filters/compute_filter.py:37
>2013-01-24 09:54:08 DEBUG nova.scheduler.host_manager [req-0c8c98e9-65a0-4d49-a624-c2ddf636c669 354ec8ca733a44dc8bf44708d977d147 af733fdee33d4c71bd739c3736c3057d] Host filter function <bound method ComputeFilter.host_passes of <nova.scheduler.filters.compute_filter.ComputeFilter object at 0x3d8a990>> failed for puma05.scl.lab.tlv.redhat.com passes_filters /usr/lib/python2.6/site-packages/nova/scheduler/host_manager.py:175
>2013-01-24 09:54:12 DEBUG nova.scheduler.filters.retry_filter [req-a7904bde-61fe-4478-a8f5-f295375737a7 354ec8ca733a44dc8bf44708d977d147 af733fdee33d4c71bd739c3736c3057d] Previously tried hosts: [].  (host=puma05.scl.lab.tlv.redhat.com) host_passes /usr/lib/python2.6/site-packages/nova/scheduler/filters/retry_filter.py:39
>2013-01-24 09:54:12 DEBUG nova.scheduler.filters.compute_filter [req-a7904bde-61fe-4478-a8f5-f295375737a7 354ec8ca733a44dc8bf44708d977d147 af733fdee33d4c71bd739c3736c3057d] host 'puma05.scl.lab.tlv.redhat.com': free_ram_mb:30051 free_disk_mb:419840 is disabled or has not been heard from in a while host_passes /usr/lib/python2.6/site-packages/nova/scheduler/filters/compute_filter.py:37
>2013-01-24 09:54:12 DEBUG nova.scheduler.host_manager [req-a7904bde-61fe-4478-a8f5-f295375737a7 354ec8ca733a44dc8bf44708d977d147 af733fdee33d4c71bd739c3736c3057d] Host filter function <bound method ComputeFilter.host_passes of <nova.scheduler.filters.compute_filter.ComputeFilter object at 0x3cb2490>> failed for puma05.scl.lab.tlv.redhat.com passes_filters /usr/lib/python2.6/site-packages/nova/scheduler/host_manager.py:175
>
>
>compute node - compute.log ( logs from puma05)
>
>2013-01-24 09:39:59 2066 DEBUG nova.virt.libvirt.driver [-] Connecting to libvirt: qemu:///system _get_connection /usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py:342
>2013-01-24 09:39:59 DEBUG nova.utils [req-da4bc736-718f-43d2-9fc0-85796cc15788 None None] backend <module 'nova.db.sqlalchemy.api' from '/usr/lib/python2.6/site-packages/nova/db/sqlalchemy/api.pyc'> __get_backend /usr/lib/python2.6/site-packages/nova/utils.py:502
>2013-01-24 09:40:00 DEBUG nova.compute.manager [req-da4bc736-718f-43d2-9fc0-85796cc15788 None None] [instance: 8c8bcf56-c304-44ec-b32d-82709755a2c4] Checking state _get_power_state /usr/lib/python2.6/site-packages/nova/compute/manager.py:335
>2013-01-24 09:40:00 DEBUG nova.compute.manager [req-da4bc736-718f-43d2-9fc0-85796cc15788 None None] [instance: 8c8bcf56-c304-44ec-b32d-82709755a2c4] Current state is 4, state in DB is 1. init_host /usr/lib/python2.6/site-packages/nova/compute/manager.py:289
>2013-01-24 09:40:00 DEBUG nova.virt.libvirt.vif [req-da4bc736-718f-43d2-9fc0-85796cc15788 None None] [instance: 8c8bcf56-c304-44ec-b32d-82709755a2c4] Ensuring vlan 88 and bridge br88 plug /usr/lib/python2.6/site-packages/nova/virt/libvirt/vif.py:98
>2013-01-24 09:40:00 DEBUG nova.utils [req-da4bc736-718f-43d2-9fc0-85796cc15788 None None] Got semaphore "ensure_vlan" for method "ensure_vlan"... inner /usr/lib/python2.6/site-packages/nova/utils.py:721
>2013-01-24 09:40:00 DEBUG nova.utils [req-da4bc736-718f-43d2-9fc0-85796cc15788 None None] Attempting to grab file lock "ensure_vlan" for method "ensure_vlan"... inner /usr/lib/python2.6/site-packages/nova/utils.py:725
>2013-01-24 09:40:00 DEBUG nova.utils [req-da4bc736-718f-43d2-9fc0-85796cc15788 None None] Got file lock "ensure_vlan" for method "ensure_vlan"... inner /usr/lib/python2.6/site-packages/nova/utils.py:751
>2013-01-24 09:40:00 DEBUG nova.utils [req-da4bc736-718f-43d2-9fc0-85796cc15788 None None] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf ip link show dev vlan88 execute /usr/lib/python2.6/site-packages/nova/utils.py:183
>2013-01-24 09:40:00 DEBUG nova.utils [req-da4bc736-718f-43d2-9fc0-85796cc15788 None None] Result was 255 execute /usr/lib/python2.6/site-packages/nova/utils.py:199
>2013-01-24 09:40:00 DEBUG nova.network.linux_net [req-da4bc736-718f-43d2-9fc0-85796cc15788 None None] Starting VLAN inteface vlan88 ensure_vlan /usr/lib/python2.6/site-packages/nova/network/linux_net.py:1125
>2013-01-24 09:40:00 DEBUG nova.utils [req-da4bc736-718f-43d2-9fc0-85796cc15788 None None] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf ip link add link eth4 name vlan88 type vlan id 88 execute /usr/lib/python2.6/site-packages/nova/utils.py:183
>2013-01-24 09:40:00 DEBUG nova.utils [req-da4bc736-718f-43d2-9fc0-85796cc15788 None None] Result was 255 execute /usr/lib/python2.6/site-packages/nova/utils.py:199
>2013-01-24 09:40:00 2066 CRITICAL nova [-] Unexpected error while running command.
>Command: sudo nova-rootwrap /etc/nova/rootwrap.conf ip link add link eth4 name vlan88 type vlan id 88
>

Actions: View

Attachments on bug 903568: 686623