Attachment 674871 Details for Bug 862862 – rhel5 hvm guest kernel crash with 2.6.18-308.20.1.el5.bz862862_ixgbevf_crash_4

rhel5 hvm guest kernel crash with 2.6.18-308.20.1.el5.bz862862_ixgbevf_crash_4

serial-console-log-2.6.18-308.20.1.el5.bz862862_ixgbevf_crash_4.txt (text/plain), 33.77 KB, created by Pasi Karkkainen on 2013-01-08 14:16:42 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Pasi Karkkainen

Created: 2013-01-08 14:16:42 UTC

Size: 33.77 KB

patch

obsolete

>Linux version 2.6.18-308.20.1.el5.bz862862_ixgbevf_crash_4 (mockbuild@x86-010.build.bos.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-54)) #1 SMP Mon Jan 7 13:30:26 EST 2013
>Command line: ro root=/dev/VolGroup00/LogVol00 ignore_loglevel console=tty0 console=ttyS0,115200,8n1
>BIOS-provided physical RAM map:
> BIOS-e820: 0000000000010000 - 000000000009fc00 (usable)
> BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
> BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
> BIOS-e820: 0000000000100000 - 000000003fffac00 (usable)
> BIOS-e820: 000000003fffac00 - 0000000040000000 (reserved)
>DMI 2.4 present.
>DMI: Red Hat HVM domU, BIOS 3.1.2-308.13.1.el5 09/27/2012
>ACPI: RSDP (v002    Xen                                ) @ 0x00000000000eb340
>ACPI: XSDT (v001    Xen      HVM 0x00000000 HVML 0x00000000) @ 0x00000000000eb2b0
>ACPI: FADT (v004    Xen      HVM 0x00000000 HVML 0x00000000) @ 0x00000000000eb0c0
>ACPI: MADT (v002    Xen      HVM 0x00000000 HVML 0x00000000) @ 0x00000000000eb1c0
>ACPI: HPET (v001    Xen      HVM 0x00000000 HVML 0x00000000) @ 0x00000000000eb240
>ACPI: SSDT (v002    Xen      HVM 0x00000000 HVML 0x00000000) @ 0x00000000000eb278
>ACPI: DSDT (v002    Xen      HVM 0x00000000 INTL 0x20110413) @ 0x0000000000000000
>No NUMA configuration found
>Faking a node at 0000000000000000-000000003fffa000
>Bootmem setup node 0 0000000000000000-000000003fffa000
>bootmap start 18000 pages 8
>Memory for crash kernel (0x0 to 0x0) notwithin permissible range
>disabling kdump
>Setting up node 0 0-3fffa
>On node 0 totalpages: 256941
>  DMA zone: 2426 pages, LIFO batch:0
>  DMA32 zone: 254515 pages, LIFO batch:31
>ACPI: PM-Timer IO Port: 0x1f48
>ACPI: Local APIC address 0xfee00000
>ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
>Processor #0 6:12 APIC version 20
>ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
>IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-47
>ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
>ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 low level)
>ACPI: INT_SRC_OVR (bus 0 bus_irq 7 global_irq 7 low level)
>ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 low level)
>ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 low level)
>ACPI: IRQ0 used by override.
>ACPI: IRQ2 used by override.
>ACPI: IRQ5 used by override.
>ACPI: IRQ7 used by override.
>ACPI: IRQ9 used by override.
>ACPI: IRQ10 used by override.
>ACPI: IRQ11 used by override.
>Setting APIC routing to physical flat
>ACPI: HPET id: 0x8086a201 base: 0xfed00000
>Using ACPI (MADT) for SMP configuration information
>Nosave address range: 000000000009f000 - 00000000000a0000
>Nosave address range: 00000000000a0000 - 00000000000e0000
>Nosave address range: 00000000000e0000 - 0000000000100000
>Allocating PCI resources starting at 50000000 (gap: 40000000:c0000000)
>SMP: Allowing 1 CPUs, 0 hotplug CPUs
>Built 1 zonelists.  Total pages: 256941
>Kernel command line: ro root=/dev/VolGroup00/LogVol00 ignore_loglevel console=tty0 console=ttyS0,115200,8n1
>debug: ignoring loglevel setting.
>Initializing CPU#0
>PID hash table entries: 4096 (order: 12, 32768 bytes)
>Console: colour VGA+ 80x25
>Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes)
>Inode-cache hash table entries: 65536 (order: 7, 524288 bytes)
>Checking aperture...
>ACPI: DMAR not present
>Memory: 1022448k/1048552k available (2620k kernel code, 25652k reserved, 1673k data, 224k init)
>Calibrating delay loop (skipped), value calculated using timer frequency.. 4533.60 BogoMIPS (lpj=2266802)
>Security Framework v1.0.0 initialized
>SELinux:  Initializing.
>SELinux:  Starting in permissive mode
>selinux_register_security:  Registering secondary module capability
>Capability LSM initialized as secondary
>Mount-cache hash table entries: 256
>CPU: L1 I cache: 32K, L1 D cache: 32K
>CPU: L2 cache: 256K
>CPU: L3 cache: 12288K
>MCE: Machine Check Exception Reporting is disabled.
>SMP alternatives: switching to UP code
>Freeing SMP alternatives: 32k freed
>ACPI: Core revision 20060707
>Using local APIC timer interrupts.
>WARNING calibrate_APIC_clock: the APIC timer calibration may be wrong.
>Detected 6.250 MHz APIC timer.
>Brought up 1 CPUs
>time.c: Using 70.837629 MHz WALL HPET GTOD HPET/TSC timer.
>time.c: Detected 2266.802 MHz processor.
>sizeof(vma)=176 bytes
>sizeof(page)=56 bytes
>sizeof(inode)=560 bytes
>sizeof(dentry)=216 bytes
>sizeof(ext3inode)=760 bytes
>sizeof(buffer_head)=96 bytes
>sizeof(skbuff)=248 bytes
>checking if image is initramfs... it is
>Freeing initrd memory: 3419k freed
>NET: Registered protocol family 16
>ACPI: bus type pci registered
>PCI: Using configuration type 1
>ACPI: Interpreter enabled
>ACPI: Using IOAPIC for interrupt routing
>ACPI: No dock devices found.
>ACPI: PCI Root Bridge [PCI0] (0000:00)
>PCI quirk: region 1f40-1f7f claimed by PIIX4 ACPI
>ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
>ACPI: PCI Interrupt Link [LNKA] (IRQs *5 7 10 11)
>ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *7 10 11)
>ACPI: PCI Interrupt Link [LNKC] (IRQs 5 7 *10 11)
>ACPI: PCI Interrupt Link [LNKD] (IRQs 5 7 10 *11)
>Linux Plug and Play Support v0.97 (c) Adam Belay
>pnp: PnP ACPI init
>pnp: PnP ACPI: found 12 devices
>usbcore: registered new driver usbfs
>usbcore: registered new driver hub
>PCI: Using ACPI for IRQ routing
>PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
>NetLabel: Initializing
>NetLabel:  domain hash size = 128
>NetLabel:  protocols = UNLABELED CIPSOv4
>NetLabel:  unlabeled traffic allowed by default
>hpet0: at MMIO 0xfed00000 (virtual 0xffffffffff5fe000), IRQs 2, 8, 0
>hpet0: 3 64-bit timers, 70837629 Hz
>ACPI: DMAR not present
>PCI-GART: No AMD northbridge found.
>pnp: 00:00: iomem range 0x0-0x9ffff could not be reserved
>pnp: 00:02: ioport range 0x10c0-0x10c2 has been reserved
>pnp: 00:02: ioport range 0xb044-0xb047 has been reserved
>NET: Registered protocol family 2
>IP route cache hash table entries: 32768 (order: 6, 262144 bytes)
>TCP established hash table entries: 131072 (order: 9, 2097152 bytes)
>TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
>TCP: Hash tables configured (established 131072 bind 65536)
>TCP reno registered
>audit: initializing netlink socket (disabled)
>type=2000 audit(1357654237.869:1): initialized
>Total HugeTLB memory allocated, 0
>VFS: Disk quotas dquot_6.5.1
>Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
>SELinux:  Registering netfilter hooks
>Initializing Cryptographic API
>alg: No test for crc32c (crc32c-generic)
>ksign: Installing public key data
>Loading keyring
>- Added public key 1183259DB2FEFEAF
>- User ID: Red Hat, Inc. (Kernel Module GPG key)
>io scheduler noop registered
>io scheduler anticipatory registered
>io scheduler deadline registered
>io scheduler cfq registered (default)
>Limiting direct PCI/PCI transfers.
>PCI: PIIX3: Enabling Passive Release on 0000:00:01.0
>Activating ISA DMA hang workarounds.
>Boot video device is 0000:00:02.0
>pci_hotplug: PCI Hot Plug PCI Core version: 0.5
>Real Time Clock Driver v1.12ac
>hpet_resources: 0xfed00000 is busy
>Non-volatile memory driver v1.2
>Linux agpgart interface v0.101 (c) Dave Jones
>Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
>ÿserial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16450
>00:0b: ttyS0 at I/O 0x3f8 (irq = 4) is a 16450
>brd: module loaded
>Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
>ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
>PIIX3: IDE controller at PCI slot 0000:00:01.1
>PIIX3: chipset revision 0
>PIIX3: not 100% native mode: will probe irqs later
>PCI: Setting latency timer of device 0000:00:01.1 to 64
>    ide0: BM-DMA at 0xc000-0xc007, BIOS settings: hda:pio, hdb:pio
>    ide1: BM-DMA at 0xc008-0xc00f, BIOS settings: hdc:pio, hdd:pio
>Probing IDE interface ide0...
>hda: QEMU HARDDISK, ATA DISK drive
>ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
>Probing IDE interface ide1...
>hdc: QEMU CD-ROM, ATAPI CD/DVD-ROM drive
>ide1 at 0x170-0x177,0x376 on irq 15
>hda: max request size: 512KiB
>hda: 41943040 sectors (21474 MB) w/256KiB Cache, CHS=16383/255/63, (U)DMA
>hda: cache flushes supported
> hda: hda1 hda2
>ide-floppy driver 0.99.newide
>usbcore: registered new driver hiddev
>usbcore: registered new driver usbhid
>drivers/usb/input/hid-core.c: v2.6:USB HID core driver
>PNP: PS/2 Controller [PNP0303:PS2K,PNP0f13:PS2M] at 0x60,0x64 irq 1,12
>serio: i8042 KBD port at 0x60,0x64 irq 1
>serio: i8042 AUX port at 0x60,0x64 irq 12
>mice: PS/2 mouse device common for all mice
>md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
>md: bitmap version 4.39
>TCP bic registered
>Initializing IPsec netlink socket
>NET: Registered protocol family 1
>NET: Registered protocol family 17
>input: AT Translated Set 2 keyboard as /class/input/input0
>ACPI: (supports S5)
>Initalizing network drop monitor service
>Freeing unused kernel memory: 224k freed
>Write protecting the kernel read-only data: 531k
>Red Hat nash version 5.1.19.6 starting
>Mounting proc filesystem
>Mounting sysfs filesystem
>Creating /dev
>Creating initial device nodes
>Setting up hotplug.
>input: ImExPS/2 Generic Explorer Mouse as /class/input/input1
>Creating block device nodes.
>Loading ehci-hcd.ko module
>Loading ohci-hcd.ko module
>ohci_hcd: 2005 April 22 USB 1.1 'Open' Host Controller (OHCI) Driver (PCI)
>Loading uhci-hcd.ko module
>USB Universal Host Controller Interface driver v3.0
>GSI 16 sharing vector 0xA9 and IRQ 16
>ACPI: PCI Interrupt 0000:00:01.3[D] -> GSI 23 (level, low) -> IRQ 169
>PCI: Setting latency timer of device 0000:00:01.3 to 64
>uhci_hcd 0000:00:01.3: UHCI Host Controller
>uhci_hcd 0000:00:01.3: new USB bus registered, assigned bus number 1
>uhci_hcd 0000:00:01.3: irq 169, io base 0x0000c020
>usb usb1: configuration #1 chosen from 1 choice
>hub 1-0:1.0: USB hub found
>hub 1-0:1.0: 2 ports detected
>Loading jbd.ko module
>Loading ext3.ko module
>Loading scsi_mod.ko module
>SCSI subsystem initialized
>Loading sd_mod.ko module
>Loading libata.ko module
>libata version 3.00 loaded.
>Loading ata_piix.ko module
>Loading dm-mod.ko module
>device-mapper: uevent: version 1.0.3
>device-mapper: ioctl: 4.11.6-ioctl (2011-02-18) initialised: dm-devel@redhat.com
>Loading dm-log.ko module
>Loading dm-mirror.ko module
>Loading dm-zero.ko module
>Loading dm-snapshot.ko module
>Loading dm-mem-cache.ko module
>Loading dm-region_hash.ko module
>Loading dm-message.ko module
>Loading dm-raid45.ko module
>device-mapper: dm-raid45: initialized v0.2594l
>usb 1-2: new full speed USB device using uhci_hcd and address 2
>Waiting for driver initialization.
>usb 1-2: configuration #1 chosen from 1 choice
>input: QEMU 0.8.2 QEMU USB Tablet as /class/input/input2
>input,hidraw0: USB HID v0.01 Pointer [QEMU 0.8.2 QEMU USB Tablet] on usb-0000:00:01.3-2
>Scanning and configuring dmraid supported devices
>Scanning logical volumes
>  Reading all physical volumes.  This may take a while...
>  Found volume group "VolGroup00" using metadata type lvm2
>Activating logical volumes
>  2 logical volume(s) in volume group "VolGroup00" now active
>Trying to resume from /dev/VolGroup00/LogVol01
>No suspend signature on swap, not resuming.
>Creating root device.
>Mounting root filesystem.
>kjournald starting.  Commit interval 5 seconds
>EXT3-fs: mounted filesystem with ordered data mode.
>Setting up other filesystems.
>Setting up new root fs
>no fstab.sys, mounting internal defaults
>Switching to new root and running init.
>unmounting old /dev
>unmounting old /proc
>unmounting old /sys
>SELinux:  Disabled at runtime.
>SELinux:  Unregistering netfilter hooks
>type=1404 audit(1357654261.350:2): selinux=0 auid=4294967295 ses=4294967295
>
INIT: version 2.86 booting

>		Welcome to Red Hat Enterprise Linux Server
>		Press 'I' to enter interactive startup.
>Setting clock  (localtime): Tue Jan  8 14:11:02 EET 2013 [  OK  ]

>Starting udev: input: PC Speaker as /class/input/input3
>802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
>All bugs added by David S. Miller <davem@redhat.com>
>ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function Network Driver - version 2.1.0-k
>Copyright (c) 2009 - 2010 Intel Corporation.
>ixgbevf 0000:00:06.0: ixgbevf_probe() entry
>ixgbevf 0000:00:06.0: ixgbevf_probe() pci_using_dac=1
>PCI: Setting latency timer of device 0000:00:06.0 to 64
>ixgbevf 0000:00:06.0: ixgbevf_assign_netdev_ops() entry
>ixgbevf 0000:00:06.0: ixgbevf_assign_netdev_ops() exit
>ixgbevf 0000:00:06.0: ixgbevf_sw_init() entry
>hdc: ATAPI 4X CD-ROM drive, 512kB Cache, (U)DMA
>Uniform CD-ROM driver Revision: 3.20
>ixgbevf 0000:00:06.0: ixgbevf_sw_init() init_hw OK
>ixgbevf 0000:00:06.0: ixgbevf_sw_init() exit (0)
>ixgbevf 0000:00:06.0: ixgbevf_probe() ixgbevf_sw_init=0 (ignored)
>ixgbevf 0000:00:06.0: ixgbevf_init_interrupt_scheme() entry
>ixgbevf 0000:00:06.0: ixgbevf_set_num_queues() entry
>ixgbevf 0000:00:06.0: ixgbevf_set_num_queues() exit
>ixgbevf 0000:00:06.0: ixgbevf_set_interrupt_capability() entry
>ixgbevf 0000:00:06.0: ixgbevf_set_interrupt_capability() num_rx_queues=1 num_tx_queues=1 v_budget=3
>ixgbevf 0000:00:06.0: ixgbevf_set_interrupt_capability() v_budget alloc OK
>ixgbevf 0000:00:06.0: ixgbevf_acquire_msix_vectors() entry: vectors=3
>ixgbevf 0000:00:06.0: ixgbevf_acquire_msix_vectors() acquiring vectors=3
>ixgbevf 0000:00:06.0: ixgbevf_acquire_msix_vectors() success
>ixgbevf 0000:00:06.0: ixgbevf_acquire_msix_vectors(): allocated 3 MSI-X interrupts
>ixgbevf 0000:00:06.0: ixgbevf_acquire_msix_vectors() exit
>ixgbevf 0000:00:06.0: ixgbevf_set_interrupt_capability() exit (0)
>ixgbevf 0000:00:06.0: ixgbevf_alloc_q_vectors() entry num_msix_vectors=3 num_q_vectors=2 napi_vectors=1
>ixgbevf 0000:00:06.0: ixgbevf_alloc_q_vectors() exit (success)
>ixgbevf 0000:00:06.0: ixgbevf_alloc_queues() entry num_tx_queues=1 num_rx_queues=1 tx_ring_count=1024 rx_ring_count=512
>ixgbevf 0000:00:06.0: ixgbevf_alloc_queues() tx_ring[0] @ ffff81003fc90ec0
>ixgbevf 0000:00:06.0: ixgbevf_alloc_queues() rx_ring[0] @ ffff81003fc90340
>ixgbevf 0000:00:06.0: ixgbevf_alloc_queues() exit (success)
>ixgbevf 0000:00:06.0: ixgbevf_init_interrupt_scheme() exit (success) num_rx_queues=1 num_tx_queues=1
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_probe() registered with name=eth0
>ixgbevf 0000:00:06.0: ixgbevf_init_last_counter_stats() entry
>ixgbevf 0000:00:06.0: ixgbevf_init_last_counter_stats() exit
>ixgbevf 0000:00:06.0: ixgbevf_probe() exit (OK) hw_mac_type=1 mac=02:9C:00:8D:36:F0 cards_found=1
>ixgbevf 0000:00:07.0: ixgbevf_probe() entry
>ixgbevf 0000:00:07.0: ixgbevf_probe() pci_using_dac=1
>PCI: Setting latency timer of device 0000:00:07.0 to 64
>ixgbevf 0000:00:07.0: ixgbevf_assign_netdev_ops() entry
>ixgbevf 0000:00:07.0: ixgbevf_assign_netdev_ops() exit
>ixgbevf 0000:00:07.0: ixgbevf_sw_init() entry
>ixgbevf 0000:00:07.0: ixgbevf_sw_init() init_hw OK
>ixgbevf 0000:00:07.0: ixgbevf_sw_init() exit (0)
>ixgbevf 0000:00:07.0: ixgbevf_probe() ixgbevf_sw_init=0 (ignored)
>ixgbevf 0000:00:07.0: ixgbevf_init_interrupt_scheme() entry
>ixgbevf 0000:00:07.0: ixgbevf_set_num_queues() entry
>ixgbevf 0000:00:07.0: ixgbevf_set_num_queues() exit
>ixgbevf 0000:00:07.0: ixgbevf_set_interrupt_capability() entry
>ixgbevf 0000:00:07.0: ixgbevf_set_interrupt_capability() num_rx_queues=1 num_tx_queues=1 v_budget=3
>ixgbevf 0000:00:07.0: ixgbevf_set_interrupt_capability() v_budget alloc OK
>ixgbevf 0000:00:07.0: ixgbevf_acquire_msix_vectors() entry: vectors=3
>ixgbevf 0000:00:07.0: ixgbevf_acquire_msix_vectors() acquiring vectors=3
>ixgbevf 0000:00:07.0: ixgbevf_acquire_msix_vectors() success
>ixgbevf 0000:00:07.0: ixgbevf_acquire_msix_vectors(): allocated 3 MSI-X interrupts
>ixgbevf 0000:00:07.0: ixgbevf_acquire_msix_vectors() exit
>ixgbevf 0000:00:07.0: ixgbevf_set_interrupt_capability() exit (0)
>ixgbevf 0000:00:07.0: ixgbevf_alloc_q_vectors() entry num_msix_vectors=3 num_q_vectors=2 napi_vectors=1
>ixgbevf 0000:00:07.0: ixgbevf_alloc_q_vectors() exit (success)
>ixgbevf 0000:00:07.0: ixgbevf_alloc_queues() entry num_tx_queues=1 num_rx_queues=1 tx_ring_count=1024 rx_ring_count=512
>ixgbevf 0000:00:07.0: ixgbevf_alloc_queues() tx_ring[0] @ ffff81003fc90140
>ixgbevf 0000:00:07.0: ixgbevf_alloc_queues() rx_ring[0] @ ffff81003fc90740
>ixgbevf 0000:00:07.0: ixgbevf_alloc_queues() exit (success)
>ixgbevf 0000:00:07.0: ixgbevf_init_interrupt_scheme() exit (success) num_rx_queues=1 num_tx_queues=1
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_probe() registered with name=eth1
>ixgbevf 0000:00:07.0: ixgbevf_init_last_counter_stats() entry
>ixgbevf 0000:00:07.0: ixgbevf_init_last_counter_stats() exit
>ixgbevf 0000:00:07.0: ixgbevf_probe() exit (OK) hw_mac_type=1 mac=4A:45:2E:CF:3B:E0 cards_found=2
>Intel(R) PRO/1000 Network Driver - version 7.3.21-k4-3-NAPI
>Copyright (c) 1999-2006 Intel Corporation.
>GSI 17 sharing vector 0xE9 and IRQ 17
>ACPI: PCI Interrupt 0000:00:04.0[A] -> GSI 32 (level, low) -> IRQ 233
>PCI: Setting latency timer of device 0000:00:04.0 to 64
>e1000: 0000:00:04.0: e1000_probe: (PCI:33MHz:32-bit) 00:26:9f:15:15:25
>e1000: eth2: e1000_probe: Intel(R) PRO/1000 Network Connection
>GSI 18 sharing vector 0x32 and IRQ 18
>ACPI: PCI Interrupt 0000:00:03.0[A] -> GSI 28 (level, low) -> IRQ 50
>Xen version 3.1.
>Hypercall area is 1 pages.
>Grant table initialized
>dev_change_name: 'eth2' -> 'eth0' enter
>dev_change_name: exit (EEXIST)
>dev_change_name: 'eth0' -> '__tmp1686290279' enter
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>dev_change_name: exit (class_device_rename=0)
>dev_change_name: 'eth2' -> 'eth0' enter
>dev_change_name: exit (class_device_rename=0)
>xen_mem: Initialising balloon driver.
>Registering block device major 3
>register_blkdev: cannot get major 3 for ide
>xen_blk: can't get major 3 with name ide
>vbd vbd-768: 19 xlvbd_add at /local/domain/0/backend/vbd/20/768
>Registering block device major 3
>register_blkdev: cannot get major 3 for ide
>xen_blk: can't get major 3 with name ide
>vbd vbd-768: 19 xlvbd_add at /local/domain/0/backend/vbd/20/768
>Registering block device major 22
>register_blkdev: cannot get major 22 for ide
>xen_blk: can't get major 22 with name ide
>vbd vbd-5632: 19 xlvbd_add at /local/domain/0/backend/vbd/20/5632
>Registering block device major 22
>register_blkdev: cannot get major 22 for ide
>xen_blk: can't get major 22 with name ide
>vbd vbd-5632: 19 xlvbd_add at /local/domain/0/backend/vbd/20/5632
>netfront: Initialising virtual ethernet driver.
>netfront: device eth2 has copying receive path.
>dev_change_name: 'eth2' -> 'eth0' enter
>dev_change_name: exit (EEXIST)
>dev_change_name: 'eth0' -> 'eth0' enter
>dev_change_name: exit (EEXIST)
>[  OK  ]

>FDC 0 is a S82078B
>lp: driver loaded but no devices found
>Loading default keymap (fi): [  OK  ]

>Setting hostname rhel58x64hvm01.localdomain:  [  OK  ]

>ACPI: Mapper loaded
>dell-wmi: No known WMI GUID found
>md: Autodetecting RAID arrays.
>md: autorun ...
>md: ... autorun DONE.
>device-mapper: multipath: version 1.0.6 loaded
>Setting up Logical Volume Management:   2 logical volume(s) in volume group "VolGroup00" now active
>[  OK  ]

>Checking filesystems
>Checking all file systems.
>[/sbin/fsck.ext3 (1) -- /] fsck.ext3 -a /dev/VolGroup00/LogVol00 
>/dev/VolGroup00/LogVol00: clean, 135270/4597728 files, 1246754/4595712 blocks
>[/sbin/fsck.ext3 (1) -- /boot] fsck.ext3 -a /dev/hda1 
>/boot: clean, 53/128520 files, 58232/514048 blocks
>[  OK  ]

>Remounting root filesystem in read-write mode:  EXT3 FS on dm-0, internal journal
>[  OK  ]

>Mounting local filesystems:  kjournald starting.  Commit interval 5 seconds
>EXT3 FS on hda1, internal journal
>EXT3-fs: mounted filesystem with ordered data mode.
>[  OK  ]

>Enabling local filesystem quotas:  [  OK  ]

>Enabling /etc/fstab swaps:  Adding 2064376k swap on /dev/VolGroup00/LogVol01.  Priority:-1 extents:1 across:2064376k
>[  OK  ]
>
INIT: Entering runlevel: 3

>Entering non-interactive startup
>Applying Intel CPU microcode update: IA-32 Microcode Update Driver: v1.14a <tigran@veritas.com>
>[  OK  ]

>Starting background readahead: [  OK  ]

>Checking for hardware changes [  OK  ]

>Applying ip6tables firewall rules: NET: Registered protocol family 10
>lo: Disabled Privacy Extensions
>IPv6 over IPv4 tunneling driver
>sit0: Disabled Privacy Extensions
>ip6_tables: (C) 2000-2006 Netfilter Core Team
>[  OK  ]

>Applying iptables firewall rules: ip_tables: (C) 2000-2006 Netfilter Core Team
>Netfilter messages via NETLINK v0.30.
>ip_conntrack version 2.4 (4095 buckets, 32760 max) - 304 bytes per conntrack
>[  OK  ]

>Loading additional iptables modules: ip_conntrack_netbios_ns [  OK  ]

>Bringing up loopback interface:  ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>[  OK  ]

>Bringing up interface eth0:  
>Determining IP information for eth0...ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>e1000: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
>8021q: adding VLAN 0 to HW filter on device eth0
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
> done.
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>[  OK  ]

>Starting system logger: [  OK  ]

>Starting kernel logger: [  OK  ]

>Starting irqbalance: [  OK  ]

>Mounting other filesystems:  [  OK  ]

>Starting acpi daemon: [  OK  ]

>Starting monitoring for VG VolGroup00:   2 logical volume(s) in volume group "VolGroup00" monitored
>[  OK  ]

>Starting sshd: [  OK  ]

>Starting xinetd: [  OK  ]

>Starting console mouse services: [  OK  ]

>Starting crond: [  OK  ]

>Starting anacron: [  OK  ]

>Starting atd: [  OK  ]
[  OK  ]

>Starting rhsmcertd 240 1440[  OK  ]

>Starting yum-updatesd: [  OK  ]

>Starting smartd: [  OK  ]

>
>Red Hat Enterprise Linux Server release 5.8 (Tikanga)
>Kernel 2.6.18-308.20.1.el5.bz862862_ixgbevf_crash_4 on an x86_64
>
>rhel58x64hvm01.localdomain login: eth0: no IPv6 routers present
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:07.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_open() entry
>ixgbevf 0000:00:06.0: ixgbevf_setup_all_tx_resources() entry
>ixgbevf 0000:00:06.0: ixgbevf_setup_tx_resources() entry tx_ring=ffff81003fc90ec0
>ixgbevf 0000:00:06.0: ixgbevf_setup_tx_resources() exit (OK)
>ixgbevf 0000:00:06.0: ixgbevf_setup_all_tx_resources() exit (0)
>ixgbevf 0000:00:06.0: ixgbevf_setup_all_rx_resources() entry
>ixgbevf 0000:00:06.0: ixgbevf_setup_rx_resources() entry rx_ring=ffff81003fc90340
>ixgbevf 0000:00:06.0: ixgbevf_setup_rx_resources() exit (OK)
>ixgbevf 0000:00:06.0: ixgbevf_setup_all_rx_resources() exit (0)
>ixgbevf 0000:00:06.0: ixgbevf_configure() entry
>ixgbevf 0000:00:06.0: ixgbevf_set_rx_mode() entry mc_count=1
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() entry
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() exit addr=ffff810001a53148
>ixgbevf 0000:00:06.0: ixgbevf_set_rx_mode() exit
>ixgbevf 0000:00:06.0: ixgbevf_restore_vlan() entry
>ixgbevf 0000:00:06.0: ixgbevf_vlan_rx_register() entry
>ixgbevf 0000:00:06.0: ixgbevf_vlan_rx_register() exit
>ixgbevf 0000:00:06.0: ixgbevf_restore_vlan() exit
>ixgbevf 0000:00:06.0: ixgbevf_configure_tx() entry
>ixgbevf 0000:00:06.0: ixgbevf_configure_tx() exit
>ixgbevf 0000:00:06.0: ixgbevf_configure_rx() entry
>ixgbevf 0000:00:06.0: ixgbevf_configure_rx() mtu=1500 flags=0x00000003 rx_buf_len=1522 rdlen=8192
>ixgbevf 0000:00:06.0: ixgbevf_configure_srrctl() entry index=0
>ixgbevf 0000:00:06.0: ixgbevf_configure_srrctl() not IXGBE_FLAG_RX_PS_ENABLED
>ixgbevf 0000:00:06.0: ixgbevf_configure_srrctl() MAXIMUM_ETHERNET_VLAN_SIZE
>ixgbevf 0000:00:06.0: ixgbevf_configure_srrctl() exit
>ixgbevf 0000:00:06.0: ixgbevf_configure_rx() exit
>ixgbevf 0000:00:06.0: ixgbevf_configure() exit
>ixgbevf 0000:00:06.0: ixgbevf_map_rings_to_vectors() entry num_msix_vectors=3 q_vectors=2 num_rx_queues=1 num_tx_queues=1
>ixgbevf 0000:00:06.0: ixgbevf_map_rings_to_vectors() one queue per vector
>ixgbevf 0000:00:06.0: map_vector_to_rxq() entry v_idx=0 r_idx=0
>ixgbevf 0000:00:06.0: map_vector_to_rxq() exit rxr_count=1
>ixgbevf 0000:00:06.0: map_vector_to_txq() entry v_idx=1 t_idx=0
>ixgbevf 0000:00:06.0: map_vector_to_txq() exit txr_count=1
>ixgbevf 0000:00:06.0: ixgbevf_map_rings_to_vectors() exit (0)
>ixgbevf 0000:00:06.0: ixgbevf_up_complete() entry hw_mac_type=1
>ixgbevf 0000:00:06.0: ixgbevf_rx_desc_queue_enable() entry rxr=0
>ixgbevf 0000:00:06.0: ixgbevf_rx_desc_queue_enable() exit
>ixgbevf 0000:00:06.0: ixgbevf_configure_msix() entry num_msix_vectors=3 q_vectors=2
>ixgbevf 0000:00:06.0: ixgbevf_configure_msix() configuring RX queues for MSI-X v_idx=0 rxr_count=1
>ixgbevf 0000:00:06.0: ixgbevf_configure_msix() rx_ring_idx=0
>ixgbevf 0000:00:06.0: ixgbevf_set_ivar() entry direction=0 queue=0 msix_vector=0
>ixgbevf 0000:00:06.0: ixgbevf_set_ivar() exit
>ixgbevf 0000:00:06.0: ixgbevf_configure_msix() configuring TX queues for MSI-X v_idx=0 txr_count=0
>ixgbevf 0000:00:06.0: ixgbevf_configure_msix() configuring RX queues for MSI-X v_idx=1 rxr_count=0
>ixgbevf 0000:00:06.0: ixgbevf_configure_msix() configuring TX queues for MSI-X v_idx=1 txr_count=1
>ixgbevf 0000:00:06.0: ixgbevf_configure_msix() tx_ring_idx=0
>ixgbevf 0000:00:06.0: ixgbevf_set_ivar() entry direction=1 queue=0 msix_vector=1
>ixgbevf 0000:00:06.0: ixgbevf_set_ivar() exit
>ixgbevf 0000:00:06.0: ixgbevf_set_ivar() entry direction=-1 queue=1 msix_vector=2
>ixgbevf 0000:00:06.0: ixgbevf_set_ivar() exit
>ixgbevf 0000:00:06.0: ixgbevf_configure_msix() exit
>ixgbevf 0000:00:06.0: ixgbevf_up_complete() addr=02:9C:00:8D:36:F0
>ixgbevf 0000:00:06.0: ixgbevf_up_complete() set_rar=0
>ixgbevf 0000:00:06.0: ixgbevf_napi_enable_all() entry num_msix_vectors=3 q_vectors=2
>ixgbevf 0000:00:06.0: ixgbevf_napi_enable_all() q_idx=0 rxr_count=1
>ixgbevf 0000:00:06.0: ixgbevf_napi_enable_all() q_idx=1 rxr_count=0
>ixgbevf 0000:00:06.0: ixgbevf_napi_enable_all() exit
>ixgbevf 0000:00:06.0: ixgbevf_save_reset_stats() entry save=0
>ixgbevf 0000:00:06.0: ixgbevf_save_reset_stats() exit
>ixgbevf 0000:00:06.0: ixgbevf_init_last_counter_stats() entry
>ixgbevf 0000:00:06.0: ixgbevf_init_last_counter_stats() exit
>ixgbevf 0000:00:06.0: ixgbevf_up_complete() exit (OK)
>ixgbevf 0000:00:06.0: ixgbevf_request_irq() entry
>ixgbevf 0000:00:06.0: ixgbevf_request_msix_irqs() entry num_msix_vectors=3 q_vectors=2
>ixgbevf 0000:00:06.0: ixgbevf_request_msix_irqs() requesting irq=185 for vector=0 with name=__tmp1686290279-rx-0
>ixgbevf 0000:00:06.0: ixgbevf_watchdog_task() NIC Link is Up, 10 Gbps
>ixgbevf 0000:00:06.0: ixgbevf_request_msix_irqs() requesting irq=193 for vector=1 with name=__tmp1686290279-tx-0
>ixgbevf 0000:00:06.0: ixgbevf_request_msix_irqs() exit (OK)
>ixgbevf 0000:00:06.0: ixgbevf_request_irq() exit (0)
>ixgbevf 0000:00:06.0: ixgbevf_irq_enable() entry queues=1 flush=1
>ixgbevf 0000:00:06.0: ixgbevf_irq_enable() exit
>ixgbevf 0000:00:06.0: ixgbevf_open() exit (OK)
>ixgbevf 0000:00:06.0: ixgbevf_set_rx_mode() entry mc_count=1
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() entry
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() exit addr=ffff810001a53148
>ixgbevf 0000:00:06.0: ixgbevf_set_rx_mode() exit
>ixgbevf 0000:00:06.0: ixgbevf_get_stats() entry / exit
>ixgbevf 0000:00:06.0: ixgbevf_set_rx_mode() entry mc_count=2
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() entry
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() exit addr=ffff8100339e9f88
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() entry
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() exit addr=ffff810001a53148
>ixgbevf 0000:00:06.0: ixgbevf_set_rx_mode() exit
>8021q: adding VLAN 0 to HW filter on device __tmp1686290279
>ixgbevf 0000:00:06.0: ixgbevf_vlan_rx_add_vid() entry vid=0
>ixgbevf 0000:00:06.0: ixgbevf_vlan_rx_add_vid() set_vfta=0
>ixgbevf 0000:00:06.0: ixgbevf_vlan_rx_add_vid() exit
>ixgbevf 0000:00:06.0: ixgbevf_set_rx_mode() entry mc_count=3
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() entry
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() exit addr=ffff8100339e9f48
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() entry
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() exit addr=ffff8100339e9f88
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() entry
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() exit addr=ffff810001a53148
>ixgbevf 0000:00:06.0: ixgbevf_set_rx_mode() exit
>ixgbevf 0000:00:06.0: ixgbevf_set_rx_mode() entry mc_count=3
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() entry
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() exit addr=ffff8100339e9f48
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() entry
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() exit addr=ffff8100339e9f88
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() entry
>ixgbevf 0000:00:06.0: ixgbevf_addr_list_itr() exit addr=ffff810001a53148
>ixgbevf 0000:00:06.0: ixgbevf_set_rx_mode() exit
>general protection fault: 0000 [1] SMP 
>last sysfs file: /block/ram0/dev
>CPU 0 
>Modules linked in: ip_conntrack_netbios_ns ipt_REJECT xt_state ip_conntrack nfnetlink iptable_filter ip_tables ip6t_REJECT xt_tcpudp ip6table_filter ip6_tables x_tables ipv6 xfrm_nalgo crypto_api dm_multipath scsi_dh video backlight sbs power_meter hwmon i2c_ec dell_wmi wmi button battery asus_acpi acpi_memhotplug ac parport_pc lp parport floppy joydev xen_vnif xen_vbd xen_balloon xen_platform_pci e1000 ide_cd ixgbevf cdrom serio_raw 8021q i2c_piix4 pcspkr i2c_core tpm_tis tpm tpm_bios dm_raid45 dm_message dm_region_hash dm_mem_cache dm_snapshot dm_zero dm_mirror dm_log dm_mod ata_piix libata sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd
>Pid: 1915, comm: ifconfig Not tainted 2.6.18-308.20.1.el5.bz862862_ixgbevf_crash_4 #1
>RIP: 0010:[<ffffffff8820213d>]  [<ffffffff8820213d>] :ixgbevf:ixgbevf_maybe_stop_tx+0xc/0x7b
>RSP: 0018:ffffffff804b1d38  EFLAGS: 00010246
>RAX: 0000000000000000 RBX: ffff81003fd62000 RCX: 0000000000000001
>RDX: 0000000000000000 RSI: ffff0078626d3a39 RDI: ffff81003fd62000
>RBP: ffff81003fd62000 R08: 0000000000000000 R09: 000000000000004c
>R10: 0000000080000000 R11: ffffffff8844c9c0 R12: 0000000000000001
>R13: ffff81003e8ce1c0 R14: ffff81003fd62000 R15: ffff0078626d3a39
>FS:  00002ab0d1bdb710(0000) GS:ffffffff80432000(0000) knlGS:0000000000000000
>CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>CR2: 00002ab0d1556a9c CR3: 000000003dfaa000 CR4: 00000000000006a0
>Process ifconfig (pid: 1915, threadinfo ffff81003dbb6000, task ffff81003d7067e0)
>Stack:  ffff81003fd62000 ffff81003e8ce1c0 ffffffff805c60d0 ffffffff8820690e
> ffff81003fd62000 ffff81003fd62500 00000000804b1e90 ffffffff8844c9f4
> 0000000100000000 ffffffff8844a3b0 ffff81003fd62000 00000000805ba218
>Call Trace:
> <IRQ>  [<ffffffff8820690e>] :ixgbevf:ixgbevf_xmit_frame+0xe8/0x8b1
> [<ffffffff8023a081>] dev_hard_start_xmit+0x1b7/0x28a
> [<ffffffff8024a246>] __qdisc_run+0x136/0x1f6
> [<ffffffff883e07eb>] :ipv6:mld_dev_queue_xmit+0x0/0xc4
> [<ffffffff8002f701>] dev_queue_xmit+0x17e/0x2a3
> [<ffffffff883e01f8>] :ipv6:mld_sendpack+0x1fe/0x339
> [<ffffffff883e0fc7>] :ipv6:mld_ifc_timer_expire+0x1dc/0x20c
> [<ffffffff883e0deb>] :ipv6:mld_ifc_timer_expire+0x0/0x20c
> [<ffffffff8009a217>] run_timer_softirq+0x18d/0x23a
> [<ffffffff8001258d>] __do_softirq+0x89/0x133
> [<ffffffff8005e2fc>] call_softirq+0x1c/0x28
> <EOI>  [<ffffffff8006d646>] do_softirq+0x2c/0x7d
> [<ffffffff80096e9c>] local_bh_enable_ip+0x48/0x5a
> [<ffffffff80239336>] dev_change_flags+0x69/0x119
> [<ffffffff8026dbba>] devinet_ioctl+0x235/0x59c
> [<ffffffff8023021b>] sock_ioctl+0x1c1/0x1e5
> [<ffffffff80041fb1>] do_ioctl+0x21/0x6b
> [<ffffffff8002fffc>] vfs_ioctl+0x457/0x4b9
> [<ffffffff8001aed1>] remove_vma+0x5d/0x64
> [<ffffffff8004c3b5>] sys_ioctl+0x59/0x78
> [<ffffffff8005d116>] system_call+0x7e/0x83
>
>
>Code: 8b 46 20 48 89 f3 39 46 24 77 03 8b 56 1c 8b 43 24 ff c8 2b 
>RIP  [<ffffffff8820213d>] :ixgbevf:ixgbevf_maybe_stop_tx+0xc/0x7b
> RSP <ffffffff804b1d38>
> <0>Kernel panic - not syncing: Fatal exception
>

Actions: View

Attachments on bug 862862: 624960 | 629093 | 653948 | 653952 | 663166 | 663204 | 663206 | 672173 | 672220 | 674634 | 674635 | 674871 | 674973 | 675360 | 675802