Bug 1302644 - centos 6 grub 'press any key to continue' takes a long time if no serial device attached
centos 6 grub 'press any key to continue' takes a long time if no serial devi...
Status: CLOSED NOTABUG
Product: Virtualization Tools
Classification: Community
Component: libguestfs (Show other bugs)
unspecified
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Richard W.M. Jones
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-28 05:19 EST by David Caro
Modified: 2016-06-26 19:52 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-05-02 15:19:38 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description David Caro 2016-01-28 05:19:42 EST
Description of problem:

I'm running a vm (CentOs 6.7 guest on Fedora 23 host) that should be booting from disk, but when booting, after the 'Booting from Hard Disk...' message, it takes >5min to go through all the 'Press any key to continue' messages until it really boots.

If you hit any key, the boot process goes on normally though.

The disk was generated with virt-builder centos-6

The xmldump of the domain is:
<domain type='kvm' id='40'>
  <name>d1d39aae-agent_vm</name>
  <uuid>d852708a-a7e2-46f0-8a87-9d72149fecba</uuid>
  <memory unit='KiB'>524288</memory>
  <currentMemory unit='KiB'>524288</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.4'>hvm</type>
    <bootmenu enable='yes'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='2' cores='1' threads='1'/>
  </cpu>
  <clock offset='utc'>
    <timer name='kvmclock'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/home/dcaro/Work/lago-project/demo/myprefix/images/agent_vm_root.qcow2'/>
      <backingStore type='file' index='1'>
        <format type='raw'/>
        <source file='/home/dcaro/lago/store/dcaro_test:el6-base:latest'/>
        <backingStore/>
      </backingStore>
      <target dev='vda' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <controller type='usb' index='0'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='54:52:c0:a8:ca:02'/>
      <source network='d1d39aae-lago' bridge='d1d39aae-lag'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/d1d39aae-agent_vm.org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='5900' autoport='yes' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='cirrus' vram='16384' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </memballoon>
    <rng model='virtio'>
      <backend model='random'>/dev/random</backend>
      <alias name='rng0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </rng>
  </devices>
  <seclabel type='dynamic' model='selinux' relabel='yes'>
    <label>system_u:system_r:svirt_t:s0:c560,c989</label>
    <imagelabel>system_u:object_r:svirt_image_t:s0:c560,c989</imagelabel>
  </seclabel>
</domain>


Version-Release number of selected component (if applicable):
libvirt-1.2.18.2-1.fc23.x86_64
libvirt-client-1.2.18.2-1.fc23.x86_64
libvirt-daemon-1.2.18.2-1.fc23.x86_64
libvirt-daemon-config-network-1.2.18.2-1.fc23.x86_64
libvirt-daemon-config-nwfilter-1.2.18.2-1.fc23.x86_64
libvirt-daemon-driver-interface-1.2.18.2-1.fc23.x86_64
libvirt-daemon-driver-libxl-1.2.18.2-1.fc23.x86_64
libvirt-daemon-driver-lxc-1.2.18.2-1.fc23.x86_64
libvirt-daemon-driver-network-1.2.18.2-1.fc23.x86_64
libvirt-daemon-driver-nodedev-1.2.18.2-1.fc23.x86_64
libvirt-daemon-driver-nwfilter-1.2.18.2-1.fc23.x86_64
libvirt-daemon-driver-qemu-1.2.18.2-1.fc23.x86_64
libvirt-daemon-driver-secret-1.2.18.2-1.fc23.x86_64
libvirt-daemon-driver-storage-1.2.18.2-1.fc23.x86_64
libvirt-daemon-driver-uml-1.2.18.2-1.fc23.x86_64
libvirt-daemon-driver-vbox-1.2.18.2-1.fc23.x86_64
libvirt-daemon-driver-xen-1.2.18.2-1.fc23.x86_64
libvirt-daemon-kvm-1.2.18.2-1.fc23.x86_64
libvirt-gconfig-0.2.2-1.fc23.x86_64
libvirt-glib-0.2.2-1.fc23.x86_64
libvirt-gobject-0.2.2-1.fc23.x86_64
libvirt-python-1.2.18-1.fc23.x86_64
qemu-kvm-2.4.1-5.fc23.x86_64


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 1 David Caro 2016-01-28 07:27:35 EST
When creating the vm from virt-manager, it adds a serial pty, adding that element speeds up a lot the boot (as expected)
Comment 2 David Caro 2016-01-28 07:37:47 EST
Confirmed, adding this to the devices section works around the issue:

    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
Comment 3 Cole Robinson 2016-04-10 19:26:55 EDT
I think this was a qemu regression... I swear there was a commit for it but I can't think of what to grep for in the logs. Can you reproduce with latest qemu from f23? And if that still has the issue, can you grab newer qemu from https://fedoraproject.org/wiki/Virtualization_Preview_Repository
Comment 4 David Caro 2016-05-02 07:20:55 EDT
I still have the issue on fc23 with the above repo and qemu:

$ rpm -qa qemu\*
qemu-kvm-2.6.0-0.2.rc3.fc23.x86_64
qemu-user-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-s390x-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-m68k-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-aarch64-2.6.0-0.2.rc3.fc23.x86_64
qemu-common-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-arm-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-x86-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-unicore32-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-moxie-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-cris-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-tricore-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-microblaze-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-sparc-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-ppc-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-lm32-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-sh4-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-mips-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-alpha-2.6.0-0.2.rc3.fc23.x86_64
qemu-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-xtensa-2.6.0-0.2.rc3.fc23.x86_64
qemu-system-or32-2.6.0-0.2.rc3.fc23.x86_64
qemu-guest-agent-2.4.1-8.fc23.x86_64
qemu-img-2.6.0-0.2.rc3.fc23.x86_64

Tested a similar setup (vm disk created with virt-builder centos-6) and the domxml:
<domain type='kvm' id='3'>
  <name>46df9c20-lago_functional_tests_vm01</name>
  <uuid>86813091-9e3e-45a5-9dd6-a84b98d20383</uuid>
  <memory unit='KiB'>524288</memory>
  <currentMemory unit='KiB'>524288</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.6'>hvm</type>
    <bootmenu enable='yes'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <cpu mode='host-passthrough'>
    <topology sockets='2' cores='1' threads='1'/>
  </cpu>
  <clock offset='utc'>
    <timer name='kvmclock'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/home/dcaro/Work/lago-project/lago/tests/functional/fixtures/deploy/.lago/default/images/lago_functional_tests_vm01_root.qcow2'/>
      <backingStore type='file' index='1'>
        <format type='raw'/>
        <source file='/home/dcaro/lago/store/phx_repo:el6-base:latest'/>
        <backingStore/>
      </backingStore>
      <target dev='vda' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <controller type='usb' index='0'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='54:52:c0:a8:c8:02'/>
      <source network='46df-785d5195a1' bridge='46df-785d519'/>
      <target dev='vnet0'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/46df9c20-lago_functional_tests_vm01.org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='5900' autoport='yes' listen='0.0.0.0' keymap='en-us'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='cirrus' vram='16384' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='none'>
      <alias name='balloon0'/>
    </memballoon>
    <rng model='virtio'>
      <backend model='random'>/dev/random</backend>
      <alias name='rng0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </rng>
  </devices>
  <seclabel type='dynamic' model='selinux' relabel='yes'>
    <label>system_u:system_r:svirt_t:s0:c433,c921</label>
    <imagelabel>system_u:object_r:svirt_image_t:s0:c433,c921</imagelabel>
  </seclabel>
</domain>
Comment 5 Cole Robinson 2016-05-02 15:19:38 EDT
Okay, I reproduced. But I don't think this is a qemu bug, probably a centos 6 grub bug, if a bug at all.

virt-builder images have grub configured to send output to the serial console. Maybe that's the default config, I didn't confirm. Either way, I suspect grub is trying to detect serial hardware which is what causes the delay. I thought maybe it was some serial buffer filling up because there was no host side reader (which has been an issue in the past), but once it gets to the grub screen things proceed normally.

So I'm closing this as NOTABUG for the virt stack. Maybe rjones can provide more info though
Comment 6 Richard W.M. Jones 2016-05-04 10:35:46 EDT
(In reply to Cole Robinson from comment #5)
> Okay, I reproduced. But I don't think this is a qemu bug, probably a centos
> 6 grub bug, if a bug at all.
> 
> virt-builder images have grub configured to send output to the serial
> console. Maybe that's the default config, I didn't confirm. Either way, I
> suspect grub is trying to detect serial hardware which is what causes the
> delay. I thought maybe it was some serial buffer filling up because there
> was no host side reader (which has been an issue in the past), but once it
> gets to the grub screen things proceed normally.
> 
> So I'm closing this as NOTABUG for the virt stack. Maybe rjones can provide
> more info though

I think it's because we create the base images using a serial
console, so anaconda configures one.  Then when virt-builder creates
a disk image from this template and the user boots it, it will still
have the serial console.

eg: https://github.com/libguestfs/libguestfs/blob/master/builder/website/centos.sh#L94

I had seen this problem was puzzled by it before, so at least it's good
to see the analysis of why it happens.

I guess the solutions include:

 - somehow(?) remove the serial console in virt-builder

 - fix GRUB so it doesn't hang for 5 minutes when there's no serial console

 - users can add a serial console

Note You need to log in before you can comment on or make changes to this bug.