Bug 607294

This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

libvirt-0_8_1-11_el6 has been built in RHEL-6-candidate with the fix.

Dave

I test with the following steps:

1,  setup a tftp server, edit /etc/xinetd.d/tftp file , set "disable = no" to enable tftp server

2, restart xinetd service /etc/init.d/xinetd restart

3, cp /usr/share/syslinux/pxelinux.0 /var/lib/tftpboot

4, dowload vmlinuz and initrd.img under the /var/lib/tftpboot folder

 example: http://download.englab.nay.redhat.com/pub/rhel/rel-eng/RHEL6.0-Beta1-4.1/6.0/Server/x86_64/os/images/pxeboot/

5, mkdir /var/lib/tftpboot/pxelinux.cfg

6, create a file name default under the pxelinux.cfg folder

7, edit the file with the contents

DISPLAY boot.txt
DEFAULT rhel6
LABEL rhel6
        kernel vmlinuz
        append initrd=initrd.img
PROMPT 1
TIMEOUT 0

8, create a new virtual network , the network.xml description is as follows

<network>
  <name>netboot</name>
  <bridge name="virbr1" stp='off' delay='1'/>
  <forward/>
  <ip address="192.168.10.1" netmask="255.255.255.0">
    <tftp root="/var/lib/tftproot" />
    <dhcp>
      <range start="192.168.10.2" end="192.168.10.254" />
      <bootp file="pxelinux.0" />
    </dhcp>
  </ip>
</network>

9, define and start network

#virsh net-define network.xml
#virsh net-start netboot
#virsh net-list --all
Name                 State      Autostart
-----------------------------------------
default              active     yes       
netboot              active     no       

#virsh dumpxml pxe2
<domain type='kvm' id='11'>
  <name>pxe2</name>
  <uuid>3fa76e60-75a9-339e-90a8-f97b7f6b0de8</uuid>
  <memory>2048288</memory>
  <currentMemory>2048000</currentMemory>
  <vcpu>2</vcpu>
  <os>
    <type arch='x86_64' machine='rhel6.0.0'>hvm</type>
    <boot dev='network'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source file='/var/lib/libvirt/images/pxe.img'/>
      <target dev='hda' bus='ide'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' unit='0'/>
    </disk>
    <controller type='ide' index='0'>
      <alias name='ide0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:e6:0b:9e'/>
      <source network='netboot'/>
      <target dev='vnet3'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/6'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/6'>
      <source path='/dev/pts/6'/>
      <target port='0'/>
      <alias name='serial0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='5903' autoport='yes'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='selinux'>
    <label>system_u:system_r:svirt_t:s0:c295,c957</label>
    <imagelabel>system_u:object_r:svirt_image_t:s0:c295,c957</imagelabel>
  </seclabel>
</domain>

#iptables -I INPUT -i virbr1 -p udp -m udp --dport 69 -j ACCEPT
#virsh start pxe2

screen output shown in the attachment : no bootable device

Is these steps right? If it is right, it seems that there are also some problems.

Created attachment 435536 [details]
pxe-tftp boot failed

sorry, forget to list the components version:
[root@redhat ~]# rpm -qa libvirt
libvirt-0.8.1-20.el6.x86_64
[root@redhat ~]# uname -r
2.6.32-54.el6.x86_64

First, for libvirt virtual network, tftp is provided by dnsmasq so keep /etc/xinetd.d/tftp disabled, I guess it even conflicts here, check /var/log/messages.

Also, on F13 I had to have tftproot under /var/lib/dnsmasq/ to get the correct selinux context.

Hi Alan,
I want to ask where I can get the livecd-iso-to-pxeboot and how to use livecd-iso-to-pxeboot to create a tftp root directory for a LiveCD image? I think I did wrong steps before. I just boot with default tftproot.

livecd-iso-to-pxeboot is provided by livecd-tools RPMs
Usage: livecd-iso-to-pxeboot <isopath>

Hi Alan,
Could you please give me a detail steps on testing? I am in rhel6 for testing it. After I set /etc/xinetd.d/tftp disabled, the error is same as before. I don't know what's wrong with my testing steps. Thanks very much

I have tested on F13 using libvirt-0.8.2-2.fc13
and <tftp root='/var/lib/dnsmasq/tftpboot' /> in "default" network.

First, confirm tftp port 69 is opened:
# virsh net-destroy default
Network default destroyed

# iptables -S INPUT|grep virbr

# virsh net-start default
Network default started

# iptables -S INPUT|grep virbr
-A INPUT -i virbr0 -p udp -m udp --dport 69 -j ACCEPT 
-A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT 
-A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT 
-A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT 
-A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT 

Then generate pxe image (it can be any, not just livecd):
* cd /var/lib/dnsmasq/
* livecd-iso-to-pxeboot /tmp/rhev-hypervisor.iso
 (rhevh is convenient since it's only ~100MB size)
* virsh start VM (with <boot dev='network'/>)
  see gPXE initializing and loading PXELINUX then vmlinuz and initrd...

You probably hit bug 594682 in your testing. It works on F13 becuase it contains newer gpxe-roms-qemu package. However, that bug has a known workaround which is covered by bug 623951. Could you retest, once this workaround bug is fixed?

libvirt-0.8.1-25.el6 has been built in RHEL-6-candidate with the fix
for issue 623951 so testing should be retried with that version,

Daniel

Verify this bug with libvirt-0.8.1-25.el6.x86_64, and PASSED.

1. cp /usr/share/syslinux/pxelinux.0 /var/lib/tftpboot

2. Dowload vmlinuz and initrd.img under the /var/lib/tftpboot folder, example:
http://download.englab.nay.redhat.com/pub/rhel/rel-eng/RHEL6.0-20100811.2/6.0/Server/x86_64/os/images/pxeboot/

3. mkdir /var/lib/tftpboot/pxelinux.cfg

4. Create a file named "default" under the pxelinux.cfg folder with the following lines.

DISPLAY boot.txt
DEFAULT rhel6
LABEL rhel6
        kernel vmlinuz
        append initrd=initrd.img
PROMPT 1
TIMEOUT 0

5. Create a new virtual network with the following xml:

<network>
  <name>netboot</name>
  <bridge name="virbr1" stp='off' delay='1'/>
  <forward/>
  <ip address="192.168.10.1" netmask="255.255.255.0">
    <tftp root="/var/lib/tftpboot" />
    <dhcp>
      <range start="192.168.10.2" end="192.168.10.254" />
      <bootp file="pxelinux.0" />
    </dhcp>
  </ip>
</network>

6. Define and start network
# virsh net-define network.xml
# virsh net-start netboot

7. Check iptable rules, and dnsmasq process.
# service iptables status
...
Table: filter
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:69 
2    ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:53 
3    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:53 
4    ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:67 
5    ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:67 
...

tftp prot - 69 is allowed.

# ps -ef | grep dns |grep netboot
nobody   18370     1  0 19:03 ?        00:00:00 /usr/sbin/dnsmasq --strict-order --bind-interfaces --pid-file=/var/run/libvirt/network/netboot.pid --conf-file=  --listen-address 192.168.10.1 --except-interface lo --dhcp-range 192.168.10.2,192.168.10.254 --dhcp-lease-max=253 --dhcp-no-override --enable-tftp --tftp-root /var/lib/tftpboot --dhcp-boot pxelinux.0

The workaround for 623951 is fixed, since "--dhcp-no-override" is seen here.

8. Set selinux to permissive
# setenforce 0

9. Install VM from pxe.
# virt-install --name pxe --ram 1024 --disk path=/var/lib/libvirt/images/pxe.img,size=4 --pxe --network network=netboot

Installation is started successfully.

This bug is verified successfully, so needinfo flag is removed.

Verified with libvirt-0.8.1-27.el6.x86_64 & qemu-kvm-0.12.1.2-2.113.el6.x86_64.

# service iptables status
...
Table: filter
Chain INPUT (policy ACCEPT)
num  target     prot opt source               destination         
1    ACCEPT     udp  --  0.0.0.0/0            0.0.0.0/0           udp dpt:69 
...

Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.