Bug 1007759

I could reproduce this issue with:
libvirt-1.1.1-4.el7.x86_64

This is now fixed upstream by v1.2.2-292-g6dc75f5..v1.2.2-295-g585076c:

commit 6dc75f565418a169d6b86e059bcce3d0082c8e3e
Author: Jiri Denemark <jdenemar>
Date:   Thu Mar 20 13:04:06 2014 +0100

    Fix usage of virDomainDefCompatibleDevice
    
    A device needs to be checked for compatibility with the domain
    definition it corresponds to. Specifically, for VIR_DOMAIN_AFFECT_CONFIG
    case we should check against persistent def rather than active def.
    
    Signed-off-by: Jiri Denemark <jdenemar>

commit 1c13166134bd2dd1fc5711e421ac5f983bece6a6
Author: Jiri Denemark <jdenemar>
Date:   Thu Mar 20 13:39:20 2014 +0100

    Pass action to virDomainDefCompatibleDevice
    
    When checking compatibility of a device with a domain definition, we
    should know what we're going to do with the device. Because we may need
    to check for different things when we're attaching a new device versus
    detaching an existing device.
    
    Signed-off-by: Jiri Denemark <jdenemar>

commit 563a12b3463e9a665fae02589437ecd6dba89f6b
Author: Jiri Denemark <jdenemar>
Date:   Thu Mar 20 22:34:00 2014 +0100

    conf: Introduce virDomainDeviceGetInfo API
    
    The offset of virDomainDeviceInfo structure within a device definition
    varies with device type and some types do not contain the info structure
    at all. This new API makes it easier to access the info structure from a
    generic virDomainDeviceDef structure.
    
    Signed-off-by: Jiri Denemark <jdenemar>

commit 585076c7e602bb52d34fad83963035fc03652248
Author: Jiri Denemark <jdenemar>
Date:   Thu Mar 20 15:34:32 2014 +0100

    Check boot order on device attach
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1007754
    
    When attaching a new device, we need to check if its boot order
    configuration is compatible with current domain definition.
    
    Signed-off-by: Jiri Denemark <jdenemar>

*** Bug 1030789 has been marked as a duplicate of this bug. ***

I can produce this bug on libvirt-0.10.2-46.el6.x86_64

verify it on 
libvirt-1.2.8-8.el7.x86_64
qemu-kvm-rhev-2.1.2-12.el7.x86_64
kernel-3.10.0-208.el7.x86_64

verify steps:

1. prepare a guest in shut off
# virsh list --all
 Id    Name                           State
----------------------------------------------------
-     rhel7new                       shut off
# virsh dumpxml rhel7new
<domain type='kvm' id='24'>
  <name>rhel7new</name>
  <uuid>e73cdb84-e02c-4bb3-939f-f76d8d3c140a</uuid>
  <memory unit='KiB'>1048576</memory>
  <currentMemory unit='KiB'>1048576</currentMemory>
  <vcpu placement='static'>1</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-rhel7.1.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <cpu mode='custom' match='exact'>
    <model fallback='allow'>SandyBridge</model>
  </cpu>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/rhel7new.qcow2'/>
      <backingStore/>
      <target dev='hda' bus='ide'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb0'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb0'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb0'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:c2:c6:e4'/>
      <source network='default'/>
      <target dev='vnet2'/>
      <model type='rtl8139'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/10'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/10'>
      <source path='/dev/pts/10'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='spice' port='5902' autoport='yes' listen='127.0.0.1'>
      <listen type='address' address='127.0.0.1'/>
    </graphics>
    <sound model='ich6'>
      <alias name='sound0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </sound>
    <video>
      <model type='qxl' ram='65536' vram='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <redirdev bus='usb' type='spicevmc'>
      <alias name='redir0'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <alias name='redir1'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <alias name='redir2'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <alias name='redir3'/>
    </redirdev>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='selinux' relabel='yes'>
    <label>system_u:system_r:svirt_t:s0:c732,c897</label>
    <imagelabel>system_u:object_r:svirt_image_t:s0:c732,c897</imagelabel>
  </seclabel>
</domain>

2. prepare a network device xml with boot order
# cat /tmp/nic.xml 
<interface type='network'>
  <source network='default'/>
  <boot order='1'/>
</interface>

3. attach it to guest with --config, xml check error reported right
# virsh attach-device rhel7new /tmp/nic.xml --config
error: Failed to attach device from /tmp/nic.xml
error: unsupported configuration: per-device boot elements cannot be used together with os/boot elements

4.start guest success, and dumpxml without the network device
# virsh start rhel7new
Domain rhel7new started

# virsh dumpxml rhel7new |grep boot
    <boot dev='hd'/>
  <on_reboot>restart</on_reboot>

5. restart libvirtd, guest still in running and work well after login
# service libvirtd restart
Redirecting to /bin/systemctl restart  libvirtd.service

# virsh list --all
 Id    Name                           State
----------------------------------------------------
 25    rhel7new                       running

6. remove <boot dev='hd'/> and add boot order in disk from guest xml
...
 <os>
    <type arch='x86_64' machine='pc-i440fx-rhel7.1.0'>hvm</type>
  </os>
...
<disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/rhel7new.qcow2'/>
      <target dev='hda' bus='ide'/>
      <boot order='1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
...
7. prepare network device xml with boot order = 2
# cat /tmp/nic1.xml 
<interface type='network'>
   <source network='default'/>
   <boot order='2'/>
</interface>

8. attach device it to guest success with --config
# virsh attach-device rhel7new /tmp/nic1.xml --config
Device attached successfully

# virsh dumpxml rhel7new |grep boot
  <on_reboot>restart</on_reboot>
      <boot order='1'/>
      <boot order='2'/>

9. start guest success without error and login guest, work well

#  virsh start rhel7new
Domain rhel7new started

10. re attach the nic1.xml with boot order =2
# virsh attach-device rhel7new /tmp/nic1.xml --config
error: Failed to attach device from /tmp/nic1.xml
error: unsupported configuration: boot order 2 is already used by another device

11 prepare a disk device with boot order =5 and attach it with --config
# cat /tmp/disk.xml 
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/var/lib/libvirt/images/test.img'/>
<target dev='hdb' bus='ide'/>
<boot order='5'/>
</disk>
# virsh attach-device rhel7new /tmp/disk.xml --config
Device attached successfully

# virsh start rhel7new
Domain rhel7new started

# virsh dumpxml rhel7new |grep boot
  <on_reboot>restart</on_reboot>
      <boot order='1'/>
      <boot order='5'/>
      <boot order='2'/>
12. restart libvirtd again, then check guest running and works normal

# service libvirtd restart
Redirecting to /bin/systemctl restart  libvirtd.service
# virsh list --all
 Id    Name                           State
----------------------------------------------------
 28    rhel7new                       running


move this bug to verified

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0323.html