I don't think this should be handled in libvirt.  We are trying to enable most of the options that the hypervisor supports.  We cannot hence check if all settings make sense for each possible scenario.  In this case the user can use different shared memory for the application.  Whether this is part of bigger deployment or not, the management application (or user) should create configuration sensible for its use case.

Let me know if you disagree, but I'm most probably going to close this as a NOTABUG.

"In this case the user can use different shared memory for the application." I don't know what that means, can you explain? Without guest-host shared memory configured in the XML, what else can the user do for their vhostuser ports to start working? Is there some post-start command or switch? AFAIK, none exists - but if it does that would be good to know because it would mean no warning is needed. However, without such a post-start command or switch, the user has configured network interfaces which can never work.

Please let me know if I'm mistaken.

I think you might be able to use ivshmem for that as well (element <shmem/> in libvirt's XML), but even if not, that can happen in the future and we would limit user and ourselves in that regard.  As a compromise, I think the best option would be to emit a warning in the log, so users searching for that will find it, but on the other hand it will not limit them in doing so.  It *may* be also usable for testing, etc.

"As a compromise, I think the best option would be to emit a warning in the log,"

No compromise needed - the warning log is what I want :)

Patch proposed upstream:

https://www.redhat.com/archives/libvir-list/2015-December/msg00306.html

Fixed upstream by v1.3.0-62-g686eb7a24f7d:

commit 686eb7a24f7d71048f5cd45bb0095a63d0787875
Author: Martin Kletzander <mkletzan>
Date:   Tue Dec 8 14:15:47 2015 +0100

    qemu: Warn when using vhost-user without shared memory

Verified with packages:
qemu-kvm-rhev-2.6.0-19.el7.x86_64
libvirt-2.0.0-4.el7.x86_64

1. Start a VM with vhostuser network device but shared memory is not configured.
# virsh start r7
Domain r7 started
# virsh dumpxml r7
<domain type='kvm' id='6'>
  <name>r7</name>
  <uuid>ea6b3d7e-eb8e-42df-8b32-e8101d887983</uuid>
  <memory unit='KiB'>1024000</memory>
  <currentMemory unit='KiB'>1024000</currentMemory>
  <memoryBacking>
    <hugepages>
      <page size='2048' unit='KiB' nodeset='0'/>
    </hugepages>
  </memoryBacking>
  <vcpu placement='static'>5</vcpu>
  <numatune>
    <memory mode='strict' nodeset='0'/>
  </numatune>
......
  <cpu mode='host-model'>
    <model fallback='allow'/>
    <numa>
      <cell id='0' cpus='0-1' memory='1024000' unit='KiB'/>
    </numa>
  </cpu>
......
    <interface type='vhostuser'>
      <mac address='52:54:00:bb:25:74'/>
      <source type='unix' path='/var/run/openvswitch/vhost-user1' mode='client'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

2. Check there is warning in the libvirtd.log:
2016-08-11 07:04:40.987+0000: 14130: debug : qemuProcessStartValidate:4390 : Checking for any possible (non-fatal) issues
2016-08-11 07:04:40.987+0000: 14130: warning : qemuProcessStartWarnShmem:4315 : Detected vhost-user interface without any shared memory, the interface might not be operational

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2577.html