Bug 1527740

Summary: NVRAM/pflash on a backing device is not created in the mount namespace
Product: Red Hat Enterprise Linux 7 Reporter: Meina Li <meili>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: Meina Li <meili>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.5CC: gveitmic, hhan, jdenemar, jiyan, lmen, mprivozn, pkrempa, xuzhang, yafu, yisun
Target Milestone: rcKeywords: Regression, Upstream
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-4.3.0-1.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-30 09:52:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Meina Li 2017-12-20 02:44:08 UTC
Description of problem:
Guest start failed when allow block device for NVRAM backing Store.

Version-Release number of selected component (if applicable):
libvirt-3.9.0-6.el7.x86_64


How reproducible:
100%

Steps to Reproduce:
1. Create a block device named nvram.
# dmsetup create nvram --table="0 256 linear /dev/sdb1 0"
# ll /dev/mapper/
total 0
crw-------. 1 root root 10, 236 Mar 31 22:25 control
lrwxrwxrwx. 1 root root       7 Mar 31 22:38 nvram -> ../dm-0

2. Define a guest using the xml:
...
  <os>
    <type arch='x86_64' machine='pc-q35-rhel7.5.0'>hvm</type>
    <loader readonly='yes' secure='no' type='pflash'>/usr/share/OVMF/OVMF_CODE.secboot.fd</loader>
    <nvram template='/usr/share/OVMF/OVMF_VARS.fd'>/dev/mapper/nvram</nvram>
    <bootmenu enable='yes' timeout='3000'/>
    <smbios mode='sysinfo'/>
  </os>
...
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/var/lib/libvirt/images/lmo.qcow2'/>
      <target dev='sda' bus='sata'/>
      <boot order='1'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
...

3. Start the guest.
# virsh start lmo
error: Failed to start domain lmo
error: internal error: child reported: unable to stat: /dev/mapper/nvram: No such file or directory

4. Disable namespace support in /etc/libvirt/qemu.conf and restart:
# vim /etc/libvirt/qemu.conf
namespaces = [  ]
# systemctl restart libvirtd
# virsh start lmo
Domain lmo started

# virsh list --all
 Id    Name                           State
----------------------------------------------------
 2     lmo                            running
 -     lmn                            shut off
 -     rhel7                          shut off

5. Use virt-manager to check the boot up of the guest.
guest can't boot up, the display in screen is black.

Actual results:
As above step 3 and step 5.

Expected results:
guest can be booted successfully.


Additional info for step 3:
The log of guest:
2017-12-07 03:11:44.832+0000: 26241: debug : virCommandHandshakeChild:435 : Notifying parent for handshake start on 30
2017-12-07 03:11:44.832+0000: 26241: debug : virCommandHandshakeChild:443 : Waiting on parent for handshake complete on 31
libvirt:  error : libvirtd quit during handshake: Input/output error
2017-12-07 03:11:44.893+0000: shutting down, reason=failed

Comment 2 Peter Krempa 2017-12-20 10:12:13 UTC
This is a regression from introduction of the mount namespace support.

Comment 4 Peter Krempa 2017-12-20 10:15:47 UTC
(In reply to Meina Li from comment #0)

[...]

> 2. Define a guest using the xml:
> ...
>   <os>
>     <type arch='x86_64' machine='pc-q35-rhel7.5.0'>hvm</type>
>     <loader readonly='yes' secure='no'
> type='pflash'>/usr/share/OVMF/OVMF_CODE.secboot.fd</loader>
>     <nvram template='/usr/share/OVMF/OVMF_VARS.fd'>/dev/mapper/nvram</nvram>
>     <bootmenu enable='yes' timeout='3000'/>
>     <smbios mode='sysinfo'/>
>   </os>

[...]

> 
> 5. Use virt-manager to check the boot up of the guest.
> guest can't boot up, the display in screen is black.

This problem is a misconfiguration. You are using a secure boot OVMF image but you did not enable secure boot via the 'secure' attribute in the XML above. Without that the OVMF image refuses to boot.

This issue is not related to the namespace support for block device backed OVMF images.

Comment 5 Michal Privoznik 2018-01-03 07:30:18 UTC
I agree with Peter that combination of secure loader and secure='no' is not supported. However, libvirt is not creating the NVRAM store. Patch proposed upstream:

https://www.redhat.com/archives/libvir-list/2018-January/msg00045.html

Comment 6 Michal Privoznik 2018-01-09 07:37:25 UTC
I've pushed the patch upstream:

commit 058b7fd0fe7a3cd2ab805fb02ecb1bd4cd2a4df5
Author:     Michal Privoznik <mprivozn>
AuthorDate: Wed Jan 3 08:08:07 2018 +0100
Commit:     Michal Privoznik <mprivozn>
CommitDate: Tue Jan 9 08:29:51 2018 +0100

    qemu: Prepare BIOS/UEFI when starting a domain
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1527740
    
    Users might use a block device as UEFI VAR store. Or even have
    OVMF stored there. Therefore, when starting a domain and separate
    mount namespace is used, we have to create all the /dev entries
    that are configured for the domain.
    
    Signed-off-by: Michal Privoznik <mprivozn>
    Reviewed-by: John Ferlan <jferlan>

v3.10.0-140-g058b7fd0f

Comment 9 Meina Li 2018-02-27 10:04:11 UTC
Hi, Michal
For comment 4 and comment 5,  I know that combination of secure loader and secure='no' is not supported and if  we do it, the OVMF image refuses to boot. 
But when I run it with this combination, the guest can boot successfully. So whether there is no limit to this combination because it is not supported or some other reasons?
Thank you in advance.

XML format:
  <os>
    <type arch='x86_64' machine='pc-q35-rhel7.5.0'>hvm</type>
    <loader readonly='yes' secure='no' type='pflash'>/usr/share/OVMF/OVMF_CODE_secboot.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/secure_VARS.fd</nvram>
  </os>

guest:
# bootctl  status | grep Secure
Secure Boot: enabled

Comment 10 Michal Privoznik 2018-02-27 11:02:00 UTC
(In reply to Meina Li from comment #9)
> Hi, Michal
> For comment 4 and comment 5,  I know that combination of secure loader and
> secure='no' is not supported and if  we do it, the OVMF image refuses to
> boot. 
> But when I run it with this combination, the guest can boot successfully. So
> whether there is no limit to this combination because it is not supported or
> some other reasons?
> Thank you in advance.
> 
> XML format:
>   <os>
>     <type arch='x86_64' machine='pc-q35-rhel7.5.0'>hvm</type>
>     <loader readonly='yes' secure='no'
> type='pflash'>/usr/share/OVMF/OVMF_CODE_secboot.fd</loader>
>     <nvram>/var/lib/libvirt/qemu/nvram/secure_VARS.fd</nvram>
>   </os>
> 
> guest:
> # bootctl  status | grep Secure
> Secure Boot: enabled

Yeah, this is not supported. Libvirt does not examine the passed image in any way. Nor varstore. So it doesn't know whether the image has secure boot enabled or not.  The reason why bootctl reports secure boot enabled is because the OVMF image has it enabled in some table and bootctl merely just reads the table. Anyway, for more detailed reasoning I suggest contacting lersek. From Libvirt's POV this is working as expected.

Comment 12 Meina Li 2018-07-26 06:13:43 UTC
Verified on libvirt-4.5.0-4.el7.x86_64.

Test steps:

1. Create a block device named nvram.
# dmsetup create nvram --table="0 256 linear /dev/sdb1 0"
# ll /dev/mapper/
total 0
crw-------. 1 root root 10, 236 Mar 31 22:25 control
lrwxrwxrwx. 1 root root       7 Mar 31 22:38 nvram -> ../dm-0

2. Define a guest using the xml:
...
 <os>
    <type arch='x86_64' machine='pc-q35-rhel7.6.0'>hvm</type>
    <loader readonly='yes' secure='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.secboot.fd</loader>
    <nvram>/dev/mapper/nvram</nvram>
    <bootmenu enable='yes' timeout='3000'/>
  </os>
...

3. Start the guest.
# virsh start q35
Domain q35 started
# ps aux | grep qemu | grep nvram
...-drive file=/dev/mapper/nvram,if=pflash,format=raw,unit=1 ...

4. Undefine the guest.
# virsh undefine ovmf --nvram
# ll /dev/mapper/
total 0
crw-------. 1 root root 10, 236 Mar 31 22:25 control

The result is expected, so move this bug to verified.

Comment 14 errata-xmlrpc 2018-10-30 09:52:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:3113