Bug 1224486

Summary:	virt-resize should preserve GPT partition UUIDs, else EFI guests become unbootable
Product:	Red Hat Enterprise Linux 7	Reporter:	Richard W.M. Jones <rjones>
Component:	libguestfs	Assignee:	Richard W.M. Jones <rjones>
Status:	CLOSED ERRATA	QA Contact:	Virtualization Bugs <virt-bugs>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	7.1	CC:	huzhan, leiwang, lersek, linl, ptoscano, rbalakri, wshi
Target Milestone:	rc
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	libguestfs-1.28.1-1.23.el7	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1189284	Environment:
Last Closed:	2015-11-19 07:01:12 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1189284
Bug Blocks:

Description Richard W.M. Jones 2015-05-23 15:25:25 UTC

+++ This bug was initially created as a clone of Bug #1189284 +++

Description of problem:

Rawhide host (I have patched qemu with UEFI support).
RHELSA 7.1 guest.
libvirt which includes <loader/>, <nvram/> support.

When I try to boot the RHELSA guest, it fails with an
exception in UEFI.  The attached boot.log gives the
complete output.

Version-Release number of selected component (if applicable):

Host:
libvirt-1.2.11-1.fc22.aarch64
AAVMF-20141113-3.git77d5dac.sa1.4.aarch64
kernel 3.19.0-0.rc5.git2.1.fc22.aarch64

Guest:
RHELSA 7.1

How reproducible:

100%

The qemu command line is:

/usr/bin/qemu-system-aarch64 -name packstack-rhelsa -S -machine virt,accel=kvm,usb=off -cpu host -drive file=/usr/share/AAVMF/AAVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/libvirt/nvram/packstack-rhelsa_VARS.fd,if=pflash,format=raw,unit=1 -m 12288 -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid f3930280-b152-4343-9d48-a924ed342fa4 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/packstack-rhelsa.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device virtio-serial-device,id=virtio-serial0 -usb -drive file=/dev/vg_hdd/packstack-rhelsa,if=none,id=drive-virtio-disk0,format=raw,cache=writeback -device virtio-blk-device,scsi=off,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0 -device virtio-net-device,netdev=hostnet0,id=net0,mac=52:54:00:73:d4:aa -serial pty -msg timestamp=on

--- Additional comment from Richard W.M. Jones on 2015-02-04 16:35:34 EST ---



--- Additional comment from Richard W.M. Jones on 2015-02-04 16:38:33 EST ---

It would be really useful if there was a tool for inspecting
these nvram files, eg. to list out the values of variables
inside.  hexdump is not very helpful.

--- Additional comment from Richard W.M. Jones on 2015-02-05 02:40:49 EST ---

By a process of elimination I found out that this is actually
caused by resizing the image (ie. virt-resize) while building.

I've no idea why virt-resize would affect the EFI partition (it
doesn't resize it), nor why EFI would then crash.

In any case, reassigning the bug to libguestfs.

--- Additional comment from Richard W.M. Jones on 2015-02-05 02:48:34 EST ---

Partition layout before resizing (this guest boots OK):

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          198655   96.0 MiB    EF00  EFI System Partition
   2          198656         1222655   500.0 MiB   0700  
   3         1222656         2482175   615.0 MiB   8200  
   4         2482176        12578815   4.8 GiB     0700  

Partition layout after resizing (this guest fails to boot):

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          198655   96.0 MiB    EF00  EFI
   2          198656         1222655   500.0 MiB   0700  primary
   3         1222656         2482175   615.0 MiB   8200  primary
   4         2482176        20968831   8.8 GiB     0700  primary

The first (EFI) partition has a different name, but the
same boundaries etc.

--- Additional comment from Richard W.M. Jones on 2015-02-05 03:00:05 EST ---

Using gdisk I can get the detailed partition info for the partition:

Before:
Partition GUID code: C12A7328-F81F-11D2-BA4B-00A0C93EC93B (EFI System)
Partition unique GUID: 6A3C69C8-FD2E-41B7-A2DC-0FFDB0B1FA34
First sector: 2048 (at 1024.0 KiB)
Last sector: 198655 (at 97.0 MiB)
Partition size: 196608 sectors (96.0 MiB)
Attribute flags: 0000000000000000
Partition name: 'EFI System Partition'

After:
Partition GUID code: C12A7328-F81F-11D2-BA4B-00A0C93EC93B (EFI System)
Partition unique GUID: A8ADB729-77FC-4D2B-B140-194E2BA7EE23
First sector: 2048 (at 1024.0 KiB)
Last sector: 198655 (at 97.0 MiB)
Partition size: 196608 sectors (96.0 MiB)
Attribute flags: 0000000000000000
Partition name: 'EFI'

The fields which changed are: Partition unique GUID and Partition name.

First I changed the Partition name, but that did not help.

Second I changed the Partition unique GUID (you have to use the
'x' expert menu in gdisk).  This fixes the problem.

So I conclude that virt-resize needs to preserve the partition
GUID.

--- Additional comment from Richard W.M. Jones on 2015-02-05 03:41:31 EST ---

Patch series posted upstream:
https://www.redhat.com/archives/libguestfs/2015-February/msg00032.html

--- Additional comment from Richard W.M. Jones on 2015-02-06 05:31:28 EST ---

Upstream commits:
https://github.com/libguestfs/libguestfs/commit/b3e3750b13a96fb6d1b79f7c0dabb4eeb37de5ef
https://github.com/libguestfs/libguestfs/commit/40c133b2c81666f6dde43704e66bf59206d5c111
https://github.com/libguestfs/libguestfs/commit/f630677c14c7d5528e1ab39e4f805e0957b2ee3e

Fix included in:
libguestfs-1.29.24-3.fc22
libguestfs-1.28.1-1.23.aa7a

--- Additional comment from Laszlo Ersek on 2015-05-23 07:18:33 EDT ---

Hi Rich,

I completely forgot about this BZ (and the fact that I CC'd myself on it), but last night I ran into the same issue, and now I found the BZ again with Google.

I tried to resize the C: partition (/dev/sda4) of a Windows Server 2012 R2 guest (x86_64, OVMF) on my RHEL-7.1 laptop. The UEFI boot option was lost (in retrospect surely because of the GUID change), but interestingly, the GUID seems to be hardcoded in other parts of Windows as well, because it rejected to boot even after I recreated the UEFI boot option manually, in OVMF. At that point the UEFI boot loader started, but later it encountered an error. A repair attempt with the installer ISO failed too.

So, my question here -- any chance this fix could be ported to RHEL-7.2? Thanks.

Comment 1 Richard W.M. Jones 2015-05-23 15:27:15 UTC

It's basically a trivial backport.  The only requirement is 'sgdisk'
which is already included in RHEL since 7.0.

Comment 2 Laszlo Ersek 2015-05-23 22:26:43 UTC

Thanks for creating this clone.

In the meantime I played a little bit more with my Windows 8 disk image
(earlier I named Windows Server 2012 R2, but then I moved on to another VM
of mine; in any case these two OSes should be identical for this purpose).

So, I think I figured out a set of operations that can make it work. I don't
claim each and every one of these ops is necessary, but together they are
certainly sufficient.

So, I had a 20GB win8 disk image, and I wanted to resize it to 40GB. First I
converted the src image from qcow2 to raw (because I had no better idea for
poking at the image, with "losetup" and "gdisk"), then resized the raw image
with virt-resize, then manually copied some attributes from the src raw
image to the target (resized) raw image, using gdisk. (Finally I converted
the target image to qcow2.)

The virt-resize command I used is part of:
libguestfs-tools-c-1.28.1-1.18.el7.x86_64

So, these were the attributes of the source image (printed with the "p" and
"i" gdisk commands) -- click "Expand All Comments" near the top, the lines
are a bit wide:

> GPT fdisk (gdisk) version 0.8.6
>
> Partition table scan:
>   MBR: protective
>   BSD: not present
>   APM: not present
>   GPT: present
>
> Found valid GPT with protective MBR; using GPT.
>
> Command (? for help): p
> Disk /dev/loop3: 41943040 sectors, 20.0 GiB
> Logical sector size: 512 bytes
> Disk identifier (GUID): 839D053F-53B2-4914-8337-E90F4B6168FB
> Partition table holds up to 128 entries
> First usable sector is 34, last usable sector is 41943006
> Partitions will be aligned on 2048-sector boundaries
> Total free space is 4029 sectors (2.0 MiB)
>
> Number  Start (sector)    End (sector)  Size       Code  Name
>    1            2048          616447   300.0 MiB   2700  Basic data partition
>    2          616448          819199   99.0 MiB    EF00  EFI system partition
>    3          819200         1081343   128.0 MiB   0C01  Microsoft reserved part
>    4         1081344        41940991   19.5 GiB    0700  Basic data partition
>
> Command (? for help): i
> Partition number (1-4): 1
> Partition GUID code: DE94BBA4-06D1-4D40-A16A-BFD50179D6AC (Windows RE)
> Partition unique GUID: 2BA22933-4E37-4A2B-BB43-6C04D240F23E
> First sector: 2048 (at 1024.0 KiB)
> Last sector: 616447 (at 301.0 MiB)
> Partition size: 614400 sectors (300.0 MiB)
> Attribute flags: 8000000000000001
> Partition name: 'Basic data partition'
>
> Command (? for help): i
> Partition number (1-4): 2
> Partition GUID code: C12A7328-F81F-11D2-BA4B-00A0C93EC93B (EFI System)
> Partition unique GUID: 2F675DC7-6443-460F-BA95-66842D6D060B
> First sector: 616448 (at 301.0 MiB)
> Last sector: 819199 (at 400.0 MiB)
> Partition size: 202752 sectors (99.0 MiB)
> Attribute flags: 8000000000000000
> Partition name: 'EFI system partition'
>
> Command (? for help): i
> Partition number (1-4): 3
> Partition GUID code: E3C9E316-0B5C-4DB8-817D-F92DF00215AE (Microsoft reserved)
> Partition unique GUID: 6E1C74A7-BBA2-4402-8AC4-4AC2D3D976DB
> First sector: 819200 (at 400.0 MiB)
> Last sector: 1081343 (at 528.0 MiB)
> Partition size: 262144 sectors (128.0 MiB)
> Attribute flags: 8000000000000000
> Partition name: 'Microsoft reserved partition'
>
> Command (? for help): i
> Partition number (1-4): 4
> Partition GUID code: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 (Microsoft basic data)
> Partition unique GUID: 45A56CDC-48B1-4DC8-A1CE-2D3C6EF93CEB
> First sector: 1081344 (at 528.0 MiB)
> Last sector: 41940991 (at 20.0 GiB)
> Partition size: 40859648 sectors (19.5 GiB)
> Attribute flags: 0000000000000000
> Partition name: 'Basic data partition'
>
> Command (? for help): q

First of all, the "p" command of the gdisk utility is somewhat lossy; the
"Microsoft reserved part" name is truncated (for display only). In the GUID
partition entry format
<http://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_entries>, 36
UTF-16LE code units are available for the partition name, and "Microsoft
reserved partition" certainly fits. Anyway, that's a gdisk problem, plus the
"i" command prints the name fine.

Second, I found that while virt-resize had correctly done its "basic" job
(logically speaking -- I'm sure there's nothing basic about the
implementation), it had lost quite a few metadata in the target image:

Broken field                                       I set it manually from
                                                   the source GPT, with
                                                   the gdisk command
------------------------------------------------   -------------------------
"Disk identifier (GUID)" was randomized            experts' menu, "g"
"Partition name" was truncated at first space      normal menu, "c"
"Partition unique GUID" was randomized             experts' menu, "c"
Partition's "Attribute flags" were zero            experts' menu, "a"

(The "expert command" "a" works by toggling bits individually in the 64-bit
attribute mask of the selected partition.)

The first issue was disk-level, the other three had to be checked for each
partition separately.

After these changes I could boot windows 8 just fine.

Again, I'm unsure if all of these changes were necessary to keep Windows
happy, but they were sufficient -- I sought to eliminate any changes (except
those related to the resize) between the source GPT and the target GPT.

Thanks.

Comment 3 Richard W.M. Jones 2015-05-29 11:20:14 UTC

Ha!  Turns out this is already fixed in libguestfs 1:1.28.1-1.23
(which is post-RHEL 7.1, but was included sort of accidentally in the
RHELSA package that we didn't really ship to any customers).

Anyway, makes it easy ...

Comment 5 Richard W.M. Jones 2015-09-11 10:50:27 UTC

I just checked upstream virt-resize (libguestfs-tools-c-1.31.5-1.fc24.x86_64)
and it now preserves correctly:

- partition GUID code
- partition unique GUID (bug 1189284)
- attribute flags

It changes:

- partition name (doesn't seem to matter)
- disk identifier GUID

I think the problem with resizing images was caused because the
partition unique GUID changed.  As that is now preserved, I think
we're OK.

No one has reported any other problem with resizing Windows 8 EFI
disks, so I guess changing the disk identifier GUID doesn't matter.

Comment 6 Hu Zhang 2015-09-16 12:25:45 UTC

Verified with the package:
libguestfs-1.28.1-1.55.el7.aarch64

Verify steps:
1. resize the original images.
# virt-resize --resize /dev/sda1=+200M --resize /dev/sda2=+200M --expand /dev/sda3 rhelsa-72.raw rhelsa-72-new.raw

2. Check the original image partitions' GUID.
# gdisk 
GPT fdisk (gdisk) version 0.8.6

Type device filename, or press <Enter> to exit: rhelsa-72.raw
Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk rhelsa-72.raw.bak: 20971520 sectors, 10.0 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 49910C84-1C72-4B41-9BFC-89E40AC6D347
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 20971486
Partitions will be aligned on 2048-sector boundaries
Total free space is 4029 sectors (2.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          411647   200.0 MiB   EF00  EFI System Partition
   2          411648         1435647   500.0 MiB   0700  
   3         1435648        20969471   9.3 GiB     8E00  

Command (? for help): i
Partition number (1-3): 1
Partition GUID code: C12A7328-F81F-11D2-BA4B-00A0C93EC93B (EFI System)
Partition unique GUID: BC06255A-8298-4B5D-AA05-58BB9A97C0A6
First sector: 2048 (at 1024.0 KiB)
Last sector: 411647 (at 201.0 MiB)
Partition size: 409600 sectors (200.0 MiB)
Attribute flags: 0000000000000000
Partition name: 'EFI System Partition'

Command (? for help): i
Partition number (1-3): 2
Partition GUID code: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 (Microsoft basic data)
Partition unique GUID: 8A617DA3-B289-42EF-91BD-9D92CD1B4949
First sector: 411648 (at 201.0 MiB)
Last sector: 1435647 (at 701.0 MiB)
Partition size: 1024000 sectors (500.0 MiB)
Attribute flags: 0000000000000000
Partition name: ''

Command (? for help): i
Partition number (1-3): 3
Partition GUID code: E6D6D379-F507-44C2-A23C-238F2A3DF928 (Linux LVM)
Partition unique GUID: 5D9AE9E1-D549-4991-BF19-3334D6CC9739
First sector: 1435648 (at 701.0 MiB)
Last sector: 20969471 (at 10.0 GiB)
Partition size: 19533824 sectors (9.3 GiB)
Attribute flags: 0000000000000000
Partition name: ''

3. Check the new image partitions' GUID.
# gdisk 
GPT fdisk (gdisk) version 0.8.6

Type device filename, or press <Enter> to exit: rhelsa-72-new.raw
Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk rhelsa-72-new: 23068672 sectors, 11.0 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 31E20D66-8D5E-4410-9A63-E023C948EACF
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 23068638
Partitions will be aligned on 2048-sector boundaries
Total free space is 4541 sectors (2.2 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          821247   400.0 MiB   EF00  EFI
   2          821248         2254847   700.0 MiB   0700  primary
   3         2254848        23066111   9.9 GiB     8E00  primary

Command (? for help): i
Partition number (1-3): 1
Partition GUID code: C12A7328-F81F-11D2-BA4B-00A0C93EC93B (EFI System)
Partition unique GUID: BC06255A-8298-4B5D-AA05-58BB9A97C0A6
First sector: 2048 (at 1024.0 KiB)
Last sector: 821247 (at 401.0 MiB)
Partition size: 819200 sectors (400.0 MiB)
Attribute flags: 0000000000000000
Partition name: 'EFI'

Command (? for help): i
Partition number (1-3): 2
Partition GUID code: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7 (Microsoft basic data)
Partition unique GUID: 8A617DA3-B289-42EF-91BD-9D92CD1B4949
First sector: 821248 (at 401.0 MiB)
Last sector: 2254847 (at 1.1 GiB)
Partition size: 1433600 sectors (700.0 MiB)
Attribute flags: 0000000000000000
Partition name: 'primary'

Command (? for help): i
Partition number (1-3): 3
Partition GUID code: E6D6D379-F507-44C2-A23C-238F2A3DF928 (Linux LVM)
Partition unique GUID: 5D9AE9E1-D549-4991-BF19-3334D6CC9739
First sector: 2254848 (at 1.1 GiB)
Last sector: 23066111 (at 11.0 GiB)
Partition size: 20811264 sectors (9.9 GiB)
Attribute flags: 0000000000000000
Partition name: 'primary'

4. check if the new image is bootable. Print the disk info.
# fdisk -l
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.

Disk /dev/sda: 11.8 GB, 11811160064 bytes, 23068672 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: gpt


#         Start          End    Size  Type            Name
 1         2048       821247    400M  EFI System      EFI
 2       821248      2254847    700M  Microsoft basic primary
 3      2254848     23066111    9.9G  Linux LVM       primary

Disk /dev/mapper/rhel-root: 8883 MB, 8883535872 bytes, 17350656 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/rhel-swap: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

So from step2 and step3, the partition GUID code, partition unique GUID and attribute flags are not change. The partition name is changed.
And the image after resize is bootable. So verified.

Comment 8 errata-xmlrpc 2015-11-19 07:01:12 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2183.html