Bug 1253223 - Installing RHEL 6 from USB key writes grub device map referencing /dev/sdb (instead of /dev/sda)
Installing RHEL 6 from USB key writes grub device map referencing /dev/sdb (i...
Status: CLOSED WONTFIX
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: anaconda (Show other bugs)
6.7
x86_64 Unspecified
medium Severity medium
: rc
: ---
Assigned To: Anaconda Maintenance Team
Release Test Team
Clayton Spicer
:
: 1254139 1254857 (view as bug list)
Depends On:
Blocks: 1356047 1361708
  Show dependency treegraph
 
Reported: 2015-08-13 05:10 EDT by mxie@redhat.com
Modified: 2017-03-16 16:14 EDT (History)
14 users (show)

See Also:
Fixed In Version:
Doc Type: Known Issue
Doc Text:
The `device.map` configuration file generated by `Anaconda` is sometimes incorrect Due to limitations in the kernel, the `device.map` configuration file that is used to map BIOS drives to operating system devices might be generated incorrectly in certain situations, particularly when installing from a USB key. As a consequence, booting sometimes fails after installation. To work around this problem, manually update the `device.map` file in the `/boot/grub` directory. After updating `device.map` so that it correctly maps devices on the system, Red Hat Enterprise Linux 6 will boot as expected.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-10-20 14:15:43 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
screenshot of RHEL6.7-ISCSI-device.map (76.59 KB, image/png)
2015-08-13 05:10 EDT, mxie@redhat.com
no flags Details
Conversion log of RHEL6.7-X64-ISCSI (55.36 KB, application/zip)
2015-08-13 22:02 EDT, mxie@redhat.com
no flags Details

  None (edit)
Description mxie@redhat.com 2015-08-13 05:10:42 EDT
Created attachment 1062412 [details]
screenshot of RHEL6.7-ISCSI-device.map

Description of problem:
device.map of RHEL6.7-x86_64 guest isn't updated correctly after conversion to RHEV by virt-p2v

Version-Release number of selected component (if applicable):
virt-v2v-1.28.1-1.49.el7.x86_64
virt-p2v-1.28.1-1.49.el7-1.iso

How reproducible:
100%

Steps to Reproduce:
 1.Install rhel6.7-x86_64 on iscsi storage
 2.Boot the source machine into virt-p2v via DVD-ROM and input the ip and password of conversion server.
 3.Convert rhel6.7-x86_64 iscsi host to RHEV via virt-p2v client
 4.After conversion,boot into rhel6.7-x86_64 guest and check points on RHEV
 5.Using command #cat /boot/grub/device.map to check device.map of rhel6.7-x86_64 guest on RHEV as below,it is not updated to vda
   #this device map was generated by anaconda
    (hd0)  /dev/sdb
 6.But type command #fdisk -l to check disk, it shows "/dev/vda", pls refer to attach screenshot

Actual results:
device.map of RHEL6.7-x86_64 guest isn't updated to vda on RHEV

Expected results:
device.map of RHEL6.7-x86_64 guest should be updated to vda on RHEV 

Additional info:
Comment 2 Richard W.M. Jones 2015-08-13 11:23:44 EDT
Can you attach the debug info.  It should be found in
/tmp on the Conversion Server, in a directory called
something like:

/tmp/virt-p2v-20150813-XXXXXX/
Comment 3 mxie@redhat.com 2015-08-13 22:02:54 EDT
Created attachment 1062855 [details]
Conversion log of RHEL6.7-X64-ISCSI
Comment 4 Richard W.M. Jones 2015-08-14 04:20:25 EDT
This is basically a bug in anaconda.  During installation anaconda
wrote an incorrect device.map file, referencing /dev/sdb (instead of
/dev/sda).  This can happen when you install RHEL from a USB key, since
/dev/sda in that case is the USB key, not the hard disk.

During conversion it would have printed a warning:

libguestfs: trace: aug_get "/files/boot/grub/device.map/hd0"
guestfsd: main_loop: new request, len 0x4c
guestfsd: main_loop: proc 19 (aug_get) took 0.00 seconds
libguestfs: trace: aug_get = "/dev/sdb"
virt-v2v: warning: /files/boot/grub/device.map/hd0 references unknown 
device "sdb".  You may have to fix this entry manually after conversion.

Anyway, as long as the guest boots, I'm not very concerned about this.
Comment 5 Richard W.M. Jones 2015-08-14 04:33:08 EDT
The anaconda bug is probably:
https://bugzilla.redhat.com/show_bug.cgi?id=909582

Anyway I don't think this is a bug in virt-v2v.  The machine had
an incorrect device.map to start with.

Can you confirm this on the original host?  device.map on the
original host references /dev/sdb.  Can you confirm that the
original hard disk layout uses /dev/sda and not /dev/sdb?  (Try
running 'lsblk' on the source).
Comment 6 mxie@redhat.com 2015-08-14 06:17:31 EDT
I will reinstall OS to confirm the original rhel6.7's hard disk layout, in addition, I meet same bug as below:

Description of problem:
Device.map of RHEL7.2-x86_64 guest is also not updated correctly after conversion to RHEV by virt-p2v

Source host info:
the original OS rhel7.2 is installed on fc storage and check device.map and "lsblk "as below
#cat /boot/gurb2/device.map
  # this device map was generated by anaconda
 (hd0)      /dev/mapper/mpatha
 (hd1)      /dev/mapper/mpatha
# lsblk
NAME                                                  MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                                                     8:0    0 232.9G  0 disk  
├─sda1                                                  8:1    0   243M  0 part  
├─sda2                                                  8:2    0   244M  0 part  
├─sda3                                                  8:3    0   244M  0 part  
└─sda4                                                  8:4    0 232.1G  0 part  
  ├─HostVG-Swap                                       253:11   0   7.9G  0 lvm   
  ├─HostVG-Config                                     253:12   0     8M  0 lvm   
  ├─HostVG-Logging                                    253:13   0     2G  0 lvm   /run/media
  └─HostVG-Data                                       253:14   0 222.3G  0 lvm   /run/media
sdb                                                     8:16   0    20G  0 disk  
└─mpatha                                              253:1    0    20G  0 mpath 
  └─mpatha1                                           253:2    0    20G  0 part  /
sdc                                                     8:32   0   1.2T  0 disk  
└─mpathb                                              253:0    0   1.2T  0 mpath 
  ├─45729e32--8959--4d58--89aa--0ee0ab61dc36-metadata 253:3    0   512M  0 lvm   
  ├─45729e32--8959--4d58--89aa--0ee0ab61dc36-outbox   253:4    0   128M  0 lvm   
  ├─45729e32--8959--4d58--89aa--0ee0ab61dc36-leases   253:5    0     2G  0 lvm   
  ├─45729e32--8959--4d58--89aa--0ee0ab61dc36-ids      253:6    0   128M  0 lvm   
  ├─45729e32--8959--4d58--89aa--0ee0ab61dc36-inbox    253:7    0   128M  0 lvm   
  ├─45729e32--8959--4d58--89aa--0ee0ab61dc36-master   253:8    0     1G  0 lvm   /run/media
  ├─45729e32--8959--4d58--89aa--0ee0ab61dc36-888004a8--c6a8--41b5--a5f0--00ba94028458
                                                      253:9    0   5.9G  0 lvm   
  └─45729e32--8959--4d58--89aa--0ee0ab61dc36-6efe101f--3992--4193--a1e2--03104a1ef5a0
                                                      253:10   0   5.9G  0 lvm   
sdd                                                     8:48   0    20G  0 disk  
└─mpatha                                              253:1    0    20G  0 mpath 
  └─mpatha1                                           253:2    0    20G  0 part  /
sde                                                     8:64   0   1.2T  0 disk  
└─mpathb                                              253:0    0   1.2T  0 mpath 
  ├─45729e32--8959--4d58--89aa--0ee0ab61dc36-metadata 253:3    0   512M  0 lvm   
  ├─45729e32--8959--4d58--89aa--0ee0ab61dc36-outbox   253:4    0   128M  0 lvm   
  ├─45729e32--8959--4d58--89aa--0ee0ab61dc36-leases   253:5    0     2G  0 lvm   
  ├─45729e32--8959--4d58--89aa--0ee0ab61dc36-ids      253:6    0   128M  0 lvm   
  ├─45729e32--8959--4d58--89aa--0ee0ab61dc36-inbox    253:7    0   128M  0 lvm   
  ├─45729e32--8959--4d58--89aa--0ee0ab61dc36-master   253:8    0     1G  0 lvm   /run/media
  ├─45729e32--8959--4d58--89aa--0ee0ab61dc36-888004a8--c6a8--41b5--a5f0--00ba94028458
                                                      253:9    0   5.9G  0 lvm   
  └─45729e32--8959--4d58--89aa--0ee0ab61dc36-6efe101f--3992--4193--a1e2--03104a1ef5a0
                                                      253:10   0   5.9G  0 lvm   
sr0                                                    11:0    1     3G  0 rom   /run/media



Guest info on RHEV:
check device.map and "lsblk""as below
#cat /boot/gurb2/device.map
  # this device map was generated by anaconda
 (hd0)      /dev/mapper/mpatha
 (hd1)      /dev/mapper/mpatha
#lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0     11:0    1 1024M  0 rom  
vda    253:0    0   20G  0 disk 
└─vda1 253:1    0   20G  0 part /
Comment 7 mxie@redhat.com 2015-08-17 02:55:55 EDT
I have checked disk info of original rhel6.7_x64 which installed iSCSI as below:
#cat /boot/grub/device.map
  # this device map was generated by anaconda
 (hd0)      /dev/sdb

#lsblk
NAME   MAJ:MIN RM   SIZE  RO TYPE MOUNTPOINT
sdb     8:16    0 931.5G  0  disk  
sda     8:0     0    30G  0  disk 
└─sda1  8:1     0    30G  0  part /
sr0    11:0     1  1024M  0  rom


The rhel6.7_x64 guest's disk info on rhev as below:
#cat /boot/grub/device.map
  # this device map was generated by anaconda
 (hd0)      /dev/sdb

#lsblk
NAME   MAJ:MIN RM   SIZE  RO TYPE MOUNTPOINT
sr0    11:0     1  1024M  0  rom 
vda     8:0     0    30G  0  disk 
└─vda1  8:1     0    30G  0  part /
Comment 12 tingting zheng 2015-08-17 06:20:38 EDT
Refer to comment 4 and comment 7,move this bug to anacoda,pls correct it if I am wrong.
Comment 13 Richard W.M. Jones 2015-08-17 13:45:55 EDT
Updating the summary so it's clearer.  The bug as I understand it
is this: When you install RHEL 6 from a USB boot device, anaconda
writes a grub device map that contains:

  # this device map was generated by anaconda
 (hd0)      /dev/sdb

When the machine boots however, the hard disk is /dev/sda (because
the USB key is removed and the BIOS changes the disk slot).

Not actually sure this causes any problems, except for confusing
virt-p2v slightly.
Comment 16 Richard W.M. Jones 2015-08-24 04:39:14 EDT
*** Bug 1254857 has been marked as a duplicate of this bug. ***
Comment 17 Richard W.M. Jones 2015-09-01 10:27:20 EDT
*** Bug 1254139 has been marked as a duplicate of this bug. ***
Comment 18 Brian Lane 2015-12-09 19:56:18 EST
After looking at this a bit I don't think there's anything we can, or should, do to correct this.

In the situations described in this bug it is just a warning, so it isn't preventing the conversion from happening or from the result from booting.

When installing we have no way to know how the system will enumerate the drives when the installation media is removed. We could try guessing, but I'm worried that would then break actual cases where device.map does matter.

Looking at current Fedora and RHEL7 code they all work the same, writing useless device.map files when using removable media.

I'll cc pjones to see if he has any thoughts on this, but I'm leaning towards WONTFIX
Comment 20 Peter Jones 2015-12-10 13:39:53 EST
The kernel does not guarantee the order it enumerates block devices in *any way*, so it's never going to be possible ensure it's right.  In this case it'll need to be fixed if you ever need to re-install grub, but it's not really used during booting.  grub-install can do this with the --recheck option.

Also leaning towards WONTFIX.  At most we could provide a release note; that should probably be up to the maintainer of the RHEL 6 grub package.
Comment 24 mxie@redhat.com 2016-05-13 03:46:17 EDT
I can reproduce with builds:

P2V client:
virt-p2v-1.28.1-1.51.3.el7.1.noarch.rpm

Conversion server:
virt-v2v-1.32.4-1.el7.x86_64
libguestfs-1.32.4-1.el7.x86_64
kernel:3.10.0-394.el7.x86_64
libvirt-1.3.4-1.el7.x86_64
qemu-kvm-1.5.3-111.el7.x86_64

Steps:
1.Prepare 30G iscsi storage on the the QLE4060c iscsi card
2.Install last RHEL6.8 x86 on the iscsi storage
3.Finish os installation. make sure boot into the OS normally, check some info as below:

3.1 #lsblk
NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda      8:0    0  30G  0 disk
├─sda1   8:1    0   2G  0 part [SWAP]
└─sda2   8:2    0  28G  0 part /

3.2#uname -a
Linux bootp-73-75-38.lab.eng.pek2.redhat.com 2.6.32-642.el6.i686 #1 SMP Wed Apr 13 00:50:26 EDT 2016 i686 i686 i386 GNU/Linux

4.Boot the machine into the p2v client via iso file and input the ip and password of conversion server.

5.After conversion, check guest's device.map on RHEV as below,it is not updated to vda
 #cat /boot/grub/device.map
   #this device map was generated by anaconda
    (hd0)  /dev/sdb
6.But using commend #fdisk -l to check disk info, it shows "/dev/vda"

Disk /dev/vda: 32.1 GB, 32144097280 bytes
16 heads, 63 sectors/track, 62283 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c55ca

   Device Boot      Start         End      Blocks   Id  System
/dev/vda1               3        4066     2048000   82  Linux swap / Solaris
Partition 1 does not end on cylinder boundary.
/dev/vda2   *        4066       62284    29341696   83  Linux
Partition 2 does not end on cylinder boundary.
Comment 26 David Kaspar [Dee'Kej] 2016-09-27 04:47:29 EDT
Hello guys,

I'm inclined to close this BZ as WONTFIX.

(In reply to Peter Jones from comment #20)
> The kernel does not guarantee the order it enumerates block devices in *any
> way*, so it's never going to be possible ensure it's right.

And same applies to GRUB itself because kernel does not guarantee the order. Excerpt from 'info grub':

3.3 Installing GRUB using grub-install
======================================

*Caution:* This procedure is definitely less safe, because there are
several ways in which your computer can become unbootable. For example,
most operating systems don't tell GRUB how to map BIOS drives to OS
devices correctly--GRUB merely "guesses" the mapping. This will succeed
in most cases, but not always. Therefore, GRUB provides you with a map
file called the "device map", which you must fix if it is wrong.

Therefore, fixing something like this would actually be an enhancement/new feature of grub. Since we are in the phase2 of RHEL6 lifecycle, this is not allowed anymore.


NOTE: Anaconda will create the device.map file during installation, but the file will not be correctly applicable once the USB is removed. IMHO, anaconda does not see a diffrence between installation from CD/DVD and USB. Usually, we tend to count on the fact that people do not remove their CD/DVD mechanics so the disk layout stays the same after the installation. This, however, does not apply to installation from USB, which is usually removed after the installation.

(In reply to Peter Jones from comment #20)
> In this case
> it'll need to be fixed if you ever need to re-install grub, but it's not
> really used during booting.  grub-install can do this with the --recheck
> option.

What the --recheck option actually does is making backup of device.map file, and generating new one from current disk layout, which is "guessed". This could actually help workaround this problem:

* install RHEL6 on the machine
* remove the USB after installation
* boot the machine, run 'grub-install --recheck'
* (reboot? the machine)
* start the conversion to RHEV

I have discussed this option with mxie already. He said he will try it and let us know.
Comment 27 Richard W.M. Jones 2016-09-29 05:33:10 EDT
The problem with that theory is there is only one disk.
Comment 28 mxie@redhat.com 2016-09-30 00:02:50 EDT
Hi David,

I try to use vm to verify the command 'grub-install --recheck' because our machine is located at remote lab so that can't install os with usb CD/DVD


Steps
1.Create a new vm in virt-manager and add 2 disk, one type is scsi and another one type is sata  

2.Install rhel6.8 on scsi disk,remove the sata disk after finishing installation,then check disk info in guest as below:
# lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sr0     11:0    1     1024M  0 rom  
sda      8:0    0        9G  0 disk 
├─sda1   8:1    0      7.8G  0 part /
└─sda2   8:2    0      1.2G  0 part [SWAP]

#cat /boot/grub/device.map 
# this device map was generated by anaconda
(hd0)     /dev/sdb


3.Using command 'grub-install --recheck' to recover device.map
#grub-install --recheck /dev/sda

4.Check device.map again
#cat /boot/grub/device.map 
# this device map was generated by anaconda
(fd0)     /dev/fd0
(hd0)     /dev/sda

5.Using v2v to convert this guest
# virt-v2v rhel6.8 -on after
[   0.0] Opening the source -i libvirt rhel6.8
[   0.0] Creating an overlay to protect the source from being modified
[   0.2] Initializing the target -o libvirt -os default
[   0.2] Opening the overlay
[   2.0] Inspecting the overlay
[   5.9] Checking for sufficient free disk space in the guest
[   5.9] Estimating space required on target for each disk
[   5.9] Converting Red Hat Enterprise Linux Server release 6.8 (Santiago) to run on KVM
virt-v2v: warning: /files/etc/sysconfig/grub/boot references unknown device 
"sdb".  You may have to fix this entry manually after conversion.
virt-v2v: This guest has virtio drivers installed.
[  32.9] Mapping filesystem data to avoid copying unused and blank areas
[  33.0] Closing the overlay
[  33.0] Checking if the guest needs BIOS or UEFI to boot
[  33.0] Assigning disks to buses
[  33.0] Copying disk 1/1 to /var/lib/libvirt/images/after-sda (qcow2)
    (100.00/100%)
[  34.2] Creating output metadata
Pool default refreshed

Domain after defined from /tmp/v2vlibvirtdacd93.xml

[  34.7] Finishing off

6.check disk info in guest 'after' after v2v conversion
# lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sr0     11:0    1     1024M  0 rom  
sda      8:0    0        9G  0 disk 
├─vda1   8:1    0      7.8G  0 part /
└─vda2   8:2    0      1.2G  0 part [SWAP]


# cat /boot/grub/device.map 
# this device map was generated by anaconda
(fd0)     /dev/fd0
(hd0)     /dev/vda

So the command 'grub-install --recheck' could recover device.map and it can be updated to vda after v2v conversion, but as step5 showing, still there is a virt-v2v: warning: /files/etc/sysconfig/grub/boot references unknown device "sdb". I check grub info in /etc/sysconfig/ in original guest, the info is /dev/sdb, is there command to recover this grub info? Thanks

#cat /etc/sysconfig/grub
boot=/dev/sdb
forcelba=0
Comment 29 David Kaspar [Dee'Kej] 2016-09-30 07:13:17 EDT
Hello mxie,

thanks for the steps provided. I will try again to reproduce it.

Anyway, regarding the /etc/sysconfig/grub file. As far as I know, grub does not use that file at all. This should be file generated by grubby. I don't know much about grubby, so I will have to look into.

I will get back to you, once I find something.
Comment 31 David Kaspar [Dee'Kej] 2016-10-11 07:09:49 EDT
Hello guys,

currently I do not have time to investigate how to make grubby regenerate its's config file (/etc/sysconfig/grub). I'm switching this to grubby, maybe Peter Jones will know some way.
Comment 33 David Kaspar [Dee'Kej] 2016-10-11 09:46:42 EDT
OK, so according to chat with Peter on IRC, it is most likely anaconda which writes the /etc/sysconfig/grub file, even though that file is owned by grubby (according to yum).

Looking at the source code, grubby only reads that file, it does not write into it.

So, right now, I can only think of 2 possibilites:
1) Fix the /etc/sysconfig/grub file manually after installation.
2) Fix this in anaconda. IMHO, it should be able to determine if the installation is being done from USB flash disk.
Comment 36 Samantha N. Bueno 2016-10-20 13:39:33 EDT
(In reply to David Kaspar [Dee'Kej] from comment #33)
> OK, so according to chat with Peter on IRC, it is most likely anaconda which
> writes the /etc/sysconfig/grub file, even though that file is owned by
> grubby (according to yum).
> 
> Looking at the source code, grubby only reads that file, it does not write
> into it.
> 
> So, right now, I can only think of 2 possibilites:
> 1) Fix the /etc/sysconfig/grub file manually after installation.
> 2) Fix this in anaconda. IMHO, it should be able to determine if the
> installation is being done from USB flash disk.

I'm leaning toward the first option, especially given comment 18 and comment 20. We're in production phase II now. This bug doesn't prevent installation in any way. We can document that you need to manually edit /etc/sysconfig/grub as a workaround.
Comment 37 RHEL Product and Program Management 2016-10-20 14:15:43 EDT
Development Management has reviewed and declined this request.
You may appeal this decision by reopening this request.

Note You need to log in before you can comment on or make changes to this bug.