Bug 1253223
| Summary: | Installing RHEL 6 from USB key writes grub device map referencing /dev/sdb (instead of /dev/sda) | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | mxie <mxie> | ||||||
| Component: | anaconda | Assignee: | Anaconda Maintenance Team <anaconda-maint-list> | ||||||
| Status: | CLOSED WONTFIX | QA Contact: | Release Test Team <release-test-team-automation> | ||||||
| Severity: | medium | Docs Contact: | Clayton Spicer <cspicer> | ||||||
| Priority: | medium | ||||||||
| Version: | 6.7 | CC: | cspicer, deekej, juzhou, mganisin, mxie, mzhan, ovasik, pjones, ptoscano, rjones, sbueno, thozza, tzheng, xiaodwan | ||||||
| Target Milestone: | rc | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | x86_64 | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Known Issue | |||||||
| Doc Text: |
The `device.map` configuration file generated by `Anaconda` is sometimes incorrect
Due to limitations in the kernel, the `device.map` configuration file that is used to map BIOS drives to operating system devices might be generated incorrectly in certain situations, particularly when installing from a USB key. As a consequence, booting sometimes fails after installation. To work around this problem, manually update the `device.map` file in the `/boot/grub` directory. After updating `device.map` so that it correctly maps devices on the system, Red Hat Enterprise Linux 6 will boot as expected.
|
Story Points: | --- | ||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2016-10-20 18:15:43 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Bug Depends On: | |||||||||
| Bug Blocks: | 1356047, 1361708 | ||||||||
| Attachments: |
|
||||||||
Can you attach the debug info. It should be found in /tmp on the Conversion Server, in a directory called something like: /tmp/virt-p2v-20150813-XXXXXX/ Created attachment 1062855 [details]
Conversion log of RHEL6.7-X64-ISCSI
This is basically a bug in anaconda. During installation anaconda wrote an incorrect device.map file, referencing /dev/sdb (instead of /dev/sda). This can happen when you install RHEL from a USB key, since /dev/sda in that case is the USB key, not the hard disk. During conversion it would have printed a warning: libguestfs: trace: aug_get "/files/boot/grub/device.map/hd0" guestfsd: main_loop: new request, len 0x4c guestfsd: main_loop: proc 19 (aug_get) took 0.00 seconds libguestfs: trace: aug_get = "/dev/sdb" virt-v2v: warning: /files/boot/grub/device.map/hd0 references unknown device "sdb". You may have to fix this entry manually after conversion. Anyway, as long as the guest boots, I'm not very concerned about this. The anaconda bug is probably: https://bugzilla.redhat.com/show_bug.cgi?id=909582 Anyway I don't think this is a bug in virt-v2v. The machine had an incorrect device.map to start with. Can you confirm this on the original host? device.map on the original host references /dev/sdb. Can you confirm that the original hard disk layout uses /dev/sda and not /dev/sdb? (Try running 'lsblk' on the source). I will reinstall OS to confirm the original rhel6.7's hard disk layout, in addition, I meet same bug as below:
Description of problem:
Device.map of RHEL7.2-x86_64 guest is also not updated correctly after conversion to RHEV by virt-p2v
Source host info:
the original OS rhel7.2 is installed on fc storage and check device.map and "lsblk "as below
#cat /boot/gurb2/device.map
# this device map was generated by anaconda
(hd0) /dev/mapper/mpatha
(hd1) /dev/mapper/mpatha
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 232.9G 0 disk
├─sda1 8:1 0 243M 0 part
├─sda2 8:2 0 244M 0 part
├─sda3 8:3 0 244M 0 part
└─sda4 8:4 0 232.1G 0 part
├─HostVG-Swap 253:11 0 7.9G 0 lvm
├─HostVG-Config 253:12 0 8M 0 lvm
├─HostVG-Logging 253:13 0 2G 0 lvm /run/media
└─HostVG-Data 253:14 0 222.3G 0 lvm /run/media
sdb 8:16 0 20G 0 disk
└─mpatha 253:1 0 20G 0 mpath
└─mpatha1 253:2 0 20G 0 part /
sdc 8:32 0 1.2T 0 disk
└─mpathb 253:0 0 1.2T 0 mpath
├─45729e32--8959--4d58--89aa--0ee0ab61dc36-metadata 253:3 0 512M 0 lvm
├─45729e32--8959--4d58--89aa--0ee0ab61dc36-outbox 253:4 0 128M 0 lvm
├─45729e32--8959--4d58--89aa--0ee0ab61dc36-leases 253:5 0 2G 0 lvm
├─45729e32--8959--4d58--89aa--0ee0ab61dc36-ids 253:6 0 128M 0 lvm
├─45729e32--8959--4d58--89aa--0ee0ab61dc36-inbox 253:7 0 128M 0 lvm
├─45729e32--8959--4d58--89aa--0ee0ab61dc36-master 253:8 0 1G 0 lvm /run/media
├─45729e32--8959--4d58--89aa--0ee0ab61dc36-888004a8--c6a8--41b5--a5f0--00ba94028458
253:9 0 5.9G 0 lvm
└─45729e32--8959--4d58--89aa--0ee0ab61dc36-6efe101f--3992--4193--a1e2--03104a1ef5a0
253:10 0 5.9G 0 lvm
sdd 8:48 0 20G 0 disk
└─mpatha 253:1 0 20G 0 mpath
└─mpatha1 253:2 0 20G 0 part /
sde 8:64 0 1.2T 0 disk
└─mpathb 253:0 0 1.2T 0 mpath
├─45729e32--8959--4d58--89aa--0ee0ab61dc36-metadata 253:3 0 512M 0 lvm
├─45729e32--8959--4d58--89aa--0ee0ab61dc36-outbox 253:4 0 128M 0 lvm
├─45729e32--8959--4d58--89aa--0ee0ab61dc36-leases 253:5 0 2G 0 lvm
├─45729e32--8959--4d58--89aa--0ee0ab61dc36-ids 253:6 0 128M 0 lvm
├─45729e32--8959--4d58--89aa--0ee0ab61dc36-inbox 253:7 0 128M 0 lvm
├─45729e32--8959--4d58--89aa--0ee0ab61dc36-master 253:8 0 1G 0 lvm /run/media
├─45729e32--8959--4d58--89aa--0ee0ab61dc36-888004a8--c6a8--41b5--a5f0--00ba94028458
253:9 0 5.9G 0 lvm
└─45729e32--8959--4d58--89aa--0ee0ab61dc36-6efe101f--3992--4193--a1e2--03104a1ef5a0
253:10 0 5.9G 0 lvm
sr0 11:0 1 3G 0 rom /run/media
Guest info on RHEV:
check device.map and "lsblk""as below
#cat /boot/gurb2/device.map
# this device map was generated by anaconda
(hd0) /dev/mapper/mpatha
(hd1) /dev/mapper/mpatha
#lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr0 11:0 1 1024M 0 rom
vda 253:0 0 20G 0 disk
└─vda1 253:1 0 20G 0 part /
I have checked disk info of original rhel6.7_x64 which installed iSCSI as below: #cat /boot/grub/device.map # this device map was generated by anaconda (hd0) /dev/sdb #lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdb 8:16 0 931.5G 0 disk sda 8:0 0 30G 0 disk └─sda1 8:1 0 30G 0 part / sr0 11:0 1 1024M 0 rom The rhel6.7_x64 guest's disk info on rhev as below: #cat /boot/grub/device.map # this device map was generated by anaconda (hd0) /dev/sdb #lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 1024M 0 rom vda 8:0 0 30G 0 disk └─vda1 8:1 0 30G 0 part / Updating the summary so it's clearer. The bug as I understand it is this: When you install RHEL 6 from a USB boot device, anaconda writes a grub device map that contains: # this device map was generated by anaconda (hd0) /dev/sdb When the machine boots however, the hard disk is /dev/sda (because the USB key is removed and the BIOS changes the disk slot). Not actually sure this causes any problems, except for confusing virt-p2v slightly. *** Bug 1254857 has been marked as a duplicate of this bug. *** *** Bug 1254139 has been marked as a duplicate of this bug. *** After looking at this a bit I don't think there's anything we can, or should, do to correct this. In the situations described in this bug it is just a warning, so it isn't preventing the conversion from happening or from the result from booting. When installing we have no way to know how the system will enumerate the drives when the installation media is removed. We could try guessing, but I'm worried that would then break actual cases where device.map does matter. Looking at current Fedora and RHEL7 code they all work the same, writing useless device.map files when using removable media. I'll cc pjones to see if he has any thoughts on this, but I'm leaning towards WONTFIX The kernel does not guarantee the order it enumerates block devices in *any way*, so it's never going to be possible ensure it's right. In this case it'll need to be fixed if you ever need to re-install grub, but it's not really used during booting. grub-install can do this with the --recheck option. Also leaning towards WONTFIX. At most we could provide a release note; that should probably be up to the maintainer of the RHEL 6 grub package. I can reproduce with builds:
P2V client:
virt-p2v-1.28.1-1.51.3.el7.1.noarch.rpm
Conversion server:
virt-v2v-1.32.4-1.el7.x86_64
libguestfs-1.32.4-1.el7.x86_64
kernel:3.10.0-394.el7.x86_64
libvirt-1.3.4-1.el7.x86_64
qemu-kvm-1.5.3-111.el7.x86_64
Steps:
1.Prepare 30G iscsi storage on the the QLE4060c iscsi card
2.Install last RHEL6.8 x86 on the iscsi storage
3.Finish os installation. make sure boot into the OS normally, check some info as below:
3.1 #lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 30G 0 disk
├─sda1 8:1 0 2G 0 part [SWAP]
└─sda2 8:2 0 28G 0 part /
3.2#uname -a
Linux bootp-73-75-38.lab.eng.pek2.redhat.com 2.6.32-642.el6.i686 #1 SMP Wed Apr 13 00:50:26 EDT 2016 i686 i686 i386 GNU/Linux
4.Boot the machine into the p2v client via iso file and input the ip and password of conversion server.
5.After conversion, check guest's device.map on RHEV as below,it is not updated to vda
#cat /boot/grub/device.map
#this device map was generated by anaconda
(hd0) /dev/sdb
6.But using commend #fdisk -l to check disk info, it shows "/dev/vda"
Disk /dev/vda: 32.1 GB, 32144097280 bytes
16 heads, 63 sectors/track, 62283 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000c55ca
Device Boot Start End Blocks Id System
/dev/vda1 3 4066 2048000 82 Linux swap / Solaris
Partition 1 does not end on cylinder boundary.
/dev/vda2 * 4066 62284 29341696 83 Linux
Partition 2 does not end on cylinder boundary.
Hello guys, I'm inclined to close this BZ as WONTFIX. (In reply to Peter Jones from comment #20) > The kernel does not guarantee the order it enumerates block devices in *any > way*, so it's never going to be possible ensure it's right. And same applies to GRUB itself because kernel does not guarantee the order. Excerpt from 'info grub': 3.3 Installing GRUB using grub-install ====================================== *Caution:* This procedure is definitely less safe, because there are several ways in which your computer can become unbootable. For example, most operating systems don't tell GRUB how to map BIOS drives to OS devices correctly--GRUB merely "guesses" the mapping. This will succeed in most cases, but not always. Therefore, GRUB provides you with a map file called the "device map", which you must fix if it is wrong. Therefore, fixing something like this would actually be an enhancement/new feature of grub. Since we are in the phase2 of RHEL6 lifecycle, this is not allowed anymore. NOTE: Anaconda will create the device.map file during installation, but the file will not be correctly applicable once the USB is removed. IMHO, anaconda does not see a diffrence between installation from CD/DVD and USB. Usually, we tend to count on the fact that people do not remove their CD/DVD mechanics so the disk layout stays the same after the installation. This, however, does not apply to installation from USB, which is usually removed after the installation. (In reply to Peter Jones from comment #20) > In this case > it'll need to be fixed if you ever need to re-install grub, but it's not > really used during booting. grub-install can do this with the --recheck > option. What the --recheck option actually does is making backup of device.map file, and generating new one from current disk layout, which is "guessed". This could actually help workaround this problem: * install RHEL6 on the machine * remove the USB after installation * boot the machine, run 'grub-install --recheck' * (reboot? the machine) * start the conversion to RHEV I have discussed this option with mxie already. He said he will try it and let us know. The problem with that theory is there is only one disk. Hi David,
I try to use vm to verify the command 'grub-install --recheck' because our machine is located at remote lab so that can't install os with usb CD/DVD
Steps
1.Create a new vm in virt-manager and add 2 disk, one type is scsi and another one type is sata
2.Install rhel6.8 on scsi disk,remove the sata disk after finishing installation,then check disk info in guest as below:
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr0 11:0 1 1024M 0 rom
sda 8:0 0 9G 0 disk
├─sda1 8:1 0 7.8G 0 part /
└─sda2 8:2 0 1.2G 0 part [SWAP]
#cat /boot/grub/device.map
# this device map was generated by anaconda
(hd0) /dev/sdb
3.Using command 'grub-install --recheck' to recover device.map
#grub-install --recheck /dev/sda
4.Check device.map again
#cat /boot/grub/device.map
# this device map was generated by anaconda
(fd0) /dev/fd0
(hd0) /dev/sda
5.Using v2v to convert this guest
# virt-v2v rhel6.8 -on after
[ 0.0] Opening the source -i libvirt rhel6.8
[ 0.0] Creating an overlay to protect the source from being modified
[ 0.2] Initializing the target -o libvirt -os default
[ 0.2] Opening the overlay
[ 2.0] Inspecting the overlay
[ 5.9] Checking for sufficient free disk space in the guest
[ 5.9] Estimating space required on target for each disk
[ 5.9] Converting Red Hat Enterprise Linux Server release 6.8 (Santiago) to run on KVM
virt-v2v: warning: /files/etc/sysconfig/grub/boot references unknown device
"sdb". You may have to fix this entry manually after conversion.
virt-v2v: This guest has virtio drivers installed.
[ 32.9] Mapping filesystem data to avoid copying unused and blank areas
[ 33.0] Closing the overlay
[ 33.0] Checking if the guest needs BIOS or UEFI to boot
[ 33.0] Assigning disks to buses
[ 33.0] Copying disk 1/1 to /var/lib/libvirt/images/after-sda (qcow2)
(100.00/100%)
[ 34.2] Creating output metadata
Pool default refreshed
Domain after defined from /tmp/v2vlibvirtdacd93.xml
[ 34.7] Finishing off
6.check disk info in guest 'after' after v2v conversion
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr0 11:0 1 1024M 0 rom
sda 8:0 0 9G 0 disk
├─vda1 8:1 0 7.8G 0 part /
└─vda2 8:2 0 1.2G 0 part [SWAP]
# cat /boot/grub/device.map
# this device map was generated by anaconda
(fd0) /dev/fd0
(hd0) /dev/vda
So the command 'grub-install --recheck' could recover device.map and it can be updated to vda after v2v conversion, but as step5 showing, still there is a virt-v2v: warning: /files/etc/sysconfig/grub/boot references unknown device "sdb". I check grub info in /etc/sysconfig/ in original guest, the info is /dev/sdb, is there command to recover this grub info? Thanks
#cat /etc/sysconfig/grub
boot=/dev/sdb
forcelba=0
Hello mxie, thanks for the steps provided. I will try again to reproduce it. Anyway, regarding the /etc/sysconfig/grub file. As far as I know, grub does not use that file at all. This should be file generated by grubby. I don't know much about grubby, so I will have to look into. I will get back to you, once I find something. Hello guys, currently I do not have time to investigate how to make grubby regenerate its's config file (/etc/sysconfig/grub). I'm switching this to grubby, maybe Peter Jones will know some way. OK, so according to chat with Peter on IRC, it is most likely anaconda which writes the /etc/sysconfig/grub file, even though that file is owned by grubby (according to yum). Looking at the source code, grubby only reads that file, it does not write into it. So, right now, I can only think of 2 possibilites: 1) Fix the /etc/sysconfig/grub file manually after installation. 2) Fix this in anaconda. IMHO, it should be able to determine if the installation is being done from USB flash disk. (In reply to David Kaspar [Dee'Kej] from comment #33) > OK, so according to chat with Peter on IRC, it is most likely anaconda which > writes the /etc/sysconfig/grub file, even though that file is owned by > grubby (according to yum). > > Looking at the source code, grubby only reads that file, it does not write > into it. > > So, right now, I can only think of 2 possibilites: > 1) Fix the /etc/sysconfig/grub file manually after installation. > 2) Fix this in anaconda. IMHO, it should be able to determine if the > installation is being done from USB flash disk. I'm leaning toward the first option, especially given comment 18 and comment 20. We're in production phase II now. This bug doesn't prevent installation in any way. We can document that you need to manually edit /etc/sysconfig/grub as a workaround. Development Management has reviewed and declined this request. You may appeal this decision by reopening this request. |
Created attachment 1062412 [details] screenshot of RHEL6.7-ISCSI-device.map Description of problem: device.map of RHEL6.7-x86_64 guest isn't updated correctly after conversion to RHEV by virt-p2v Version-Release number of selected component (if applicable): virt-v2v-1.28.1-1.49.el7.x86_64 virt-p2v-1.28.1-1.49.el7-1.iso How reproducible: 100% Steps to Reproduce: 1.Install rhel6.7-x86_64 on iscsi storage 2.Boot the source machine into virt-p2v via DVD-ROM and input the ip and password of conversion server. 3.Convert rhel6.7-x86_64 iscsi host to RHEV via virt-p2v client 4.After conversion,boot into rhel6.7-x86_64 guest and check points on RHEV 5.Using command #cat /boot/grub/device.map to check device.map of rhel6.7-x86_64 guest on RHEV as below,it is not updated to vda #this device map was generated by anaconda (hd0) /dev/sdb 6.But type command #fdisk -l to check disk, it shows "/dev/vda", pls refer to attach screenshot Actual results: device.map of RHEL6.7-x86_64 guest isn't updated to vda on RHEV Expected results: device.map of RHEL6.7-x86_64 guest should be updated to vda on RHEV Additional info: