Bug 621175

Summary: anaconda chooses incorrect boot device while running in KVM with mixed virtio/ide disks
Product: Red Hat Enterprise Linux 6 Reporter: Gleb Natapov <gleb>
Component: anacondaAssignee: Ales Kozumplik <akozumpl>
Status: CLOSED ERRATA QA Contact: Release Test Team <release-test-team-automation>
Severity: medium Docs Contact:
Priority: low    
Version: 6.0CC: akozumpl, atodorov, gasmith, jzeleny, khong, knoel, lcapitulino, martin.wilck, mbanas, syeghiay
Target Milestone: rcKeywords: Reopened, RHELNAK
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: anaconda-13.21.120-1 Doc Type: Bug Fix
Doc Text:
KVM users with a mix of virtio and ata disks should verify the boot device that anaconda chooses during installation. To verify the boot device, locate the "Install Target Devices" list in the disk selection screen that follows the partitioning type screen. Verify the boot device selection, which is indicated by a selector in the left-most column of the "Install Target Devices" list.
Story Points: ---
Clone Of:
: 729694 (view as bug list) Environment:
Last Closed: 2011-12-06 10:26:51 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 668737, 704128    
Bug Blocks:    
Attachments:
Description Flags
storage.log from anaconda
none
storage log from latest rhel6 installer
none
ide is a boot drive
none
virtio is a boot drive
none
Bios image with fixed EDD
none
tarball with logs from failed install none

Description Gleb Natapov 2010-08-04 13:09:01 UTC
Created attachment 436526 [details]
storage.log from anaconda

Description of problem:
anaconda chooses incorrect boot device while running in KVM with mixed virtio/ide disks.

Version-Release number of selected component (if applicable):
Anaconda from Snapshot-8

How reproducible:

Steps to Reproduce:
1.Create three qemu images:
 qemu-img create /tmp/a.raw 10G
 qemu-img create /tmp/b.raw 10G
 qemu-img create /tmp/c.raw 10G

2. start rehl6 install inside guest:
qemu-system-x86_64  -m 1024 -drive file=/tmp/a.raw,if=virtio -drive file=/tmp/b.raw -drive file=/tmp/c.raw -cdrom RHEL6.0-20100722.0-Server-x86_64-DVD1.iso -boot order=d

3. Click through to disk selection dialog. Select all disks

4. Click through to "Install Target Tevices" selection window. Select all devices.
  
Actual results:

Virtio Block Device is marked as boot loader target

Expected results:
One of ATA devices should be marked as boot loader (the one that will became sda).

Additional info:

EDD seems to report boot device correctly:

# ls -al /sys/firmware/edd/int13_dev80/pci_dev
rwxrwxrwx. 1 root root 0 Aug  4 08:51 /sys/firmware/edd/int13_dev80/pci_dev -> ../../../devices/pci0000:00/0000:00:01.1

# lspci -s 0000:00:01.1
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]

Comment 2 RHEL Program Management 2010-08-04 13:27:36 UTC
This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **

Comment 3 RHEL Program Management 2010-08-04 14:44:42 UTC
Development Management has reviewed and declined this request.  You may appeal
this decision by reopening this request.

Comment 4 Gleb Natapov 2010-08-04 15:04:55 UTC
Reopen and move to 6.1.

Comment 6 David Lehman 2010-08-19 19:39:47 UTC
This is really only a bug in the sense that the *default* selection is incorrect. The workaround is to select the correct boot device by activating the radiobutton next the appropriate device's entry in the list.

Comment 7 Gleb Natapov 2010-08-19 20:42:43 UTC
(In reply to comment #6)
> This is really only a bug in the sense that the *default* selection is
> incorrect. The workaround is to select the correct boot device by activating
> the radiobutton next the appropriate device's entry in the list.

We want to have working and bootable system if user just click through installer. Anaconda knows better than user which disk is bootable.

Comment 8 David Cantrell 2010-08-24 00:30:27 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
KVM users with a mix of virto and ata disks should verify the boot device that anaconda chooses during installation.  To verify the boot device, users must check the "Review and modify partitioning layout" option on the screen that asks what type of installation you would like.  The boot device screen will come after anaconda completes the Formatting step.  Verify anaconda has chosen the correct boot device your your environment.  If not, click "Change device" and select the correct one.

Comment 9 David Lehman 2010-08-24 18:50:15 UTC
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1 +1 @@
-KVM users with a mix of virto and ata disks should verify the boot device that anaconda chooses during installation.  To verify the boot device, users must check the "Review and modify partitioning layout" option on the screen that asks what type of installation you would like.  The boot device screen will come after anaconda completes the Formatting step.  Verify anaconda has chosen the correct boot device your your environment.  If not, click "Change device" and select the correct one.+KVM users with a mix of virtio and ata disks should verify the boot device that anaconda chooses during installation.  To verify the boot device, locate the "Install Target Devices" list in the disk selection screen that follows the partitioning type screen. Verify the boot device selection, which is indicated by a selector in the left-most column of the "Install Target Devices" list.

Comment 10 David Cantrell 2010-08-24 19:15:37 UTC

*** This bug has been marked as a duplicate of bug 614869 ***

Comment 11 Gleb Natapov 2010-08-24 19:32:32 UTC
This bug is not duplicate of bug 614869. Please fix anaconda to correctly select boot device.

Comment 12 Luiz Capitulino 2010-08-24 23:03:45 UTC
(In reply to comment #11)
> This bug is not duplicate of bug 614869. Please fix anaconda to correctly
> select boot device.

I agree.

This works as expected when all disks are IDE or all disks are virtio.

If qemu-kvm is not exporting correct information, a new BZ should be opened, but bug 614869 doesn't seem to have anything to do with this problem.

Comment 13 David Lehman 2010-08-25 01:15:46 UTC
(In response to comments 11 and 12):

In the absence of EDD/BIOS information about drive ordering, anaconda must have some mechanism for deciding which device comes "first" between, eg: sda and vda. Currently, and since at least RHEL5, we sort vda before sda, which seems to be the basic problem here. Has something changed, or have we just never had users mixing ATA and virtio devices in the past?

Comment 14 Gleb Natapov 2010-08-25 08:11:29 UTC
(In reply to comment #13)
> (In response to comments 11 and 12):
> 
> In the absence of EDD/BIOS information about drive ordering, anaconda must have
> some mechanism for deciding which device comes "first" between, eg: sda and
> vda.

That is the point of this BZ. In RHEL6 bios provides EDD information for ide and virtio disks. If you run qemu command line from comment #0 and do:
# ls /sys/firmware/edd/
int13_dev80 int13_dev81 int13_dev82

You can see info about all three disks and you know which one is bootable by looking info int13_dev80 directory. As comment #0 shows if int13_dev80 is IDE it is easy to find the device behind it. Unfortunately virtio does not provide enough information in EDD to link int13_dev80 to actual virtio interface yet (It should be easy to add the only problem is that EDD spec defines all supported interface types and virtio is, of course, not there). If you need more info from EDD for proper anaconda support of virtualization just ask.

>     Currently, and since at least RHEL5, we sort vda before sda, which seems
> to be the basic problem here. Has something changed, or have we just never had
> users mixing ATA and virtio devices in the past?
In RHEL5 BIOS did not have native virtio support, in RHEL6 it does.

Comment 15 David Lehman 2010-08-25 15:18:48 UTC
(In reply to comment #14)
> (In reply to comment #13)
> > (In response to comments 11 and 12):
> > 
> > In the absence of EDD/BIOS information about drive ordering, anaconda must have
> > some mechanism for deciding which device comes "first" between, eg: sda and
> > vda.
> 
> That is the point of this BZ. In RHEL6 bios provides EDD information for ide
> and virtio disks. If you run qemu command line from comment #0 and do:
> # ls /sys/firmware/edd/
> int13_dev80 int13_dev81 int13_dev82
> 
> You can see info about all three disks and you know which one is bootable by
> looking info int13_dev80 directory. As comment #0 shows if int13_dev80 is IDE
> it is easy to find the device behind it. Unfortunately virtio does not provide
> enough information in EDD to link int13_dev80 to actual virtio interface yet
> (It should be easy to add the only problem is that EDD spec defines all
> supported interface types and virtio is, of course, not there). If you need
> more info from EDD for proper anaconda support of virtualization just ask.

So we now have EDD info, but it is not complete.

If the EDD info provided is not complete, anaconda is left to guess. Please provide the complete information so that things will work properly. I do not see how you can call this an anaconda bug if anaconda is forced to guess because correct information is not available from EDD and does not guess correctly.

Please either fix the EDD support or else provide some other method by which anaconda can determine drive ordering for virtio devices.

Comment 16 Gleb Natapov 2010-08-25 15:35:47 UTC
(In reply to comment #15)
> So we now have EDD info, but it is not complete.
EDD has multiple versions and extensions. Some extensions are not returned for virtio disk since they are not defined for virtio disks by spec. 

> 
> If the EDD info provided is not complete, anaconda is left to guess. Please
> provide the complete information so that things will work properly. I do not
> see how you can call this an anaconda bug if anaconda is forced to guess
> because correct information is not available from EDD and does not guess
> correctly.
In example I gave in comment #0 (have you looked at it?) all info is provided for anaconda to properly choose correct boot device, but this isn't happening so this is anaconda bug.
  
> 
> Please either fix the EDD support or else provide some other method by which
> anaconda can determine drive ordering for virtio devices.
If /sys/firmware/edd/int13_dev80/pci_dev exists use it as boot device. If it doesn't use your current method (although bios actually puts IDE before virtio in boot order calculation).

Comment 19 Ales Kozumplik 2010-12-13 15:22:57 UTC
<dlehman> akozumpl: in addition to the edd part, we apparently have the vda -v- sda comparison backwards in Storage.compareDrives
<gleb> dlehman, without edd the order between vda and sda is arbitrary

Comment 20 Ales Kozumplik 2010-12-13 15:25:19 UTC
<dlehman> akozumpl: my memory is a bit fuzzy, but I think that's for the cases when edd info is not adequate

Comment 21 Gleb Natapov 2010-12-14 14:56:41 UTC
Created attachment 468622 [details]
storage log from latest rhel6 installer

Comment 22 Ales Kozumplik 2010-12-15 10:03:33 UTC
We discussed this with Gleb on IRC in and out and we can't figure out a way to map the edd entry in sysfs to a single sysfs path to a device.

Comment 23 RHEL Program Management 2010-12-15 15:40:09 UTC
This request was evaluated by Red Hat Product Management for inclusion
in a Red Hat Enterprise Linux maintenance release. Product Management has 
requested further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed 
products. This request is not yet committed for inclusion in an Update release.

Comment 24 Ales Kozumplik 2010-12-21 16:08:20 UTC
Gleb,

can you please test with the update image and see if it fixes the issue?
http://akozumpl.fedorapeople.org/bz621175.img

Thanks.
Ales

Comment 25 Gleb Natapov 2011-01-04 08:58:41 UTC
Created attachment 471617 [details]
ide is a boot drive

Comment 26 Gleb Natapov 2011-01-04 08:59:37 UTC
Created attachment 471618 [details]
virtio is a boot drive

Comment 27 Gleb Natapov 2011-01-04 09:02:15 UTC
With image from comment #4 ide is always chosen to install MBR no matter which disk is reported as int80. I've added two attachments of storage.log. One when ide is int80 another is when virtio is int80.

Comment 28 Ales Kozumplik 2011-01-04 15:08:31 UTC
Following our discussion on IRC today: I still think the patch works OK, you just hit the situation where anaconda can not guess better (see your comment 16, the patch is doing exactly what you've asked for, that is it prefers sda as it can find the pci device there, while it can't for vda). I suggest fixing this with the proposed patch, if you'll need a fix for all the cases you need to give us a complete specification of how to map devices (like /dev/sda) to edd devices (like /sys/firmware/edd/int13_dev80). Until then there's nothing more I can do I'm afraid.

Please confirm this is OK for now. For the general problem you're welcome to open a new BZ once kernel reveals the requested information.

Comment 29 Gleb Natapov 2011-01-05 15:37:36 UTC
I am using modified bios that has PCI information in virtio EDD, so pci_dev link is created for virtio too, but the problem is that there is no way right no to find out which /dev/vd? correspond to what PCI device. This is kernel bug and it is currently looked at by virtio experts. For ata we have enough info to find /dev/sd? device for /sys/firmware/edd/int13_dev80. To do that we need to know three numbers PCI address (A), channel (C), device (D). PCI address and channel are found in /sys/firmware/edd/int13_dev80/host_bus, device is found in /sys/firmware/edd/int13_dev80/interface. Now given A, C, D we can find block device by looking at:

sprintf(path, "/sys/devices/pci0000:00/0000:%s/host%d/target%d:0:%d:0/%d:0:%d:0/block", A, C, C, D, C, D);

The name of the directory you'll find there is the same as device node.

I will attach bios that provides enough info for Linux kernel to put relevant information in /sys/firmware/edd/int13_dev80/.

Comment 30 Gleb Natapov 2011-01-05 15:38:32 UTC
Created attachment 471888 [details]
Bios image with fixed EDD

Comment 31 Ales Kozumplik 2011-01-10 14:54:51 UTC
Gleb,

I have a version that does what you are describing in http://akozumpl.fedorapeople.org/bz621175.img. 

Can you please retest this matches your expectations and attach the storage.log in case it does not?

Thanks.
Ales

Comment 32 Ales Kozumplik 2011-01-10 14:58:15 UTC
As a sidenote, I noticed the edd support exhibits strange functionality even for ATA drives: I have a kvm machine that only has two ATA drives. One of the edd entries has host_bus, interface and pci_dev files while the other one has none of those.

Comment 33 Ales Kozumplik 2011-01-11 12:53:30 UTC
As discussed with Gleb on IRC, the specification in Comment 29 doesn't apply for scsi devices. Please attach all required specifications. Also, isn't it more convenient to find the device through /sys/class/scsi_device/ than through the pci device?

Comment 37 Gleb Natapov 2011-05-12 12:39:24 UTC
Below is the algorithm that should be used to detect boot device using EDD data.

Find boot device:
   Build block device list L
   For each device B in L
      Map B to edd device (see below).
      If B maps to int13_dev80
         Return B // Install bootloader in B
      If B maps to other edd device
         Drop B from L
   If bootdevice is not found
      Return can't detect, fall back to mbr_signature checking or return first device from L if no mbr_signature available

Map block device B to edd device:
   For each dir E in /sys/firmware/edd/*
     If file /sys/firmware/edd/E/host_bus does not exist or
        file /sys/firmware/edd/E/interface does not exist
          Return can't map
     A = PCI address from /sys/firmware/edd/E/host_bus
     C = channel from host_bus /sys/firmware/edd/E/host_bus
     I = interface type from /sys/firmware/edd/E/interface (SCSI or ATA)
     If I is ATA
        D = device number from /sys/firmware/edd/E/interface
        P = /sys/devices/pci0000:00/0000:%s/host%d/target%d:0:%d:0/%d:0:%d:0/block/%s", A, C, C, D, C, D, B
     Else if I is SCSI
       D = id from /sys/firmware/edd/E/interface
       P = /sys/devices/pci0000:00/0000:%s/virtio%d/block/%s, A, D, B
     Else
       Return can't map // other interfaces can be added later
     If P exists
       Return E // B maps to E
  Return can't map

Comment 38 Ales Kozumplik 2011-05-24 15:05:04 UTC
*** Bug 694800 has been marked as a duplicate of this bug. ***

Comment 39 Ales Kozumplik 2011-05-24 15:41:17 UTC
*** Bug 694800 has been marked as a duplicate of this bug. ***

Comment 40 Ales Kozumplik 2011-05-24 15:57:49 UTC
Regarding comment 37:

I brought this up with other members of the Anaconda team, including my team leader, and there is very strong consensus we do not want anything like that in our code. You either have to:

1) Provide and maintain a small python library doing whatever is necessary. This is by far the best solution.
2) Convince kernel they need to do the guessing on their side and fix bug 668737 the best they can.
3) Convince David Cantrell (dcantrell, the Anaconda team manager) that attempting this in Anaconda is the best way to move forward with this. Such code is however highly prone to bugs and it is certain more bugs will be coming back. Without David's word in favor of this option there is no way it will pass the peer review process.

Comment 41 Martin Wilck 2011-05-24 16:12:01 UTC
I doubt that comment #37 would work for my example (SCSI case, megaraid_sas RAID controller with 2 logical volumes, boots from 2ns volume).

/sys/firmware/edd/int13_dev80/pci_dev -> ../../../devices/pci0000:00/0000:00:03.0/0000:10:00.0
/sys/firmware/edd/int13_dev81/pci_dev -> ../../../devices/pci0000:00/0000:00:03.0/0000:10:00.0

cat /sys/firmware/edd/int13_dev8*/host_bus
dev80: PCI 	10:00.0  channel: 0
dev81: PCI 	10:00.0  channel: 0

cat /sys/firmware/edd/int13_dev8*/interface
dev80: SCSI    	id: 1  lun: 0
dev81: SCSI    	id: 0  lun: 0

$ ls -d /sys/devices/pci*/*/*/host*/target*/*/block/*
/sys/devices/pci0000:00/0000:00:03.0/0000:10:00.0/host0/target0:2:0/0:2:0:0/block/sda
/sys/devices/pci0000:00/0000:00:03.0/0000:10:00.0/host0/target0:2:1/0:2:1:0/block/sdb
/sys/devices/pci0000:00/0000:00:0a.0/0000:50:00.0/host1/target1:2:0/1:2:0:0/block/sdc

This case has the additional complication that the channel number doesn't match (must be the megaraid_sas driver's fault). However, by matching the available block devices with the EDD data, guessing the boot device isn't hard in this case (for a human being, at least).

Comment 42 Martin Wilck 2011-05-24 16:18:51 UTC
(In reply to comment #40)
> Such code is however highly prone to bugs and it is certain more bugs 
> will be coming back. 

Well, there is a bug now. Anaconda fails to install in a perfectly valid setup although it has sufficient data from EDD to do better. I don't care if this is realized in anaconda itself or in some library (although I think that the number 1 use case for EDD date is bootloader installation - actually I can't easily figure out a different use case). But I'd like to have a working installer.

Comment 43 Gleb Natapov 2011-05-24 16:42:14 UTC
(In reply to comment #41)
> I doubt that comment #37 would work for my example (SCSI case, megaraid_sas
> RAID controller with 2 logical volumes, boots from 2ns volume).
The comment #37 address only ide/virtio since this is what this bug is or was about. The algorithm will fall back to current logic if neither is found. It may be extended to handle other types of HW (provided that BIOS supplies correct EDD info, currently there is a kernel bug that int13_dev80 can be created with incorrect data: bz 704128. Patch is posted to fix this issue, so your either will not have the directory at all or will have correct info there).
 
> /sys/firmware/edd/int13_dev80/pci_dev ->
> ../../../devices/pci0000:00/0000:00:03.0/0000:10:00.0
> /sys/firmware/edd/int13_dev81/pci_dev ->
> ../../../devices/pci0000:00/0000:00:03.0/0000:10:00.0
> 
> cat /sys/firmware/edd/int13_dev8*/host_bus
> dev80: PCI  10:00.0  channel: 0
> dev81: PCI  10:00.0  channel: 0
> 
> cat /sys/firmware/edd/int13_dev8*/interface
> dev80: SCSI     id: 1  lun: 0
> dev81: SCSI     id: 0  lun: 0
> 
> $ ls -d /sys/devices/pci*/*/*/host*/target*/*/block/*
> /sys/devices/pci0000:00/0000:00:03.0/0000:10:00.0/host0/target0:2:0/0:2:0:0/block/sda
> /sys/devices/pci0000:00/0000:00:03.0/0000:10:00.0/host0/target0:2:1/0:2:1:0/block/sdb
> /sys/devices/pci0000:00/0000:00:0a.0/0000:50:00.0/host1/target1:2:0/1:2:0:0/block/sdc
> 
> This case has the additional complication that the channel number doesn't match
> (must be the megaraid_sas driver's fault). However, by matching the available
> block devices with the EDD data, guessing the boot device isn't hard in this
> case (for a human being, at least).
There was a bug that it wasn't possible to figure out virtio disk from EDD info by looking at /sys. Such bug should be fixed in kernel indeed and virtio one was fixed in rhel6.1.

Comment 44 Gleb Natapov 2011-05-24 16:47:26 UTC
(In reply to comment #42)
> (In reply to comment #40)
> > Such code is however highly prone to bugs and it is certain more bugs 
> > will be coming back. 
> 
> Well, there is a bug now. Anaconda fails to install in a perfectly valid setup
Exactly. Leaving computer in non bootable state after OS install should be highest priority bug for software which purpose in life is to install an OS.
  
> although it has sufficient data from EDD to do better. I don't care if this is
> realized in anaconda itself or in some library (although I think that the
> number 1 use case for EDD date is bootloader installation - actually I can't
> easily figure out a different use case). But I'd like to have a working
> installer.
I can't think of any other use for EDD data either. I suspect there is none. If such library should exists it should be part of anaconda.

Comment 45 Martin Wilck 2011-05-25 14:17:53 UTC
(In reply to comment #43)

> The comment #37 address only ide/virtio since this is what this bug is or was
> about.

I was mentioning my case because the original bug 694800 was closed as duplicate of this one, implying that a solution for this one must also provide a solution to bug 694800.

Comment 46 Gleb Natapov 2011-05-25 16:45:28 UTC
(In reply to comment #45)
> (In reply to comment #43)
> 
> > The comment #37 address only ide/virtio since this is what this bug is or was
> > about.
> 
> I was mentioning my case because the original bug 694800 was closed as
> duplicate of this one, implying that a solution for this one must also provide
> a solution to bug 694800.

Ah yes. I posted #37 long before that though. Unfortunately I do not have HW with SCSI disk and BIOS that provides correct EDD info to investigate general SCSI case.

Comment 47 Martin Wilck 2011-06-01 12:58:44 UTC
With bios.bin from comment #30, I see no host_bus or pci_dev in the EDD directory corresponding to my IDE drive. I do see it for the virtio drive, though (F14 host with qemu-system-x86-0.13.0-1.fc14.x86_64, replaced bios.bin with the one provided by you).

Comment 48 Gleb Natapov 2011-06-01 13:07:27 UTC
(In reply to comment #47)
> With bios.bin from comment #30, I see no host_bus or pci_dev in the EDD
> directory corresponding to my IDE drive. I do see it for the virtio drive,
> though (F14 host with qemu-system-x86-0.13.0-1.fc14.x86_64, replaced bios.bin
> with the one provided by you).

Please use qemu/bios from rhel6.1. This combination should work. Or you can use upstream qemu.

Comment 49 Ales Kozumplik 2011-06-22 13:23:07 UTC
Hi,

Following our phone call last week I started working on this again but I found that the algorithm outlined in comment 37 does not work reliably:

>      Else if I is SCSI
>        D = id from /sys/firmware/edd/E/interface
>        P = /sys/devices/pci0000:00/0000:%s/virtio%d/block/%s, A, D, B
>      Else
>        Return can't map // other interfaces can be added later
>      If P exists
>        Return E // B maps to E
>   Return can't map

On my kvm system (rhel6.1 host) I have /sys/firmware/edd/int13_dev81/host_bus:
PCI 	00:05.0  channel: 0

/sys/firmware/edd/int13_dev81/interface:
SCSI    	id: 0  lun: 0

so 
 A=00:05.0
 D=0

now that would point somewhere in:

/sys/devices/pci0000:00/0000:00:05.0/virtio0/...

but 'ls -d /sys/devices/pci0000\:00/0000\:00\:05.0/virtio*' only shows 

/sys/devices/pci0000:00/0000:00:05.0/virtio2

How should I come up with the '2'?

Comment 50 Ales Kozumplik 2011-06-22 13:37:23 UTC
Martin,

I am looking into your case too, trying to come up with heuristics based on sector count. Can you please post output of the following commands?

cat /sys/firmware/edd/*/sectors
cat /sys/block/sd*/size

Thanks.
Ales

Comment 51 Gleb Natapov 2011-06-22 13:58:09 UTC
(In reply to comment #49)
> Hi,
> 
> Following our phone call last week I started working on this again but I found
> that the algorithm outlined in comment 37 does not work reliably:
> 
> >      Else if I is SCSI
> >        D = id from /sys/firmware/edd/E/interface
> >        P = /sys/devices/pci0000:00/0000:%s/virtio%d/block/%s, A, D, B
> >      Else
> >        Return can't map // other interfaces can be added later
> >      If P exists
> >        Return E // B maps to E
> >   Return can't map
> 
> On my kvm system (rhel6.1 host) I have /sys/firmware/edd/int13_dev81/host_bus:
> PCI  00:05.0  channel: 0
> 
> /sys/firmware/edd/int13_dev81/interface:
> SCSI     id: 0  lun: 0
> 
> so 
>  A=00:05.0
>  D=0
> 
> now that would point somewhere in:
> 
> /sys/devices/pci0000:00/0000:00:05.0/virtio0/...
> 
> but 'ls -d /sys/devices/pci0000\:00/0000\:00\:05.0/virtio*' only shows 
> 
> /sys/devices/pci0000:00/0000:00:05.0/virtio2
> 
> How should I come up with the '2'?

This is my mistake. Each virtio disk has only one channel now, so id from interface file is not encoded anywhere in the path. All virtio devices (including net/serial) are enumerated and the device's number is encoded in the directory name, so this number is meaningless for our purpose.

It should be like this instead:
P = /sys/devices/pci0000:00/0000:%s/virtio*/block/%s, A, B

where virtio* matches any directory which name is started from "virtio"

Comment 52 Ales Kozumplik 2011-06-23 14:40:34 UTC
Gleb,

I have a new patch ready for this. Can you please retest with
updates=http://akozumpl.fedorapeople.org/bz621175.img

and post the results. If you experience something unexpected there's a lot of information in /tmp/storage.log (search for 'edd:'), please check that and post all information you'll consider relevant.

Thank you.

Comment 53 Gleb Natapov 2011-06-27 10:09:37 UTC
(In reply to comment #52)
> Gleb,
> 
> I have a new patch ready for this. Can you please retest with
> updates=http://akozumpl.fedorapeople.org/bz621175.img
> 
> and post the results. If you experience something unexpected there's a lot of
> information in /tmp/storage.log (search for 'edd:'), please check that and post
> all information you'll consider relevant.
> 
> Thank you.

Works perfectly for me!

Comment 54 Ales Kozumplik 2011-06-29 11:50:13 UTC
Patch is awaiting review: https://www.redhat.com/archives/anaconda-devel-list/2011-June/msg00202.html

Comment 55 Ales Kozumplik 2011-07-04 08:03:53 UTC
Fixed by 2145e5d30ce6a1950a24995731f741240b45b13f on the rhel6-branch.

Comment 58 Ales Kozumplik 2011-07-04 08:42:11 UTC
Fixed on master by bcdbe9d7c757b9030216d427e96fa044c8658038.

Comment 61 Alexander Todorov 2011-08-10 13:55:56 UTC
Created attachment 517612 [details]
tarball with logs from failed install

Tested with anaconda-13.21.126 (latest nightly) using latest 6.2 as KVM host and latest 6.2 as KVM guest. 

On the guest (using virt-manager GUI) I attached 3 disks: 1 virtio and 2 IDE disks. 

I performed manual install selecting all default options in the UI. 

Anaconda UI showed that boot loader will be installed on the first IDE disk and it was named /dev/sda. 

However after restart the system is not able to boot. The console was showing the message:
Booting from Hard Disk ...

Comment 62 Alexander Todorov 2011-08-10 13:56:29 UTC
This doesn't look fixed so moving back to ASSIGNED.

Comment 63 Gleb Natapov 2011-08-10 14:01:58 UTC
(In reply to comment #61)
> Created attachment 517612 [details]
> tarball with logs from failed install
> 
> Tested with anaconda-13.21.126 (latest nightly) using latest 6.2 as KVM host
> and latest 6.2 as KVM guest. 
> 
> On the guest (using virt-manager GUI) I attached 3 disks: 1 virtio and 2 IDE
> disks. 
> 
> I performed manual install selecting all default options in the UI. 
> 
> Anaconda UI showed that boot loader will be installed on the first IDE disk and
> it was named /dev/sda. 
> 
> However after restart the system is not able to boot. The console was showing
> the message:
> Booting from Hard Disk ...
What command line virt-manager used to start the guest (use ps to check)? If there is boot=on there somewhere it is virt-manager bug.

Comment 64 Ales Kozumplik 2011-08-10 14:03:05 UTC
(In reply to comment #62)
> This doesn't look fixed so moving back to ASSIGNED.

Alex, how does the same configuration behave in 6.1? It is hard to believe this bug caused this kind of regression (while it has been confirmed by the reporter in comment 53 this fixes his issues).

Comment 65 Alexander Todorov 2011-08-10 14:06:29 UTC
(In reply to comment #63)

> What command line virt-manager used to start the guest (use ps to check)? If
> there is boot=on there somewhere it is virt-manager bug.


Using ps after the boot failed (i.e. after install) shows:

/usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name t8 -uuid 037efd2d-9ef3-f0b4-34e5-edae17b60300 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/t8.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -drive file=/var/lib/libvirt/images/disk1,if=none,id=drive-virtio-disk0,format=raw,cache=none -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/var/lib/libvirt/images/t1.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive file=/var/lib/libvirt/images/t2.img,if=none,id=drive-ide0-0-1,format=raw,cache=none -device ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev tap,fd=20,id=hostnet0,vhost=on,vhostfd=21 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:31:9d:7f,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6



There's no boot=on as far as I can see.

Comment 66 Alexander Todorov 2011-08-10 14:18:57 UTC
(In reply to comment #64)
> (In reply to comment #62)
> > This doesn't look fixed so moving back to ASSIGNED.
> 
> Alex, how does the same configuration behave in 6.1? It is hard to believe this
> bug caused this kind of regression (while it has been confirmed by the reporter
> in comment 53 this fixes his issues).

I'm having troubles getting a 6.1 system to re-test. Will let you know ASAP.

Comment 67 Gleb Natapov 2011-08-10 14:19:45 UTC
(In reply to comment #65)
> (In reply to comment #63)
> 
> > What command line virt-manager used to start the guest (use ps to check)? If
> > there is boot=on there somewhere it is virt-manager bug.
> 
> 
> Using ps after the boot failed (i.e. after install) shows:
Can you verify that during install command line was the same?

> 
> /usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 1024 -smp
> 1,sockets=1,cores=1,threads=1 -name t8 -uuid
> 037efd2d-9ef3-f0b4-34e5-edae17b60300 -nodefconfig -nodefaults -chardev
> socket,id=charmonitor,path=/var/lib/libvirt/qemu/t8.monitor,server,nowait -mon
> chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -drive
> file=/var/lib/libvirt/images/disk1,if=none,id=drive-virtio-disk0,format=raw,cache=none
> -device
> virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
> -drive
> file=/var/lib/libvirt/images/t1.img,if=none,id=drive-ide0-0-0,format=raw,cache=none
> -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive
> file=/var/lib/libvirt/images/t2.img,if=none,id=drive-ide0-0-1,format=raw,cache=none
> -device ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev
> tap,fd=20,id=hostnet0,vhost=on,vhostfd=21 -device
> virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:31:9d:7f,bus=pci.0,addr=0x3
> -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0
> -usb -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device
> intel-hda,id=sound0,bus=pci.0,addr=0x4 -device
> hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device
> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
> 
> 
> 
> There's no boot=on as far as I can see.
That was just a guess :). Now I see that virt-manager uses bootindex exactly like it should and it marks virtio disk as bootable, so anaconda should have chosen /dev/vda to install bootloader.

Comment 68 Alexander Todorov 2011-08-10 14:27:18 UTC
Using 6.2 host with 6.1 guest and the same disk setup I got the same result. Guest failed to boot. Still have to test on 6.1 host with 6.1 guest.

Comment 69 Alexander Todorov 2011-08-10 14:33:40 UTC
Command line during install is:

/usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name t8 -uuid bd2e000a-2aac-2a4d-c0aa-9499b070cee4 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/t8.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-reboot -no-shutdown -kernel /var/lib/libvirt/boot/virtinst-vmlinuz.dmUjOZ -initrd /var/lib/libvirt/boot/virtinst-initrd.img.T2k92F -append method=http://qafiler.bos.redhat.com/redhat/nightly/RHEL6.2-20110808.n.0/6/Server/x86_64/os -drive file=/var/lib/libvirt/images/disk1,if=none,id=drive-virtio-disk0,format=raw,cache=none -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -drive file=/var/lib/libvirt/images/t1.img,if=none,id=drive-ide0-0-0,format=raw,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive file=/var/lib/libvirt/images/t2.img,if=none,id=drive-ide0-0-1,format=raw,cache=none -device ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev tap,fd=20,id=hostnet0,vhost=on,vhostfd=21 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:e1:cc:22,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

Comment 70 Ales Kozumplik 2011-08-10 14:33:51 UTC
Relevant part of the log:

13:49:34,430 INFO    : edd: collected mbr signatures: {'vda': '0x0007f5ad', 'sda': '0x0002f6cf', 'sdb': '0x0007718f'}
13:49:34,431 DEBUG   : edd: data extracted from 0x80:
        type: ATA, ata_device: 0
        channel: 0, mbr_signature: 0x0002f6cf
        pci_dev: 00:01.1, scsi_id: None
        scsi_lun: None, sectors: 8388608
13:49:34,431 DEBUG   : edd: matched 0x80 to sda using pci_dev
13:49:34,431 DEBUG   : edd: data extracted from 0x81:
        type: ATA, ata_device: 1
        channel: 0, mbr_signature: 0x0007718f
        pci_dev: 00:01.1, scsi_id: None
        scsi_lun: None, sectors: 16777216
13:49:34,431 DEBUG   : edd: matched 0x81 to sdb using pci_dev
13:49:34,432 DEBUG   : edd: data extracted from 0x82:
        type: SCSI, ata_device: None
        channel: 0, mbr_signature: 0x0007f5ad
        pci_dev: 00:05.0, scsi_id: 0
        scsi_lun: 0, sectors: 20971520
13:49:34,433 DEBUG   : edd: matched 0x82 to vda using pci_dev

This shows that the sda is the 0x80 edd device (the mbr signature also matches) so the correct device was picked and so the correct boot device *was chosen*. What we are seeing is either a bootloader bug, anaconda bootloader install procedure bug or even a virt manager bug. But 621175 looks fixed to me.

Comment 71 Alexander Todorov 2011-08-10 14:35:52 UTC
This is the diff between the two. They differ but not much (used 2 different guests to get the command line so MAC/UUID is different).

--- ps.during_install	2011-08-10 10:33:54.524300008 -0400
+++ ps.after_install	2011-08-10 10:32:24.971026477 -0400
@@ -1,22 +1,18 @@
 /usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 1024 -smp
 1,sockets=1,cores=1,threads=1 -name t8 -uuid
-bd2e000a-2aac-2a4d-c0aa-9499b070cee4 -nodefconfig -nodefaults -chardev
+037efd2d-9ef3-f0b4-34e5-edae17b60300 -nodefconfig -nodefaults -chardev
 socket,id=charmonitor,path=/var/lib/libvirt/qemu/t8.monitor,server,nowait -mon
-chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-reboot
--no-shutdown -kernel /var/lib/libvirt/boot/virtinst-vmlinuz.dmUjOZ -initrd
-/var/lib/libvirt/boot/virtinst-initrd.img.T2k92F -append
-method=http://qafiler.bos.redhat.com/redhat/nightly/RHEL6.2-20110808.n.0/6/Server/x86_64/os
--drive
+chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -drive
 file=/var/lib/libvirt/images/disk1,if=none,id=drive-virtio-disk0,format=raw,cache=none
 -device
-virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
+virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
 -drive
 file=/var/lib/libvirt/images/t1.img,if=none,id=drive-ide0-0-0,format=raw,cache=none
 -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive
 file=/var/lib/libvirt/images/t2.img,if=none,id=drive-ide0-0-1,format=raw,cache=none
 -device ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev
 tap,fd=20,id=hostnet0,vhost=on,vhostfd=21 -device
-virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:e1:cc:22,bus=pci.0,addr=0x3
+virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:31:9d:7f,bus=pci.0,addr=0x3
 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0
 -usb -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device
 intel-hda,id=sound0,bus=pci.0,addr=0x4 -device

Comment 72 Gleb Natapov 2011-08-10 14:47:35 UTC
> -virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0
> +virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1

This is where the bug is! During installation virtio disk has no bootindex parameter so BIOS choose first ide as bootable. After install virt-manager tells bios to boot from virtio disk and this obviously fails. Please open the bug against virt-manager (or may be libvirt if virt-manager uses it to start guests).

Comment 73 Alexander Todorov 2011-08-10 15:02:52 UTC
Per comments #70 and #72 I'm moving this BZ to VERIFIED. Opened bug #729694 to address the virt-manager issue.

Comment 74 errata-xmlrpc 2011-12-06 10:26:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1565.html