Description of problem: Guests are limited to 16 disk devices. Version-Release number of selected component (if applicable): RHEL 5 up through RHEL 5.2 This is derived from bug 219298, which asked for > 16 Scsi devices. That issue has been closed but the issue of a 16 disk device limit needs to be looked at.
Unfortunately, this one might be tough. Looking at http://www.lanana.org/docs/device-list/devices-2.6+.txt, I can see that block major 202 has been assigned to Xen virtual block devices. However, the minor numbering goes something like: /dev/xvda = 0 /dev/xvda1 = 1 ... /dev/xvda15 = 15 /dev/xvdb = 16 /dev/xvdb1 = 17 ... /dev/xvdp = 240 ... /dev/xvdp17 = 255 As you can see, this is 16 whole disks, plus up to 15 partitions per disk. Adding additional block devices is going to require expanding into another major number, which might be possible, but has to be done carefully. Chris Lalancette
OK, just out of curiousity, I saw what I could do to increase the block devices. The method I used is completely inappropriate in a stable product, but I just wanted to see what would happen, and what exactly I needed to change. Basically I cut the number of minor numbers available to each /dev/xvd? device down to 8, which gave me a maximum of 32 device numbers. Just for posterity, though, there are actually 2 parts that need to change: 1) in the guest kernel, you need to change drivers/xen/blkfront/vbd.c to either increase the number of majors or decrease the number of minors 2) in the xen tools, you need to change python/xen/util/blkif.py to edit the blkdev_name_to_number method to fix up the re.match() for the /dev/xvd devices. Again, this was just proof of concept, but what I proved is that any solution will require at least the above 2 changes. Chris Lalancette
It would also require changes to libvirt to make the blockstats method caculate the correct device IDs for extracting stats from xenstore/sysfs.
[RE: release notes] Calvin, is this limitation true for guests that are paravirtualized, fully-virtualized, or both?
This has been tested on a paravirtualized system. I suspect that this would not be an issue on a fully virtualized system since it would have access to the full sd* block device numbers (instead of doing the weird things to the sd names that the paravirtualized system is doing after 26 characters), but I'm not entirely sure.
understood. for now, added to "Feature Updates => Virtualization => Known Issues": <quote> Paravirtualized guests can only have a maximum of 16 disk devices. </quote> please advise if any further revisions are required. note that any further revisions will go into the release notes updates, not the distro. thanks!
Has collaboration with upstream produced any approach to the fix for this yet? I don't want this to miss scheduling for 5.3.
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux maintenance release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux Update release for currently deployed products. This request is not yet committed for inclusion in an Update release.
I just posted a patch series upstream: http://lists.xensource.com/archives/html/xen-devel/2008-06/msg00709.html Assuming upstream takes this, we can then backport it into RHEL-5. I'll keep you updated on the status. Chris Lalancette
*** Bug 437349 has been marked as a duplicate of this bug. ***
Created attachment 312451 [details] Expand SCSI devices to support sdaa and up
Created attachment 312452 [details] Expand blkfront to recognize the new expanded xvd devices
Created attachment 312453 [details] Expand blktap to understand the new IOCTL and handle new and old userland
in kernel-2.6.18-104.el5 You can download this test kernel from http://people.redhat.com/dzickus/el5
Release note added. If any revisions are required, please set the "requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: Paravirtualized guests can only have a maximum of 16 disk devices.
Release note updated. If any revisions are required, please set the "requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. Diffed Contents: @@ -1 +1,3 @@ -Paravirtualized guests can only have a maximum of 16 disk devices.+In previous versions of RedHat virtualization, paravirtualized guests could have a maximum of 16 disks attached. + +This set of patches increases the number of disks a paravirtualized guest can have to a maximum of 256 disks.
Release note updated. If any revisions are required, please set the "requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. Diffed Contents: @@ -1,3 +1 @@ -In previous versions of RedHat virtualization, paravirtualized guests could have a maximum of 16 disks attached. +Previously, paravirtualized guests could only have a maximum of 16 disk devices. In this update, this limit has been increased to a maximum of 256 disk devices.- -This set of patches increases the number of disks a paravirtualized guest can have to a maximum of 256 disks.
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2009-0225.html