Bug 442723

Unfortunately, this one might be tough.  Looking at
http://www.lanana.org/docs/device-list/devices-2.6+.txt, I can see that block
major 202 has been assigned to Xen virtual block devices.  However, the minor
numbering goes something like:

/dev/xvda = 0
/dev/xvda1 = 1
...
/dev/xvda15 = 15
/dev/xvdb = 16
/dev/xvdb1 = 17
...
/dev/xvdp = 240
...
/dev/xvdp17 = 255

As you can see, this is 16 whole disks, plus up to 15 partitions per disk. 
Adding additional block devices is going to require expanding into another major
number, which might be possible, but has to be done carefully.

Chris Lalancette

OK, just out of curiousity, I saw what I could do to increase the block devices.
 The method I used is completely inappropriate in a stable product, but I just
wanted to see what would happen, and what exactly I needed to change.  Basically
I cut the number of minor numbers available to each /dev/xvd? device down to 8,
which gave me a maximum of 32 device numbers.  Just for posterity, though, there
are actually 2 parts that need to change:

1) in the guest kernel, you need to change drivers/xen/blkfront/vbd.c to either
increase the number of majors or decrease the number of minors

2)  in the xen tools, you need to change python/xen/util/blkif.py to edit the
blkdev_name_to_number method to fix up the re.match() for the /dev/xvd devices.

Again, this was just proof of concept, but what I proved is that any solution
will require at least the above 2 changes.

Chris Lalancette

It would also require changes to libvirt to make the blockstats method caculate
the correct device IDs for extracting stats from xenstore/sysfs.

[RE: release notes] Calvin, is this limitation true for guests that are
paravirtualized, fully-virtualized, or both?

This has been tested on a paravirtualized system. I suspect that this would not
be an issue on a fully virtualized system since it would have access to the full
sd* block device numbers (instead of doing the weird things to the sd names that
the paravirtualized system is doing after 26 characters), but I'm not entirely sure.

understood. for now, added to "Feature Updates => Virtualization => Known Issues":

<quote>
Paravirtualized guests can only have a maximum of 16 disk devices.
</quote>

please advise if any further revisions are required. note that any further
revisions will go into the release notes updates, not the distro. thanks!

Has collaboration with upstream produced any approach to the fix for this yet? 
I don't want this to miss scheduling for 5.3.

This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

I just posted a patch series upstream:

http://lists.xensource.com/archives/html/xen-devel/2008-06/msg00709.html

Assuming upstream takes this, we can then backport it into RHEL-5.  I'll keep
you updated on the status.

Chris Lalancette

*** Bug 437349 has been marked as a duplicate of this bug. ***

Created attachment 312451 [details]
Expand SCSI devices to support sdaa and up

Created attachment 312452 [details]
Expand blkfront to recognize the new expanded xvd devices

Created attachment 312453 [details]
Expand blktap to understand the new IOCTL and handle new and old userland

in kernel-2.6.18-104.el5
You can download this test kernel from http://people.redhat.com/dzickus/el5

in kernel-2.6.18-104.el5
You can download this test kernel from http://people.redhat.com/dzickus/el5

Release note added. If any revisions are required, please set the 
"requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

New Contents:
Paravirtualized guests can only have a maximum of 16 disk devices.

Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1 +1,3 @@
-Paravirtualized guests can only have a maximum of 16 disk devices.+In previous versions of RedHat virtualization, paravirtualized guests could have a maximum of 16 disks attached.
+
+This set of patches increases the number of disks a paravirtualized guest can have to a maximum of 256 disks.

Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1,3 +1 @@
-In previous versions of RedHat virtualization, paravirtualized guests could have a maximum of 16 disks attached.
+Previously, paravirtualized guests could only have a maximum of 16 disk devices. In this update, this limit has been increased to a maximum of 256 disk devices.-
-This set of patches increases the number of disks a paravirtualized guest can have to a maximum of 256 disks.

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2009-0225.html