The following has be reported by IBM LTC: [BETA] Insufficient space on /usr volume to install all products Please fill in each of the sections below. Hardware Environment: zSeries 9064-116, shark e20 Software Environment: taroon beta 1 under VM Steps to Reproduce: 1. put / on one volume 3390-3 volume 2. put /usr on a second 3390-3 volume 3. install EVERYTHING! Actual Results: Message: You don't appear to have enough disk space to install the packages you've selected. You need more space on the following file systems: /usr 1169M Expected Results: Either some notion during the install process how big each package is to pick and choose by size or some methodology to cut /usr into more reasonable chunks Additional Information: The stand disk size, after formatting, of the standard 3390-3 image is about 2.3 GB. The message implies to me that /usr is then greater than 3.4 GB! Many installations are bound by this "small" disk size alternative because they are using older hardware or they have an installation standard of a 3390-3 image on their RVA's or Sharks, or for performance reasons, they do not want too much data behind a single disk - subchannel - for performance reasons. The only alternative I can think of in installing a full system is to keep installing pieces, then moving these pieces to some other dasd using soft links from /usr. /usr seems to be growing in leaps and bounds. If I had 30GB disk, no problem. But in the zSeries world, 2.3 GB is still the standard.In addition, when you specify, the install process loops. It only gives you the option to say ok, which then goes back, reformats the file system, and gives the same message! It should at least take you back to the point where you specify the packages and let you deselect some.After install of a much smaller set of choices, I did a df command Filesystem 1k-blocks Used Available Use% Mounted on /dev/dasda1 2366164 114756 2131212 6% / /dev/dasdb1 2366164 1613388 632580 72% /usr So it looks like almost everything is dumped into the /usr directory! /opt and /misc are empty.Jim - if you have a standard 2.3 GB disk, maybe you should not install EVERYTHING ? However, your Comment#1 is a problem and needs to be fixed by Red Hat.In the zSeries world, IBM set the standard geometry as 3390 when they went from 3390 to RAMACI in the late '80s. At that time, the device size was also 3390-3 (2.7 GB), so that became the defacto standard in the industry. Later devices (RAMACII-III, RVA, RSA, Shark, and our competitors - EMC, hitachi) have pretty much kept to this standard because that is what the MVS device driver is set for. With some of the devices, such as RVA or Shark, you can increase the size of the volume, most notably to 3390-9 (3 times larger than a 3390-3), but, unless you have PAV's, you take a performance hit. Under linux, the actual amount of data is 4096 byte blocks times 12 blocks per track or 49152 bytes (48K) per track not the 56K you often hear because of the ECKD overhead. That makes a 3390-3 of 3339 tracks (15 tracks/cyl) 2347.73 MB. A 3390-9, the next popular size would be 7043.2 GB. This 48K track is not implemented directly on the track today as with real 3390, but, since RAMAC I, this is really a virtual mapping to a SCSI device or raid array. It is a logical, not physical construct. Why not change it? Because the customers of IBM have stated that they do not want anymore changes to the logical format. So, this standard, 3390-3, has in some sense solidified in the industry on zSeries. There is one device per subchannel. The way you get to a lot of data is that you have a lot of subchannels/devices. To get to 1.2 TB of data is 512 3390-3 class devices or 171 3390-9 class devices. Each subchannel is either ESCON at 10 MB/sec or FICON at 40/MB sec, though ESCON probably predominates still today. Therein lies the problem. Without PAV's, you are confined to one file transferring data at a time. With PAV's, you can one file per PAV transferring data at a time. 2.4 GB behind a 10 MB/sec pipe is one thing. 7 GB behind a 10 MB/sec pipe is quite another. So many people are reluctant to put go with larger volumes. You will note that the notion of larger volumes on Shark did not really get interesting until PAV's came around because putting a lot of active data on a larger volume turns out to be a bottleneck in the hardware architecture. They are now implementing 32GB devices. But that is not feasible without 1) PAV's and 2) FICON. Otherwise, you spend most of your I/O time queued behind the subchannel, waiting for it to be free - huge performance bottleneck. So, people who are not running the latest hardware really cannot do justice to large volumes and get adequate performance. And if, as a Linux customer, you have an old device that does not have the capability of larger volume, you are pretty much stuck. That is why we do our system volume testing for DB2 on 3390-3 volumes - because then we know that it will accomodate the vast majority of customers, old and new. When we try and force a configuration with larger devices, there is a lot of customer resistance for various reasons. So RedHat and the other distributors need to take this into account when implementing these systems in a zSeries environment. How can we load the a full system on a series of 3390-3 images? Jim - thanks for your good explanation! Glen/Greg - Two issues here for Red Hat to consider/fix: 1. How can we install a full system (everything) on a series of standard 3390-3 volumes (which is about 2.3GB) ? 2. When we specify which packages to install, and if there is not enough disk space, the install process loops around. It should go back and allow the user to re-specify which packages to install again. Thanks.
We provide the capability to use either software raid or LVM so you can get larger logical volumes out of ridiculously small drives.
------ Additional Comments From jlsibley.com 2003-01-08 22:07 ------- Ridiculous size volumes is purely a matter of opinion. Is Red Hat willing to make this statement to the customers who want to buy RHEL for zSeries? Generally, from what I have read in the Marist forum indicates that no-one really wants to use raid or LVM for their root volumes because of the unreliability during recovery situations. A more intelligent approach would be to allow installation of the software to alternate directories. Only the stuff that really needs to be there during bootup needs to be in /usr.
I think the key point here is that the user chose to install "EVERYTHING!". Even in taroon this is quite a lot of packages, and most of the bits go into /usr. If the user has chosen the default package set, perhaps adding in a few key apps, there would have been no problems. Additionally as Jeremy helpfully pointed out, you can use either software raid or LVM to glue smallish partitions into larger ones.
------ Additional Comments From jlsibley.com 2003-04-08 21:03 ------- Judging from the replies from RedHat, it seems they choose to take approach that it is a "user problem" rather the recognizing the real problems and constraints working with zSeries DASD. They should be thinking about creative solutions needed to resolve the issue than a "one size fits all" approach. First of all, I did pare things down significantly, and needed 78% of a 3390-3 volume to fit, for our data center, a useful system. We build a single instance of a distribution, customize it for this data center, then distribute it to all our users rather than building a customized system for each user. Than manpower would be prohibitive to customize each instance, so our compromise is to use a larger system that what would fit any one users needs. Since we have 40 active systems in LPAR mode and probably another 60 systems in VM/EC mode, using RH 7.1, RH 7.2, SuSE SLES7, and SuSE SLES8, we try to think beyond the PC mentality. As to the popularity of LVM for volumes needed during boot, RedHat ought to review the thread entitled Root "LVM - Two questions" in the Marist Forum http://www2.marist.edu/htbin/wlvgl?L=LINUX-VM&LOG=LOG0307 . The posters on Marist do not think that LVM for boot volumes is a very good idea. As to the size of zSeries volumes, that will continue to a performance constraint until PAV's are available because of the architecture of the hardware. If some of the elements of /usr were not needed during boot, they ought to be somewhere else. It makes no sense to put source, for example, in the same directory that one needs to start programs provided by RedHat but not officially part of the kernel. It is interesing to note that the stuff required for the kernel is only about 168 mb. The other several GB of stuff is dumped into /usr by the vendor.
------ Additional Comments From jlsibley.com 2003-04-08 21:11 ------- There also seems to an assumption that I have to have one copy of /usr per instance in the Linux design. Clearly, with many instances, this is a waste of disk space. A notion that we are working on, but does not seem to fit the LVM approach to /usr is a shared /usr. These should be, according to the posix spec, read only files. Certain kinds of things (such as the source) could be place on other volumes and mounted by fstab as ro during boot, saving the amount of space we need for dozens of copies of a single distribution. It appears to me that LVM really doesn't like to share volumes.
changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mranweil.com Status|REJECTED |CLOSED ------- Additional Comments From mranweil.com 2005-06-14 19:10 EDT ------- Non-duplicate rejected bug that's been rejected for over 6 months. Marking closed, you can re-open if needed.