Bug 527035 - CreateException: Could not stat device /dev/mapper/jmicron_GRAID_DEVELOPING - No such file or directory.
CreateException: Could not stat device /dev/mapper/jmicron_GRAID_DEVELOPING -...
Status: CLOSED RAWHIDE
Product: Fedora
Classification: Fedora
Component: parted (Show other bugs)
rawhide
x86_64 Linux
low Severity medium
: ---
: ---
Assigned To: Hans de Goede
Fedora Extras Quality Assurance
anaconda_trace_hash:2abe7d18f29ee9b6a...
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-10-03 11:08 EDT by Bob Schultz
Modified: 2009-10-21 09:30 EDT (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-10-10 10:59:55 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Attached traceback automatically from anaconda. (118.76 KB, text/plain)
2009-10-03 11:08 EDT, Bob Schultz
no flags Details
anaconda log from first failed "updated" test install (1.91 KB, application/octet-stream)
2009-10-05 13:20 EDT, Bob Schultz
no flags Details
syslog from failed attempted "update" install (64.39 KB, application/octet-stream)
2009-10-05 13:22 EDT, Bob Schultz
no flags Details
Attached traceback automatically from anaconda. (123.71 KB, text/plain)
2009-10-05 14:30 EDT, Bob Schultz
no flags Details
Contains naconda.log, boot.log, program.log, storage.log, yum.log Comment#12 (468.82 KB, text/plain)
2009-10-06 12:37 EDT, Bob Schultz
no flags Details
Gparted snapshot of /boot partition in extended partition. (60.40 KB, image/png)
2009-10-06 15:07 EDT, Bob Schultz
no flags Details
F11.92 /boot partition successfully installed in extended partition (79.84 KB, image/png)
2009-10-10 10:13 EDT, Bob Schultz
no flags Details
Attached traceback automatically from anaconda. (79.65 KB, text/plain)
2009-10-21 09:30 EDT, Alexander Stevenson
no flags Details

  None (edit)
Description Bob Schultz 2009-10-03 11:08:48 EDT
The following was filed automatically by anaconda:
anaconda 12.32 exception report
Traceback (most recent call first):
  File "/usr/lib64/python2.6/site-packages/parted/geometry.py", line 136, in intersect
    return Geometry(PedGeometry=self.__geometry.intersect(b.getPedGeometry()))
  File "/usr/lib64/python2.6/site-packages/parted/decorators.py", line 30, in localeC
    ret = fn(*args, **kwds)
  File "<string>", line 2, in intersect
  File "/usr/lib/anaconda/storage/partitioning.py", line 515, in getBestFreeSpaceRegion
    free_geom = extended.geometry.intersect(_range)
  File "/usr/lib/anaconda/storage/partitioning.py", line 751, in allocatePartitions
    boot=_part.req_bootable)
  File "/usr/lib/anaconda/storage/partitioning.py", line 601, in doPartitioning
    allocatePartitions(disks, partitions)
  File "/usr/lib/anaconda/iw/partition_gui.py", line 1020, in refresh
    doPartitioning(self.storage)
  File "/usr/lib/anaconda/iw/partition_gui.py", line 1128, in editPartition
    if self.refresh(justRedraw=not actions):
  File "/usr/lib/anaconda/iw/partition_gui.py", line 975, in newCB
    self.editPartition(device, isNew=1)
CreateException: Could not stat device /dev/mapper/jmicron_GRAID_DEVELOPING - No such file or directory.
Comment 1 Bob Schultz 2009-10-03 11:08:52 EDT
Created attachment 363563 [details]
Attached traceback automatically from anaconda.
Comment 2 Bob Schultz 2009-10-03 11:22:25 EDT
This bug is related to my bugs 526568 and 526063. This time install failed trying to establish first linux partition in unallocated space in extended partition on jmicron raid device. This time:

Fresh raid was created in bios with all spaces filled (to eliminate any potential whitespace problems in device name parsing).

MSDOS partition was created FAT16 ~24MB, bootable.

400GB extended partition created.

200GB ntfs partition created in extended partition.

Vista 64bit installed in ntfs partition.

Attempt to install 9/30 rawhide iso.

Get to custom layout for disk. Begin by trying to make boot partition in first unallocated space after ntfs partition. Set size for /boot at 400MB, and install fails per dump.
Comment 3 Hans de Goede 2009-10-05 08:45:43 EDT
The backtrace you are seeing in this bug is caused by a bug in pyparted,
which is described here:
https://bugzilla.redhat.com/show_bug.cgi?id=495433#c33
And has a patch in comment 34 of that bug.

I've been looking through the pyparted sources and found several more places where
the same problem exists.

Can you please see if this bug is reproducable? (IOW if you try doing the exact
same thing again, if it then happens again) ?

And if it is reproduceable, please try again using this updates.img:
http://people.fedoraproject.org/~jwrdegoede/updates-527035-i686.img

Simply pass
updates=http://people.fedoraproject.org/~jwrdegoede/updates-527035-i686.img
on the syslinux(bootloader) cmdline when starting anaconda.

This contains a fixed pyparted, this may completely fix your issue, or reveal the
real cause of the problem. This depends on if the real error which happens earlier then this backtrace but pyparted failed to catch, is one which can be
ignored, or is fatal.
Comment 4 Bob Schultz 2009-10-05 12:01:20 EDT
Bug is reproducible. Attempted updated install per instructions. Install failed with multiline traceback ending in:

ImportError: /tmp/updates/_pedmodule.so: wrong ELF Class: ELFCLASS32

Fixup appears to be 32bit i686, needs 64bit x86_64.
Comment 5 Bob Schultz 2009-10-05 13:20:47 EDT
Created attachment 363717 [details]
anaconda log from first failed "updated" test install
Comment 6 Hans de Goede 2009-10-05 13:21:36 EDT
sorry my bad, I though I looked into your anaconda logfile and saw you were using 32 bit i686, but clearly I was looking at someone else's logfile. Here is an x86_64 updates.img:
http://people.fedoraproject.org/~jwrdegoede/updates-527035-x86_64.img

Thanks & Regards,

Hans
Comment 7 Bob Schultz 2009-10-05 13:22:15 EDT
Created attachment 363718 [details]
syslog from failed attempted "update" install
Comment 8 Bob Schultz 2009-10-05 14:30:10 EDT
Created attachment 363728 [details]
Attached traceback automatically from anaconda.
Comment 9 Bob Schultz 2009-10-05 14:51:52 EDT
Hans,

The fixup failed as shown in the anaconda dump. As a point of interest, I timed the "searching for hard drives" activity. It took fully 60 seconds for the hard disk activity light to flash for the first time, and about 10 seconds more for that process to complete. Seems like an extraordinary amount of time to discover the hard drives. There are no other hard drives or USB sticks in the system. I wonder if some sort of keyword directive should be available to "kick" the install in the right direction, like the "dmraid=true" keyword on debian. Also, in the dump you can see recurring errors reported as:
<6>sr 4:0:0:0: [sr0] Unhandled sense code
<6>sr 4:0:0:0: [sr0] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
<6>sr 4:0:0:0: [sr0] Sense Key : Medium Error [current] 
<6>sr 4:0:0:0: [sr0] Add. Sense: L-EC uncorrectable error
<3>end_request: I/O error, dev sr0, sector 346104
<3>Buffer I/O error on device sr0, logical block 86526
<3>Buffer I/O error on device sr0, logical block 86527
<6>sr 4:0:0:0: [sr0] Unhandled sense code
<6>sr 4:0:0:0: [sr0] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
<6>sr 4:0:0:0: [sr0] Sense Key : Medium Error [current] 
<6>sr 4:0:0:0: [sr0] Add. Sense: L-EC uncorrectable error
<3>end_request: I/O error, dev sr0, sector 346104

Those same type errors are reported in F11 (which never gets even close to installing on this system) and other linux distributions and gparted live cd's. The DVD drive is a Samsung on the Intel AHCI sata port. These persistent error messages delay the install process of various live cd's, some of which eventually conclude successfully. I don't know if this has any relevance to the problem at hand, but I thought I'd point it out anyway.

Regards,

Bob Schultz
Comment 10 Hans de Goede 2009-10-06 04:42:10 EDT
Bob, note this comment is mainly for fellow developers, not for you :)

AARGGGHHHHHH, the traceback is still the same, that means:
https://bugzilla.redhat.com/show_bug.cgi?id=495433#c33
Is still happening at yet another place. Ok so 2 hours of wading through pyparted / libparted code, I think I've found the cause:

In various places the anaconda code logs method calls including the passed in arguments, in some places these arguments are partitions, this causes
pyparted's Partition.__str__ method to get called, which is fine, but leads to the following call trace (manually constructed):

-In pyparted's partition.py:
Partition.__str__
self.busy
self.__partition.is_busy()
-In pyparted's pydisk.c:
py_ped_partition_is_busy(self)
ped_partition_is_busy(part)
-In libparted disk.c:
ped_partition_is_busy(part)
ped_architecture->disk_ops->partition_is_busy (part)
-In libparted arch/linux.c:
linux_partition_is_busy(part)
_partition_is_mounted (part)
_partition_get_part_dev (part)
_device_stat (part->disk->dev, &dev_stat)

And this last call is sets a PartedException:
Could not stat device /dev/foo - No such file or directory.

So the problem is that ped_partition_is_busy() calls libparted's exception handler, which it really should not do.

As pyparted does not expect libparted's exception handler to be called from
ped_partition_is_busy() it does not check partedExnRaised() at this point and
later trips over it in py_ped_geometry_intersect(), causing the backtrace seen
in this bug.

It is to be expected we only see this problem with dmraid setups as normal
disks don't come and go during install as the virtual dmraid devices do. No idea
why I'm unable to reproduce this on my own dmraid hardware though.

Working on a parted patch to stop ped_partition_is_busy() from calling libparted's exception handler now.
Comment 11 Hans de Goede 2009-10-06 05:47:04 EDT
Bob,

I've put an updated updates.img here:
http://people.fedoraproject.org/~jwrdegoede/updates-527035-x86_64.img

(same location as last time) I did make sure its an x86_64 updates.img this time. This one has both an updated pyparted and an updated libparted, together they should hopefully really really fix the issue you are seeing.

Thanks for testing and for your patience.

Regards,

Hans
Comment 12 Bob Schultz 2009-10-06 11:27:44 EDT
Hans,

Where to start? This has gotten more complicated and confused. I used the latest update. This time it appeared to work, just that the mouse became non-functioning in anaconda with the pointer stuck in the middle of the screen. I had to use keyboard for all operations. Proceeded to declare /boot, / and swap in the unallocated space in extended partition right after ntfs. Install appeared to work. Grub did get installed to dmraid, but multiple problems arose:

1)Haven't been able to get to "firstboot" graphical yet. X server refuses proper startup. "nouveau init table command not found message" issued 4 times. Got a graphical menu of sorts, login button at bottom of screen resulted in a repeat of same screen due to a failure to initiate.

2)Partition Magic 8.01 no longer processes drive. Partition table invalid. Was ok before F12 test.

3)Grub menu for "other os" died. Thought I had lost the whole "other" installation. rootnoverify (hd0,4) turned out to be wrong. Needed (hd0,0).

4)Installer did not place /boot in extended partition as directed. Installer incorrectly created a Primary partition immediately following extended partition. Everything, except the FAT primary partition, should be in the extended partition. 

At the grub menu, tab completion of (hd0, yields:

Part. No.    FS Type        Part. Type
0              FAT              0x4
2              ext2fs           0x83
4              unknown          0x7
5              ext2fs           0x83
6              unknown          0x83

In a F12 shell, running parted, command print yields:
Invalid partition table on /dev/sda - wrong signature 0

There are only 3 entries then displayed.


Command dmraid -ay listed the member partitions correctly as active, but interjects "The dynamic shared library "libdmraid-events-jmicron.so" could not be loaded...no such file.

5)Was able to get shell window in F12. Attempted to fix x server probs., no luck yet. Working on it.

Will try to upload some log files, maybe I'll zip up /var/log for efficiency.

Regards,

Bob
Comment 13 Bob Schultz 2009-10-06 12:37:02 EDT
Created attachment 363853 [details]
Contains naconda.log, boot.log, program.log, storage.log, yum.log Comment#12

Concatenated text file with anaconda.log, boot.log, program.log, storage.log, yum.log for /var/log regarding Comment #12.
Comment 14 Hans de Goede 2009-10-06 13:50:49 EDT
(In reply to comment #12)
> Hans,
> 
> Where to start? This has gotten more complicated and confused. I used the
> latest update. This time it appeared to work

Great, so it seems that we've finally nailed the problem (the backtrace you posted) you filed this bug for. I'm going to do a new parted release (the package were the cause lays), and then close this bug. As for your remaining
issues see below.

I understand you are not a happy Fedora user yet, and that there are still issues, please do not take the closing of this bug as to mean that I think
everything is resolved. But we've got 14 comments in this bug already and it
really helps to track one issue per bug, to keep the discussion inside the
tickets focussed.

> just that the mouse became
> non-functioning in anaconda with the pointer stuck in the middle of the screen.
> I had to use keyboard for all operations. Proceeded to declare /boot, / and
> swap in the unallocated space in extended partition right after ntfs. Install
> appeared to work. Grub did get installed to dmraid, but multiple problems
> arose:
> 
> 1)Haven't been able to get to "firstboot" graphical yet. X server refuses
> proper startup. "nouveau init table command not found message" issued 4 times.
> Got a graphical menu of sorts, login button at bottom of screen resulted in a
> repeat of same screen due to a failure to initiate.
> 

This together with your mouse pointer stuck, I think are a problem with Xorg
on your card, please file a bug for this against component xorg-x11-drv-nouveau

> 2)Partition Magic 8.01 no longer processes drive. Partition table invalid. Was
> ok before F12 test.
> 

Hmm, nasty, could you perhaps install Fedora on some other more simple system and run Partition Magic 8.01 there ? We no longer align partition bounderies
on cylinder limits, as that really is no necessary any more now a days
(neither does Vista iirc). Maybe Partition Magic dislikes this, also is 8 the latest PM release ?

> 3)Grub menu for "other os" died. Thought I had lost the whole "other"
> installation. rootnoverify (hd0,4) turned out to be wrong. Needed (hd0,0).
> 

Ok, so we choose the ntfs partition instead of the dos one, which usually
is correct, as the dos one usually is some sort of OEM rescue partition,
note that if you do manual partitioning you do get a chance to edit
the grub entries during install. So I consider this one NOTABUG, but if
you try this again, and find that you cannot fix this from the
grub configuration screen inside anaconda, that would be a bug, in that
case please file a new bug against anaconda to track this.

> 4)Installer did not place /boot in extended partition as directed. Installer
> incorrectly created a Primary partition immediately following extended
> partition. Everything, except the FAT primary partition, should be in the
> extended partition. 

Yes, anaconda always places /boot in a primary partitin, I must admit I do not
know exactly why we do this, please file a separate bug against anaconda for this, I've been wondering for a while if there is any reason to force /boot
on a primary partition, I guess some one else in the anaconda team will know.

> At the grub menu, tab completion of (hd0, yields:
> 
> Part. No.    FS Type        Part. Type
> 0              FAT              0x4
> 2              ext2fs           0x83
> 4              unknown          0x7
> 5              ext2fs           0x83
> 6              unknown          0x83
> 

That looks fine, the 4 unknown is your NTFS partition and the 6 unknown is swap.

> In a F12 shell, running parted, command print yields:
> Invalid partition table on /dev/sda - wrong signature 0
> 
> There are only 3 entries then displayed.
> 

You should not not run parted on /dev/sda directly, sda is a raw disk which
is part of a raid set, please run it on
/dev/mapper/jmicron_GRAID_DEVELOPING

Which represents the raidset, at which level the partitioning is happening
(and which level grub talks to the disk, grub is using BIOS INT 13 which gets
intercepted by the jmicron BIOS and provides access to the set.

If it barfs on that too, you've already guessed it, please file a separate bug to track that.

> 
> Command dmraid -ay listed the member partitions correctly as active, but
> interjects "The dynamic shared library "libdmraid-events-jmicron.so" could not
> be loaded...no such file.
> 

This is "correct", there is no libdmraid-events-jmicron.so, only a
libdmraid-events-isw.so, which is for Intel BIOS RAID sets in dmraid. One could argue that dmraid should not complain in this case, as this is expected to happen. I even agree with that, please file a bug against dmraid for this.

> 5)Was able to get shell window in F12. Attempted to fix x server probs., no
> luck yet. Working on it.
> 

As said, file a bug against xorg-x11-drv-nouveau, I'm sure Ben (who works for
RedHat in the nouveau driver fulltime) will do his best to get things fixed, and
will be happy with the report.
Comment 15 Bob Schultz 2009-10-06 15:06:11 EDT
Hans,

Thanks for all your work. I am confused by this statement though:

"Yes, anaconda always places /boot in a primary partitin, I must admit I do not
know exactly why we do this, please file a separate bug against anaconda for
this, I've been wondering for a while if there is any reason to force /boot
on a primary partition, I guess some one else in the anaconda team will know."

My experience has been totally different. Ever since the earlier RH days, followed by Fedora up to F11, I have always placed the boot into an extended partition. The laptop I writing on now had /boot in extended partition. Is the fact that I pre-partitioned and pre-formatted the difference? I'll include a snapshot png of this laptop as displayed by gparted as my most accessible example.

Regards,

Bob
Comment 16 Bob Schultz 2009-10-06 15:07:56 EDT
Created attachment 363881 [details]
Gparted snapshot of /boot partition in extended partition.

Gparted snapshot of /boot in extended partition.
Comment 17 Hans de Goede 2009-10-06 15:46:37 EDT
(In reply to comment #15)
> Hans,
> 
> Thanks for all your work. I am confused by this statement though:
> 
> "Yes, anaconda always places /boot in a primary partition, ..."

Let me clear that up, the new anaconda (after the F-11 storage rewrite), always
places /boot in a primary partition. If we did not force this before and
it worked fine having it as a logical partiton (which makes sense to me, I ws a bit surprised by this behaviour of the new code myself), then please file a
bug new bug about this against anaconda.

Then hopefully the person responsible for the code for forcing a primary partition
for /boot will get back to you. Be sure to mention you had /boot inside the extended partition before (with F-10 and earlier) and that it worked fine there.
Comment 18 Hans de Goede 2009-10-06 17:49:13 EDT
This is fixed in parted-1.9.0-16.fc12, which is available here:
http://koji.fedoraproject.org/koji/buildinfo?buildID=135375

A tag request to get this included for the beta is here:
https://fedorahosted.org/rel-eng/ticket/2366
Comment 19 Bob Schultz 2009-10-10 10:13:42 EDT
Created attachment 364344 [details]
F11.92 /boot partition successfully installed in extended partition

The is a follow-up to Han's comment #17. I reinstalled rawhide on bios raid by 1)running gparted from 10/8 desktop live cd and deleting all linux partitions, followed by creation and formatting of /boot, /, and swap in extended partition. Gparted gave a spurious error msg. and incorrection completion code after each creation, but nevertheless did create the partitions properly. 2)Installing rawhide boot.iso 9/30 using Han's updates-527035-x86_64.img file, and manually choosing the pre-created partitions. Installation went smoothly. Reboot was successful, including x. Only fixup required (trivial) was (hd0,0) instead of (hd4,0), as previously explained by Hans. Thanks to everyone for successfully solving this problem.
Comment 20 Hans de Goede 2009-10-10 10:59:55 EDT
Bob, thanks for the feedback, closing.
Comment 21 Alexander Stevenson 2009-10-21 09:30:06 EDT
Created attachment 365510 [details]
Attached traceback automatically from anaconda.

Note You need to log in before you can comment on or make changes to this bug.