Bug 838792

Summary: Can't mount ext3 filesystem when drive is in USB enclosure
Product: [Fedora] Fedora Reporter: Slawomir Czarko <slawomir.czarko>
Component: kernelAssignee: fs-maint
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 17CC: esandeen, gansalmon, itamar, jforbes, jonathan, kernel-maint, madhu.chinakonda, rwheeler
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-03-25 15:08:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
fdisk output
none
sfdisk output none

Description Slawomir Czarko 2012-07-10 07:44:14 UTC
Created attachment 597229 [details]
fdisk output

Description of problem:

I have a disk with one partition which is formatted with ext3. When it's connected via SATA I can mount it without problems. When it's connected via USB I cannot mount it.

USB enclosure is shown by lsusb as:
ID 040d:6208 VIA Technologies, Inc. 

I am able to use the enclosure with another disk which has two partitions, one with ext4 and one with LVM physical volume.

dmesg shows this when mount fails:
[   48.933594] EXT4-fs (sdb1): mounting ext3 file system using the ext4 subsystem
[   48.934513] EXT4-fs (sdb1): bad geometry: block count 29304950 exceeds size of device (29304949 blocks)

and when it successds:
[  133.292619] EXT4-fs (sda1): mounting ext3 file system using the ext4 subsystem
[  133.296731] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)

fdisk and sfdisk show good partition table for that drive (attached).

Version-Release number of selected component (if applicable):
kernel-3.4.4-3.fc17.x86_64

How reproducible:
100%

Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

ext3 partition was formatted with Fedora 15.

Comment 1 Slawomir Czarko 2012-07-10 07:44:47 UTC
Created attachment 597230 [details]
sfdisk output

Comment 2 Justin M. Forbes 2012-12-07 17:07:31 UTC
Is this still an issue with 3.6.9 kernels?

Comment 3 Slawomir Czarko 2012-12-12 10:40:19 UTC
(In reply to comment #2)
> Is this still an issue with 3.6.9 kernels?

Yes.

kernel version 3.6.9-2.fc17.x86_64

Comment 4 Josh Boyer 2013-03-14 18:31:17 UTC
Is this still a problem with 3.8.2 in updates-testing?

Comment 5 Slawomir Czarko 2013-03-25 14:45:58 UTC
(In reply to comment #4)
> Is this still a problem with 3.8.2 in updates-testing?

It's still happening with 3.8.3-103.fc17.x86_64

[ 3029.907217] usb 2-2: new high-speed USB device number 4 using ehci-pci
[ 3030.027447] usb 2-2: New USB device found, idVendor=040d, idProduct=6208
[ 3030.027459] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 3030.027468] usb 2-2: Product: USB 2.0 SATA Bridge
[ 3030.027475] usb 2-2: Manufacturer: VIA Technologies Inc.
[ 3030.027482] usb 2-2: SerialNumber: F07043A31373
[ 3030.029271] scsi6 : usb-storage 2-2:1.0
[ 3031.033331] scsi 6:0:0:0: Direct-Access     Corsair  Force 3 SSD      1.3. PQ: 0 ANSI: 2
[ 3031.035650] sd 6:0:0:0: Attached scsi generic sg2 type 0
[ 3031.035871] sd 6:0:0:0: [sdb] 234441645 512-byte logical blocks: (120 GB/111 GiB)
[ 3031.036382] sd 6:0:0:0: [sdb] Write Protect is off
[ 3031.036392] sd 6:0:0:0: [sdb] Mode Sense: 00 00 00 00
[ 3031.036865] sd 6:0:0:0: [sdb] Asking for cache data failed
[ 3031.036876] sd 6:0:0:0: [sdb] Assuming drive cache: write through
[ 3031.039246] sd 6:0:0:0: [sdb] Asking for cache data failed
[ 3031.039257] sd 6:0:0:0: [sdb] Assuming drive cache: write through
[ 3031.044731]  sdb: sdb1
[ 3031.044738] sdb: p1 size 234439600 extends beyond EOD, enabling native capacity
[ 3031.046481] sd 6:0:0:0: [sdb] Asking for cache data failed
[ 3031.046486] sd 6:0:0:0: [sdb] Assuming drive cache: write through
[ 3031.049669]  sdb: sdb1
[ 3031.049676] sdb: p1 size 234439600 extends beyond EOD, truncated
[ 3031.051847] sd 6:0:0:0: [sdb] Asking for cache data failed
[ 3031.051853] sd 6:0:0:0: [sdb] Assuming drive cache: write through
[ 3031.051858] sd 6:0:0:0: [sdb] Attached SCSI disk
[ 3031.264004] EXT4-fs (sdb1): mounting ext3 file system using the ext4 subsystem
[ 3031.264605] EXT4-fs (sdb1): bad geometry: block count 29304950 exceeds size of device (29304949 blocks)

Comment 6 Eric Sandeen 2013-03-25 15:00:53 UTC
> bad geometry: block count 29304950 exceeds size of device (29304949 blocks)

Do you know which kernel you mkfs'd this under?  At that point it must have been able to see the whole device.

Spot checking between then and now to see when it regressed might be helpful.

Also I'm wondering what these mean:

> sdb: p1 size 234439600 extends beyond EOD, enabling native capacity
> sdb: p1 size 234439600 extends beyond EOD, truncated

sure looks like it thinks the first partition extends beyond the end of the disk.

(IOW, this isn't a filesystem problem, this is a block layer problem, I'm afraid)

At one point some of the partitioning tools would poke geometries directly into the kernel; I suppose it's possible that the partitioning tool was off by one, it created a too-big partition in memory, stuffed it into the kernel, and then mkfs.ext3 used that space which was one past the end of the disk (I don't think mkfs.ext3 actually reads or writes the last block, so it wouldn't notice)

What is the actual drive (model number) in the enclosure?  I'm wondering how big it is in reality.

sfdisk sure thinks the partition is too big:

> 20 heads, 1 sectors/track, 11722082 cylinders, total 234441645 sectors
but:
>    Device Boot      Start         End      Blocks   Id  System
> /dev/sdb1            2048   234441647   117219800   83  Linux

end is > total sectors.

FWIW, we can probably get your filesystem back in action; it should be possible to use debugfs to change the total number of blocks in the superblock, chopping off the last one or two,and it'd mount.  This is a bit dangerous, so don't do it unless you're comfortable.  Making a dd image first might not be a bad idea.

I would not do this until we've exhausted debugging efforts, but:

# debugfs -w /dev/sdb1
debugfs: stats
...
Block count:              29304950
...
debugfs: set_super_value blocks_count 29304949
debugfs: quit
# e2fsck -fy /dev/sdb1

and that should fix up the block counts.

If it breaks, you get to keep both pieces.  :)

-Eric

Comment 7 Ric Wheeler 2013-03-25 15:08:56 UTC
This is a limitation of the USB connected devices - they cannot address large storage. 

If you use e-sata, you will be fine.

Not a kernel issue, this is a USB enclosure issue.

Comment 8 Eric Sandeen 2013-03-25 15:13:18 UTC
Oh, I missed that it worked outside the USB enclosure.

Please *do not* do my debugfs steps ;)

But I'm not sure this is NOTABUG - it might be a usb firmware bug, but it could also be a usb storage stack bug?

The fs is only ~120G, that should be well within the USB limits.

Comment 9 Slawomir Czarko 2013-03-25 15:19:11 UTC
The filesystem works fine with the same machine if I connect the drive via SATA instead of via USB enclosure. Via SATA it also works fine in another machine.

When connected via SATA:

fdisk reports 234441648 sectors

sfdisk -s on that machine shows 117220824 which also translates into 234441648 sectors

dmesg output:
[    1.276330] sd 1:0:0:0: [sdb] 234441648 512-byte logical blocks: (120 GB/111 GiB)
[    1.276457] sd 1:0:0:0: [sdb] Write Protect is off
[    1.276459] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[    1.276491] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    1.276915]  sdb: sdb1
[    1.277256] sd 1:0:0:0: [sdb] Attached SCSI disk


(In reply to comment #7)
> This is a limitation of the USB connected devices - they cannot address
> large storage. 
> 
> If you use e-sata, you will be fine.
> 
> Not a kernel issue, this is a USB enclosure issue.

120 GB should be within the limit of the USB, right? Or are you saying that it's a limitation of this particular enclosure not of USB in general?

Comment 10 Slawomir Czarko 2013-03-25 15:20:54 UTC
Btw, with the same enclosure when trying to connect another disk (320 GB) I get:

[ 2445.532957] ata4: exception Emask 0x10 SAct 0x0 SErr 0x90202 action 0xe frozen
[ 2445.532967] ata4: irq_stat 0x00400000, PHY RDY changed
[ 2445.532975] ata4: SError: { RecovComm Persist PHYRdyChg 10B8B }
[ 2445.532987] ata4: hard resetting link
[ 2446.410123] ata4: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 2446.428583] ata4.00: configured for UDMA/133
[ 2446.428595] ata4: EH complete
[ 2517.092137] usb 2-4: new high-speed USB device number 2 using ehci_hcd
[ 2517.212557] usb 2-4: New USB device found, idVendor=040d, idProduct=6208
[ 2517.212566] usb 2-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 2517.212574] usb 2-4: Product: USB 2.0 SATA Bridge
[ 2517.212579] usb 2-4: Manufacturer: VIA Technologies Inc.
[ 2517.212584] usb 2-4: SerialNumber: A1F774751283
[ 2517.546318] Initializing USB Mass Storage driver...
[ 2517.546456] scsi6 : usb-storage 2-4:1.0
[ 2517.546538] usbcore: registered new interface driver usb-storage
[ 2517.546539] USB Mass Storage support registered.
[ 2518.548925] scsi 6:0:0:0: Direct-Access     TOSHIBA  MK3252GSX        LV01 PQ: 0 ANSI: 2
[ 2518.551226] sd 6:0:0:0: Attached scsi generic sg4 type 0
[ 2518.551514] sd 6:0:0:0: [sde] 625142445 512-byte logical blocks: (320 GB/298 GiB)
[ 2518.552049] sd 6:0:0:0: [sde] Write Protect is off
[ 2518.552060] sd 6:0:0:0: [sde] Mode Sense: 00 00 00 00
[ 2518.552637] sd 6:0:0:0: [sde] Asking for cache data failed
[ 2518.552649] sd 6:0:0:0: [sde] Assuming drive cache: write through
[ 2518.556882] sd 6:0:0:0: [sde] Asking for cache data failed
[ 2518.556886] sd 6:0:0:0: [sde] Assuming drive cache: write through
[ 2518.557847] sd 6:0:0:0: [sde]  Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 2518.557850] sd 6:0:0:0: [sde]  Sense Key : Illegal Request [current] 
[ 2518.557853] sd 6:0:0:0: [sde]  Add. Sense: Invalid field in cdb
[ 2518.557856] sd 6:0:0:0: [sde] CDB: Read(10): 28 00 00 00 00 00 00 00 01 00 00 00
[ 2518.557860] end_request: I/O error, dev sde, sector 0
[ 2518.557863] Buffer I/O error on device sde, logical block 0
[ 2518.558866] sd 6:0:0:0: [sde]  Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 2518.558869] sd 6:0:0:0: [sde]  Sense Key : Illegal Request [current] 
[ 2518.558871] sd 6:0:0:0: [sde]  Add. Sense: Invalid field in cdb
[ 2518.558874] sd 6:0:0:0: [sde] CDB: Read(10): 28 00 00 00 00 01 00 00 01 00 00 00
[ 2518.558879] end_request: I/O error, dev sde, sector 1
[ 2518.558882] Buffer I/O error on device sde, logical block 1
[ 2518.559849] sd 6:0:0:0: [sde]  Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 2518.559851] sd 6:0:0:0: [sde]  Sense Key : Illegal Request [current] 
[ 2518.559854] sd 6:0:0:0: [sde]  Add. Sense: Invalid field in cdb
[ 2518.559856] sd 6:0:0:0: [sde] CDB: Read(10): 28 00 00 00 00 02 00 00 06 00 00 00
[ 2518.559861] end_request: I/O error, dev sde, sector 2
[ 2518.559866] Buffer I/O error on device sde, logical block 2
[ 2518.559868] Buffer I/O error on device sde, logical block 3
[ 2518.559870] Buffer I/O error on device sde, logical block 4
[ 2518.559871] Buffer I/O error on device sde, logical block 5
[ 2518.559873] Buffer I/O error on device sde, logical block 6
[ 2518.559874] Buffer I/O error on device sde, logical block 7
[ 2518.560843] sd 6:0:0:0: [sde]  Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 2518.560845] sd 6:0:0:0: [sde]  Sense Key : Illegal Request [current] 
[ 2518.560847] sd 6:0:0:0: [sde]  Add. Sense: Invalid field in cdb
[ 2518.560848] sd 6:0:0:0: [sde] CDB: Read(10): 28 00 00 00 00 00 00 00 01 00 00 00
[ 2518.560852] end_request: I/O error, dev sde, sector 0
[ 2518.560854] Buffer I/O error on device sde, logical block 0
[ 2518.561845] sd 6:0:0:0: [sde]  Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 2518.561847] sd 6:0:0:0: [sde]  Sense Key : Illegal Request [current] 
[ 2518.561849] sd 6:0:0:0: [sde]  Add. Sense: Invalid field in cdb
[ 2518.561851] sd 6:0:0:0: [sde] CDB: Read(10): 28 00 00 00 00 01 00 00 01 00 00 00
[ 2518.561855] end_request: I/O error, dev sde, sector 1
[ 2518.562842] sd 6:0:0:0: [sde]  Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[ 2518.562844] sd 6:0:0:0: [sde]  Sense Key : Illegal Request [current] 
[ 2518.562846] sd 6:0:0:0: [sde]  Add. Sense: Invalid field in cdb
[ 2518.562848] sd 6:0:0:0: [sde] CDB: Read(10): 28 00 00 00 00 02 00 00 06 00 00 00
[ 2518.562851] end_request: I/O error, dev sde, sector 2

And these errors repeat.

Comment 11 Eric Sandeen 2013-03-25 15:26:49 UTC
That looks like a dying disk... Or, bad USB firmware I suppose.

Out of curiosity can you include the kernel messages related to the drive discovery when the original 120G drive (Force 3 SSD?) is connected directly via SATA?

Comment 12 Ric Wheeler 2013-03-25 15:28:38 UTC
I have a ton of USB/e-sata enclosures and they vary a lot in quality. You should try out both disks with other enclosures?

Comment 13 Slawomir Czarko 2013-03-25 16:53:00 UTC
Force 3 SSD mounts OK with a different enclosure:

[  108.266181] usb 2-4: new high-speed USB device number 3 using ehci-pci
[  108.382064] usb 2-4: New USB device found, idVendor=152d, idProduct=2339
[  108.382076] usb 2-4: New USB device strings: Mfr=1, Product=2, SerialNumber=5
[  108.382084] usb 2-4: Product: USB to ATA/ATAPI Bridge
[  108.382091] usb 2-4: Manufacturer: JMicron
[  108.382098] usb 2-4: SerialNumber: 120665000000
[  108.475589] Initializing USB Mass Storage driver...
[  108.475775] scsi5 : usb-storage 2-4:1.0
[  108.475882] usbcore: registered new interface driver usb-storage
[  108.475885] USB Mass Storage support registered.
[  109.477384] scsi 5:0:0:0: Direct-Access     Corsair  Force 3 SSD           PQ: 0 ANSI: 2 CCS
[  109.479683] sd 5:0:0:0: Attached scsi generic sg2 type 0
[  109.479747] sd 5:0:0:0: [sdb] 234441648 512-byte logical blocks: (120 GB/111 GiB)
[  109.480248] sd 5:0:0:0: [sdb] Write Protect is off
[  109.480258] sd 5:0:0:0: [sdb] Mode Sense: 00 38 00 00
[  109.480752] sd 5:0:0:0: [sdb] Asking for cache data failed
[  109.480762] sd 5:0:0:0: [sdb] Assuming drive cache: write through
[  109.483121] sd 5:0:0:0: [sdb] Asking for cache data failed
[  109.483132] sd 5:0:0:0: [sdb] Assuming drive cache: write through
[  109.485743]  sdb: sdb1
[  109.487848] sd 5:0:0:0: [sdb] Asking for cache data failed
[  109.487854] sd 5:0:0:0: [sdb] Assuming drive cache: write through
[  109.487858] sd 5:0:0:0: [sdb] Attached SCSI disk
[  109.767028] EXT4-fs (sdb1): mounting ext3 file system using the ext4 subsystem
[  109.777444] EXT4-fs (sdb1): mounted filesystem with ordered data mode. Opts: (null)
[  109.778024] SELinux: initialized (dev sdb1, type ext3), uses xattr

320GB looks like it is dying. Please ignore comment #10

Comment 14 Slawomir Czarko 2013-03-25 18:43:43 UTC
I am able to mount a smaller (80 GB) disk in the original USB enclosure:

[ 7029.697027] usb 2-4: new high-speed USB device number 4 using ehci_hcd
[ 7029.817205] usb 2-4: New USB device found, idVendor=040d, idProduct=6208
[ 7029.817208] usb 2-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 7029.817210] usb 2-4: Product: USB 2.0 SATA Bridge
[ 7029.817212] usb 2-4: Manufacturer: VIA Technologies Inc.
[ 7029.817213] usb 2-4: SerialNumber: A0E713941323
[ 7030.070756] Initializing USB Mass Storage driver...
[ 7030.074042] scsi6 : usb-storage 2-4:1.0
[ 7030.074300] usbcore: registered new interface driver usb-storage
[ 7030.074302] USB Mass Storage support registered.
[ 7031.080855] scsi 6:0:0:0: Direct-Access     ST980813 AS               3.AD PQ: 0 ANSI: 2
[ 7031.082553] sd 6:0:0:0: Attached scsi generic sg4 type 0
[ 7031.082596] sd 6:0:0:0: [sde] 156301485 512-byte logical blocks: (80.0 GB/74.5 GiB)
[ 7031.083090] sd 6:0:0:0: [sde] Write Protect is off
[ 7031.083093] sd 6:0:0:0: [sde] Mode Sense: 00 00 00 00
[ 7031.083591] sd 6:0:0:0: [sde] Asking for cache data failed
[ 7031.083594] sd 6:0:0:0: [sde] Assuming drive cache: write through
[ 7031.086216] sd 6:0:0:0: [sde] Asking for cache data failed
[ 7031.086219] sd 6:0:0:0: [sde] Assuming drive cache: write through
[ 7031.105602]  sde: sde1 sde2
[ 7031.109211] sd 6:0:0:0: [sde] Asking for cache data failed
[ 7031.109215] sd 6:0:0:0: [sde] Assuming drive cache: write through
[ 7031.109217] sd 6:0:0:0: [sde] Attached SCSI disk
[ 7031.633111] EXT4-fs (sde1): mounted filesystem with ordered data mode. Opts: (null)
[ 7031.642018] SELinux: initialized (dev sde1, type ext4), uses xattr

$ fdisk -l /dev/sde
Disk /dev/sde: 80.0 GB, 80026360320 bytes
255 heads, 63 sectors/track, 9729 cylinders, total 156301485 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000bff5b

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1   *        2048     1026047      512000   83  Linux
/dev/sde2         1026048   156301311    77637632   8e  Linux LVM

Comment 15 Eric Sandeen 2013-03-25 19:44:48 UTC
So the VIA enclosure says:

[ 3031.035871] sd 6:0:0:0: [sdb] 234441645 512-byte logical blocks: (120 GB/111 GiB)

but the JMicron enclosure says:

[  109.479747] sd 5:0:0:0: [sdb] 234441648 512-byte logical blocks: (120 GB/111 GiB)


(3 more sectors)

*shrug* seems like a problem w/ the enclosure itself I guess.