Bug 1415798
| Summary: | mount.ntfs doing execssive IO at startup | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | bug_fedora |
| Component: | ntfs-3g | Assignee: | Tom "spot" Callaway <tcallawa> |
| Status: | CLOSED NOTABUG | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 24 | CC: | jean-pierre.andre, tcallawa |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-01-24 21:03:15 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
> When I try to run fsck.ntfs, it reutnrs "Error getting bit
> value for record nnnnnn thousands of times before i CTRL-C
> out of it.
It might be useful you mention the file number leading to the error, and the layout of that file and of the MFT :
ntfsinfo -fvi nnnnnn /dev/sda12
ntfsinfo -fvi 0 /dev/sda12
Note : fsck.ntfs is a sketch of a consistency checker which does nothing useful and was orphaned ten years ago.
>It might be useful you mention the file number leading to the error, and the >layout of that file and of the MFT : It seems to be *every file*. nnnnnn increments through every number. By the time I execute and hit CTRL-C, is has already listed 1-33145. If I let it go for a few seconds, it goes past 100,000, Here is a sample of the layout of 1 of the files: [kd1yv3@KD1YV3 ~]$ sudo ntfsinfo -fvi 152001 /dev/sda12 Forced to continue. Dumping Inode 152001 (0x251c1) Upd. Seq. Array Off.: 48 (0x30) Upd. Seq. Array Count: 3 (0x3) Upd. Seq. Number: 1306 (0x51a) LogFile Seq. Number: 0x0 MFT Record Seq. Numb.: 106 (0x6a) Number of Hard Links: 1 (0x1) Attribute Offset: 56 (0x38) MFT Record Flags: IN_USE Bytes Used: 712 (0x2c8) bytes Bytes Allocated: 1024 (0x400) bytes Next Attribute Instance: 5 (0x5) MFT Padding: 00 00 Dumping attribute $STANDARD_INFORMATION (0x10) from mft record 152001 (0x251c1) Attribute length: 72 (0x48) Resident: Yes Name length: 0 (0x0) Name offset: 0 (0x0) Attribute flags: 0x0000 Attribute instance: 0 (0x0) Data size: 48 (0x30) Data offset: 24 (0x18) Resident flags: 0x00 ReservedR: 0 (0x0) File Creation Time: Wed Mar 5 22:33:19 2014 UTC File Altered Time: Fri Feb 28 16:10:13 2014 UTC MFT Changed Time: Wed Mar 5 22:33:19 2014 UTC Last Accessed Time: Wed Mar 5 22:33:19 2014 UTC File attributes: ARCHIVE (0x00000020) Dumping attribute $FILE_NAME (0x30) from mft record 152001 (0x251c1) Attribute length: 104 (0x68) Resident: Yes Name length: 0 (0x0) Name offset: 0 (0x0) Attribute flags: 0x0000 Attribute instance: 4 (0x4) Data size: 76 (0x4c) Data offset: 24 (0x18) Resident flags: 0x01 ReservedR: 0 (0x0) Parent directory: 147884 (0x241ac) File Creation Time: Wed Mar 5 22:33:19 2014 UTC File Altered Time: Fri Feb 28 16:10:13 2014 UTC MFT Changed Time: Wed Mar 5 22:33:19 2014 UTC Last Accessed Time: Wed Mar 5 22:33:19 2014 UTC Allocated Size: 344 (0x158) Data Size: 343 (0x157) Filename Length: 5 (0x5) File attributes: ARCHIVE (0x00000020) Namespace: POSIX Filename: 'fiq.S' Dumping attribute $SECURITY_DESCRIPTOR (0x50) from mft record 152001 (0x251c1) Attribute length: 104 (0x68) Resident: Yes Name length: 0 (0x0) Name offset: 0 (0x0) Attribute flags: 0x0000 Attribute instance: 1 (0x1) Data size: 80 (0x50) Data offset: 24 (0x18) Resident flags: 0x00 ReservedR: 0 (0x0) Revision: 1 Control: 0x8004 Owner SID: S-1-5-32-544 Group SID: S-1-5-32-544 System ACL: missing Discretionary ACL: Revision 2 ACE: type:allow flags:0x3 access:0x1f01ff SID: S-1-1-0 Dumping attribute $DATA (0x80) from mft record 152001 (0x251c1) Attribute length: 368 (0x170) Resident: Yes Name length: 0 (0x0) Name offset: 0 (0x0) Attribute flags: 0x0000 Attribute instance: 2 (0x2) Data size: 343 (0x157) Data offset: 24 (0x18) Resident flags: 0x00 ReservedR: 0 (0x0) End of inode reached Total runs: 0 (fragments: 0) === Here is the MFT === [kd1yv3@KD1YV3 ~]$ sudo ntfsinfo -fvi 0 /dev/sda12 Forced to continue. Dumping Inode 0 (0x0) Upd. Seq. Array Off.: 48 (0x30) Upd. Seq. Array Count: 3 (0x3) Upd. Seq. Number: 61880 (0xf1b8) LogFile Seq. Number: 0x90473fa MFT Record Seq. Numb.: 1 (0x1) Number of Hard Links: 1 (0x1) Attribute Offset: 56 (0x38) MFT Record Flags: IN_USE Bytes Used: 952 (0x3b8) bytes Bytes Allocated: 1024 (0x400) bytes Next Attribute Instance: 4 (0x4) MFT Padding: 00 00 Dumping attribute $STANDARD_INFORMATION (0x10) from mft record 0 (0x0) Attribute length: 96 (0x60) Resident: Yes Name length: 0 (0x0) Name offset: 24 (0x18) Attribute flags: 0x0000 Attribute instance: 0 (0x0) Data size: 72 (0x48) Data offset: 24 (0x18) Resident flags: 0x00 ReservedR: 0 (0x0) File Creation Time: Mon Jan 1 00:00:00 1601 UTC File Altered Time: Mon Jan 1 00:00:00 1601 UTC MFT Changed Time: Mon Jan 1 00:00:00 1601 UTC Last Accessed Time: Mon Jan 1 00:00:00 1601 UTC File attributes: HIDDEN SYSTEM (0x00000006) Maximum versions: 0 Version number: 0 Class ID: 0 User ID: 0 (0x0) Security ID: 0 (0x0) Quota charged: 0 (0x0) Update Sequence Number: 0 (0x0) Dumping attribute $FILE_NAME (0x30) from mft record 0 (0x0) Attribute length: 104 (0x68) Resident: Yes Name length: 0 (0x0) Name offset: 24 (0x18) Attribute flags: 0x0000 Attribute instance: 2 (0x2) Data size: 74 (0x4a) Data offset: 24 (0x18) Resident flags: 0x01 ReservedR: 0 (0x0) Parent directory: 5 (0x5) File Creation Time: Mon Jan 1 00:00:00 1601 UTC File Altered Time: Mon Jan 1 00:00:00 1601 UTC MFT Changed Time: Mon Jan 1 00:00:00 1601 UTC Last Accessed Time: Mon Jan 1 00:00:00 1601 UTC Allocated Size: 921993216 (0x36f48000) Data Size: 921993216 (0x36f48000) Filename Length: 4 (0x4) File attributes: HIDDEN SYSTEM (0x00000006) Namespace: Win32 & DOS Filename: '$MFT' Dumping attribute $DATA (0x80) from mft record 0 (0x0) Attribute length: 536 (0x218) Resident: No Name length: 0 (0x0) Name offset: 64 (0x40) Attribute flags: 0x0000 Attribute instance: 1 (0x1) Lowest VCN 0 (0x0) Highest VCN: 280367 (0x4472f) Mapping pairs offset: 64 (0x40) Compression unit: 0 (0x0) Data size: 1148387328 (0x44730000) Allocated size: 1148387328 (0x44730000) Initialized size: 1148387328 (0x44730000) Runlist: VCN LCN Length 0x0 0x4 0x4000 0x4000 0xd84d9f 0x45b9 0x85b9 0x1a6289 0x6c47 0xf200 0x1d160dc 0x4006 0x13206 0x5f5e364 0xb6e4 0x1e8ea 0x443d7b3 0x8fe 0x1f1e8 0x44470b0 0x46c 0x1f654 0x487f7b1 0xfff 0x20653 0x55a28a4 0xffc 0x2164f 0x3f60aae 0xffa 0x22649 0x39c0236 0xffa 0x23643 0x452f865 0x4e5 0x23b28 0x455c819 0xfef 0x24b17 0x451a8b1 0x5a1 0x250b8 0x45453f3 0x4c4 0x2557c 0x4581829 0xfe7 0x26563 0x41b97c7 0xfe9 0x2754c 0x327bd18 0xb30 0x2807c 0x329539e 0x474 0x284f0 0x32c9b9f 0xfc1 0x294b1 0x514205f 0x1bc9 0x2b07a 0x499387c 0x3da 0x2b454 0x49d790b 0x568 0x2b9bc 0x4a0da78 0x1d40 0x2d6fc 0x3ce8f20 0xfc8 0x2e6c4 0x51b9c41 0xfef 0x2f6b3 0x6d33144 0xfac 0x3065f 0x304de73 0xfcd 0x3162c 0x45d283f 0xfc1 0x325ed 0x5a99341 0xff3 0x335e0 0x240c732 0x4 0x335e4 0x241340f 0x279 0x3385d 0x27f6dcb 0xf3d 0x3479a 0x3d6aefa 0xffe 0x35798 0x5189c4e 0xfe2 0x3677a 0x497aa7e 0x152 0x368cc 0x5504c46 0xfea 0x378b6 0x483373b 0xf5d 0x38813 0x1d54d4a 0x2 0x38815 0x1d597b3 0x2 0x38817 0x1d70aa4 0x4 0x3881b 0x1d79579 0x5 0x38820 0x1d7eabf 0xd 0x3882d 0x1d80291 0xa 0x38837 0x1db5611 0x4 0x3883b 0x1dbbe60 0x8 0x38843 0x1dbd7ea 0x8 0x3884b 0x1dbf239 0x6 0x38851 0x1dc1286 0x6 0x38857 0x1dc7a86 0x6 0x3885d 0x1df7348 0x8 0x38865 0x1df9511 0x1 0x38866 0x1e084d3 0x6 0x3886c 0x1e16449 0x9 0x38875 0x1e582d4 0x7 0x3887c 0x25d6f95 0x6 0x38882 0x25ed88d 0x23 0x388a5 0x262c8f4 0x1 0x388a6 0x26346ed 0x22 0x388c8 0x26dbad7 0x4 0x388cc 0x26f6ad8 0x1 0x388cd 0x26fbbca 0x1 0x388ce 0x2707c3c 0x8 0x388d6 0x2722aa6 0x6 0x388dc 0x273ceca 0x9 0x388e5 0x2751fdb 0x2 0x388e7 0x275ef0b 0x36 0x3891d 0x2803d5f 0x1a8 0x38ac5 0x2ae0c71 0xd5f 0x39824 0x4151db0 0xf80 0x3a7a4 0x5a6e34e 0xfe2 0x3b786 0x2e19c4e 0xfb2 0x3c738 0x4809ad4 0xfcc 0x3d704 0x6785af8 0xfd0 0x3e6d4 0x357ab59 0xf20 0x3f5f4 0x4d28cac 0xf7c 0x40570 0x72b33e3 0xf65 0x414d5 0x3aac994 0xef4 0x423c9 0x4a27b85 0x1a69 0x43e32 0x6b173e5 0x8fe Dumping attribute $BITMAP (0xb0) from mft record 0 (0x0) Attribute length: 152 (0x98) Resident: No Name length: 0 (0x0) Name offset: 64 (0x40) Attribute flags: 0x0000 Attribute instance: 3 (0x3) Lowest VCN 0 (0x0) Highest VCN: 34 (0x22) Mapping pairs offset: 64 (0x40) Compression unit: 0 (0x0) Data size: 140184 (0x22398) Allocated size: 143360 (0x23000) Initialized size: 140184 (0x22398) Runlist: VCN LCN Length 0x0 0x2 0x1 0x1 0x4a8ce4 0x10 0x11 0x253a39c 0x2 0x13 0x4cb5c4a 0x1 0x14 0x2a79a6a 0x1 0x15 0x3ba1622 0x1 0x16 0x19c7340 0x1 0x17 0x487da21 0x2 0x19 0x5303c51 0x3 0x1c 0x19c8134 0x1 0x1d 0x4d9fdad 0x2 0x1f 0x29e87f9 0x2 0x21 0x4ca4c68 0x1 0x22 0x3bcfc8a 0x1 End of inode reached Total runs: 94 (fragments: 94) === >fsck.ntfs is a sketch of a consistency checker which does nothing useful and was orphaned ten years ago. What is a current utility to fsck on an NTFS volume? > If I let it go for a few seconds, it goes past 100,000, Your partition has grown up to 1,121,472 files, which is unusual and had led to excess fragmentation. However I do not think this is the cause of the bad throughput. The file 152001 is near the beginning (152001/8/4096 = 4.64) and is described in the MFT bitmap in cluster 0x4a8ce7, so I would like to check whether this cluster is readable (if it is not, 8*4096 files around it will also be unreadable, which is consistent which what you observe). Please try (as root) : dd if=/dev/sda12 of=some-file bs=4096 skip=4885735 count=1 (only post the errors, if any, the data is not useful) During the 20mn period when the partition is slow, did you check whether you have some process scanning the full partition (such as a virus scanner) ? During the mount stage, ntfs-3g has to compute the available space by reading the global bitmap (15MB not fragmented), but this should only last a few seconds and it is over when the mounting is done, so you must have something else accessing the partition. Also did you check your syslog ? > What is a current utility to fsck on an NTFS volume? I do not know of any open source ntfs checker. I use chkdsk (which I run on linux). >Please try (as root) :
No errors, list as follows. Oddly though the of file contains 4096 char of 0xFF.
dd if=/dev/sda12 of=jnk bs=4096 skip=4885735 count=1
1+0 records in
1+0 records out
4096 bytes (4.1 kB, 4.0 KiB) copied, 0.0123709 s, 331 kB/s
===
No virus scanners. I run CrashPlan cloud backup, let me see try disabling that. Also, KDE has that annoying akonadi that might be scanning. But neither of these show up under iotop,
===
Nothing unusual in /var/log/messages, only standard mount and umount
===
This is a long shot, but might this have anything to do with selinux, which is set to enforcing?
> Oddly though the of file contains 4096 char of 0xFF No worry, this means all files in the selected range are in use (IOW no deleted files) > I run CrashPlan cloud backup Examining among one million files which ones have to be backed up takes time. Also do you have file indexing activated ? > but might this have anything to do with selinux Unlikely, on a standard ntfs configuration there is no per-file checking by selinux. I do not see anything wrong with your partition data (apart from the fsck.ntfs behavior), I would try to make sure no process is accessing files. >> I run CrashPlan cloud backup >Examining among one million files which ones have to be backed up takes time. This seems to be the problem. As a test, twice, I disabled CrashPlan under /etc/init.d and the problem went away. I re-enabled CrashPlan and the problem reappeared. I will definitely pursue this with the folks at CrashPlan support. It is odd that the process in iotop shows as ntfs.mount rather than CrashPlan. During this investigation, I realized the reason for so many files. I have incrementally added backups of 4 Android mobile devices, using this filesystem as a target. While no single one pushed the count higher, the cumulative count more than doubled the files on this filesystem. Thanks for the tip. You got me following the right path. I think we can close out this issue now. |
Description of problem: Running FC24, KDE spin, I have 1 partition that is NTFS because the machine is dual-boot Win7, and I share the drive with both. I have used this configuration and same mount for several years, across other distros, plus FC 19 and up. I *think* this behavior started with FC24. At startup, the drive spins up and is quite busy for the first 20 minutes. I installed iotop and captured the following. You can see that the mount process is reading over 1 MB/sec. Total DISK READ : 1824.74 K/s | Total DISK WRITE : 0.00 B/s Actual DISK READ: 1492.97 K/s | Actual DISK WRITE: 0.00 B/s TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND 642 be/4 root 1824.74 K/s 0.00 B/s 0.00 % 58.88 % mount.ntfs /dev/sda12 /multimedia -o rw,umask=0077,uid=1000,gid=1000,fmask=0002,dmask=0002 Note that I can access the file on this mount even though everything is slow. The phisical drive is under 1 year old. The original was about 5 years old, so I proactively replaced it and cloned all partitions using CloneZilla. I thought it might be a "dirty" mount, so I went to Windows and ran chkdsk, but that reported clean. When I try to run fsck.ntfs, it reutnrs "Error getting bit value for record nnnnnn thousands of times before i CTRL-C out of it. here is my fstab entry for this device: UUID=ABABABABABABABAB /multimedia ntfs umask=0077,uid=1000,gid=1000,fmask=0002,dmask=0002 0 0 Version-Release number of selected component (if applicable): Fedora 24 KDE spin How reproducible: 100%, every time I start up Fedora Steps to Reproduce: 1. Turn on machine 2. Select default FC from GRUB menu 3. Disk spins very busy for about 20 minutes. See iotop results above Actual results: Disk spins very busy for about 20 minutes. See iotop results above Expected results: Disk should only be busy for maybe 1 minute at startup. Additional info: Additional info posted in fedoraforum.org === [mach3@MACH3 ~]$ sudo ntfsinfo -fvi 6 /dev/sda12 Dumping Inode 6 (0x6) Upd. Seq. Array Off.: 48 (0x30) Upd. Seq. Array Count: 3 (0x3) Upd. Seq. Number: 264 (0x108) LogFile Seq. Number: 0x0 MFT Record Seq. Numb.: 6 (0x6) Number of Hard Links: 1 (0x1) Attribute Offset: 56 (0x38) MFT Record Flags: IN_USE Bytes Used: 336 (0x150) bytes Bytes Allocated: 1024 (0x400) bytes Next Attribute Instance: 3 (0x3) MFT Padding: 00 00 Dumping attribute $STANDARD_INFORMATION (0x10) from mft record 6 (0x6) Attribute length: 96 (0x60) Resident: Yes Name length: 0 (0x0) Name offset: 24 (0x18) Attribute flags: 0x0000 Attribute instance: 0 (0x0) Data size: 72 (0x48) Data offset: 24 (0x18) Resident flags: 0x00 ReservedR: 0 (0x0) File Creation Time: Sat Jan 1 02:39:06 2011 UTC File Altered Time: Sat Jan 1 02:39:06 2011 UTC MFT Changed Time: Sat Jan 1 02:39:06 2011 UTC Last Accessed Time: Sat Jan 1 02:39:06 2011 UTC File attributes: HIDDEN SYSTEM (0x00000006) Maximum versions: 0 Version number: 0 Class ID: 0 User ID: 0 (0x0) Security ID: 256 (0x100) Quota charged: 0 (0x0) Update Sequence Number: 0 (0x0) Dumping attribute $FILE_NAME (0x30) from mft record 6 (0x6) Attribute length: 104 (0x68) Resident: Yes Name length: 0 (0x0) Name offset: 24 (0x18) Attribute flags: 0x0000 Attribute instance: 2 (0x2) Data size: 80 (0x50) Data offset: 24 (0x18) Resident flags: 0x01 ReservedR: 0 (0x0) Parent directory: 5 (0x5) File Creation Time: Sat Jan 1 02:39:06 2011 UTC File Altered Time: Sat Jan 1 02:39:06 2011 UTC MFT Changed Time: Sat Jan 1 02:39:06 2011 UTC Last Accessed Time: Sat Jan 1 02:39:06 2011 UTC Allocated Size: 15454208 (0xebd000) Data Size: 15451776 (0xebc680) Filename Length: 7 (0x7) File attributes: HIDDEN SYSTEM (0x00000006) Namespace: Win32 & DOS Filename: '$Bitmap' Dumping attribute $DATA (0x80) from mft record 6 (0x6) Attribute length: 72 (0x48) Resident: No Name length: 0 (0x0) Name offset: 64 (0x40) Attribute flags: 0x0000 Attribute instance: 1 (0x1) Lowest VCN 0 (0x0) Highest VCN: 3772 (0xebc) Mapping pairs offset: 64 (0x40) Compression unit: 0 (0x0) Data size: 15451776 (0xebc680) Allocated size: 15454208 (0xebd000) Initialized size: 15451776 (0xebc680) Runlist: VCN LCN Length 0x0 0xebc687 0xebd End of inode reached Total runs: 1 (fragments: 1) === [mach3@MACH3 ~]$ sudo ntfsinfo -fm /dev/sda12 [sudo] password for mach3: Volume Information Name of device: /dev/sda12 Device state: 11 Volume Name: Volume State: 91 Volume Flags: 0x0000 Volume Version: 3.1 Sector Size: 512 Cluster Size: 4096 Index Block Size: 4096 Volume Size in Clusters: 123614207 MFT Information MFT Record Size: 1024 MFT Zone Multiplier: 0 MFT Data Position: 24 MFT Zone Start: 0 MFT Zone End: 15451779 MFT Zone Position: 4 Current Position in First Data Zone: 15451779 Current Position in Second Data Zone: 0 Allocated clusters 225436 (0.2%) LCN of Data Attribute for FILE_MFT: 4 FILE_MFTMirr Size: 4 LCN of Data Attribute for File_MFTMirr: 61807103 Size of Attribute Definition Table: 2560 Number of Attached Extent Inodes: 0 FILE_Bitmap Information FILE_Bitmap MFT Record Number: 6 State of FILE_Bitmap Inode: 80 Length of Attribute List: 0 Number of Attached Extent Inodes: 0 FILE_Bitmap Data Attribute Information Decompressed Runlist: not done yet Base Inode: 6 Attribute Types: not done yet Attribute Name Length: 0 Attribute State: 3 Attribute Allocated Size: 15454208 Attribute Data Size: 15451776 Attribute Initialized Size: 15451776 Attribute Compressed Size: 0 Compression Block Size: 0 Compression Block Size Bits: 0 Compression Block Clusters: 0 Free Clusters: 41101636 (33.2%) ===