Bug 1120964
| Summary: | Windows NTFS volume corrupted beyond repair during installation | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Chris Murphy <bugzilla> | ||||||||||||||||||
| Component: | python-blivet | Assignee: | David Lehman <dlehman> | ||||||||||||||||||
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||||||||||||||
| Severity: | unspecified | Docs Contact: | |||||||||||||||||||
| Priority: | unspecified | ||||||||||||||||||||
| Version: | 21 | CC: | amulhern, anaconda-maint-list, awilliam, bcl, bugzilla, dlehman, lonelywoolf, robatino, samuel-rhbugs, vpodzime | ||||||||||||||||||
| Target Milestone: | --- | Keywords: | CommonBugs, Reopened | ||||||||||||||||||
| Target Release: | --- | ||||||||||||||||||||
| Hardware: | Unspecified | ||||||||||||||||||||
| OS: | Unspecified | ||||||||||||||||||||
| Whiteboard: | https://fedoraproject.org/wiki/Common_F21_bugs#ntfs-resize-disabled | ||||||||||||||||||||
| Fixed In Version: | python-blivet-0.61.10-1.fc21 | Doc Type: | Bug Fix | ||||||||||||||||||
| Doc Text: | Story Points: | --- | |||||||||||||||||||
| Clone Of: | Environment: | ||||||||||||||||||||
| Last Closed: | 2014-11-15 09:16:02 UTC | Type: | Bug | ||||||||||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||||||||||
| Documentation: | --- | CRM: | |||||||||||||||||||
| Verified Versions: | Category: | --- | |||||||||||||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||||||||||
| Embargoed: | |||||||||||||||||||||
| Attachments: |
|
||||||||||||||||||||
|
Description
Chris Murphy
2014-07-18 04:22:45 UTC
I see that when we align the end sector for a partition resize we always align down. This may be a mistake. In this case, we align the end sector down and that causes it to end up smaller than the filesystem.
21:18:35,978 INFO program: Running... ntfsresize -ff -s 138501M /dev/sda2
>>> end_sector = 270714879
>>> start_sector = 206848
>>> sector_size = 512
>>> length = end_sector - start_sector + 1
>>> mb = 1000.0 ** 2
>>> size_in_mb = (length * sector_size) / mb
>>> size_in_mb
138500.112384
So we did indeed corrupt your ntfs. Sorry about that.
I'm not sure there is going to be a simple solution to this, unfortunately. It involves some back and forth between device size (alignment), filesystem size (min size), and the rest of the system (new free space on disk).
The Windows VM is a throw away... Ick. Looks like ntfsresize uses MB not MiB, so it doesn't even sector align. The ntfsresize man page suggests under -f option it's possible to do multiple resizes without using chkdsk in between, so what about a shrink, followed by a rounded up and aligned partition change, then an ntfsresize -ff -x to expand it into that final space? Nominating as a beta blocker even though the applicable release criteria is for final. I think a beta release causing this kind of corruption isn't acceptable when users are actively encouraged to test it by virtue of it being offered on the home page for broad usage. "All known bugs that can cause corruption of user data must be fixed or documented at Common F21 bugs." Another rationalization for beta blocking from the beta release criteria page: A bug in a Critical Path package that: -Cannot be fixed with a future stable update -Has a severity rating of high or greater and no reasonable workaround Bug hinders execution of required Beta test plans or dramatically reduces test coverage All of these are true: anaconda is crit path package, once released in beta it can't be fixed for beta, severity is either high or urgent (not everyone will trigger this bug), we'd expect much reduced testing for Windows NTFS resize based installs since users should avoid doing them. This bug is not always triggered, but it looks like it's about a 50/50 shot whether the installer over or under estimates the partition size relative to the volume size. I think I have a pretty good handle on this one. Chris, I can get you an updates image sometime tomorrow if you are in a position to test. I can early next week. Let me know what version of blivet you're going to test against. Discussed at 2014-10-03 blocker review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2014-10-03/f21-blocker-review.2014-10-03-15.58.log.txt . Accepted as a Beta blocker per the justification in c#4 (been a while since we used the 'backstop' blocker definition, but it seems justified here). Discussed at 2014-10-08 blocker review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2014-10-08/f21-blocker-review.2014-10-08-15.58.log.txt . dlehman: could you post an updates.img against the blivet that's in TC2, that is, 0.61.3-1.fc21 ? TC2 should be reliably available for cmurf to test against when he appears. Updates image available for testing against blivet-0.61.3-1: https://dlehman.fedorapeople.org/updates/updates-1120964.0.img Expanded (and, hopefully, improved) udpates image: https://dlehman.fedorapeople.org/updates/updates-1120964.1.img I'll be using python-blivet-0.61.4-1 When I use the c11 updates image against python-blivet-0.61.4-1 I get a crash during resizing that points to parent bug 1021507. Maybe expected since the image isn't meant for this version of blivet. dlehman: does c#13 tell you anything useful, or do we need Chris to try with a new updates.img or the current one against an older blivet, or something? Thanks! This along with the udev issue in fedup is one of only two outstanding Beta blockers at this point (though we need to do more testing). Try this: https://dlehman.fedorapeople.org/updates/updates-1120964.2.img If you hit a traceback, attach the anaconda-tb-* file to this bug. Chris, if you could test this ASAP that'd be great - we're up against go/no-go as always, it's this week. (In reply to David Lehman from comment #15) > https://dlehman.fedorapeople.org/updates/updates-1120964.2.img Doesn't crash, but doesn't fix the problem either. When Windows 8.1 is started, it finds problems, attempts repairs, and is unsuccessful. Updates image applied to Fedora-Live-Workstation-x86_64-21_Beta-TC4.iso, python-blivet-0.61.5-1.fc21, will attach program and storage logs. Created attachment 948768 [details]
storage.log c17
Created attachment 948769 [details]
program.log c17
From program log: 02:02:51,479 INFO program: New volume size : 54416998912 bytes (54417 MB) From parted u s p: 2 718848s 107001855s 106283008s primary ntfs 106283008*512=54416900096 54416900096 bytes in partition 54416998912 bytes in volume -98816 shortfall in partition size, so the end of the fs was stomped on again. I don't know if it helps or is more complicated, but I really see no point in letting the user specify fractions of MB for resizes. If it were in whole MB, maybe this whole problem goes away? For the record, the basis on which this is accepted as a blocker is not that resize must work. It is a beta blocker under the fairly rarely-used 'backup criteria':
"A bug in a Critical Path package that:
Cannot be fixed with a future stable update
Has a severity rating of high or greater and no reasonable workaround (see definition of severity and priority) "
Therefore, we don't actually need to fix NTFS resizing to unblock Beta. Other viable options are to disable the functionality or simply fail in such a way that the volume is not damaged, which would make the bug lower than 'high' severity.
There is no criterion requiring resize functionality to work or even exist at Beta (or, in fact, at all) - the only resize criterion per se is for Final and states only that any resize mechanisms present must attempt resize operations sanely.
New image: https://dlehman.fedorapeople.org/updates/updates-1120964.3.img This one has several changes, the most important being: 1. account for partition alignment in partition min/max sizes 2. ensure resize targets are valid in terms of alignment 3. specify resize target to ntfsresize in bytes (not MB) The image is for use with Beta_TC4. Please provide logs if there are issues. If this image works, I'll want to try one with only the essential changes and make sure it still fixes the immediate problem. (In reply to David Lehman from comment #22) > New image: > https://dlehman.fedorapeople.org/updates/updates-1120964.3.img Still breaks the fs, will attach logs. Created attachment 949493 [details]
storage.log c23
Created attachment 949494 [details]
program.log
Created attachment 949495 [details]
program.log c23 (new)
Created attachment 949496 [details]
storage.log c23 (new)
Reproduced again with updates-1120964.3.img parted /dev/sda u s p 2 718848s 107323391s 106604544s primary ntfs So the partition is 54581526528 bytes. program.log 16:54:13,261 INFO program: New volume size : 54581998080 bytes (54582 MB) -471552 Volume is larger than partition, so the end of the volume is being stepped on still. Will attached these logs also. Created attachment 949498 [details]
program.log c27
Created attachment 949499 [details]
storage.log c27
updates-1120964.4.img appears to fix this, continuing to test anaconda-21.48.12-1.fc21, python-blivet-0.61.7-1.fc21 has been submitted as an update for Fedora 21. https://admin.fedoraproject.org/updates/python-blivet-0.61.7-1.fc21,anaconda-21.48.12-1.fc21 Package anaconda-21.48.12-1.fc21, python-blivet-0.61.7-1.fc21: * should fix your issue, * was pushed to the Fedora 21 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing anaconda-21.48.12-1.fc21 python-blivet-0.61.7-1.fc21' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2014-13542/python-blivet-0.61.7-1.fc21,anaconda-21.48.12-1.fc21 then log in and leave karma (feedback). CommonBugs suggested text: NTFS resizing support is disabled in both Automatic and Manual partitioning paths for Fedora 21 Beta. Dual boot installations are still possible by first resizing the Windows volume from within Windows, and then using the Fedora installer to install into free space. Tested with Fedora-Live-Workstation-x86_64-21_Beta-1.iso which includes anaconda-21.48.12-1.fc21 python-blivet-0.61.7-1.fc21; against Windows 8.1 Enterprise (BIOS+vbox) default installation. We do have the option now, since we've slipped, to consider taking the 'real' fix, if David thinks there's enough time to review and test it. To give a time frame, I'd very much want to get the 'expected final' compose done by Tuesday at the latest. If that's not enough time, let's stick with the disablement. Confirmed that RC1 reports ntfs as 'Not resizable', so for blocker purposes this is VERIFIED. (We probably should keep the bug open after the update goes stable, though, as the thing in RC1 is a short term bodge/workaround, not a fix for the bug). anaconda-21.48.13-1.fc21, python-blivet-0.61.8-1.fc21 has been submitted as an update for Fedora 21. https://admin.fedoraproject.org/updates/python-blivet-0.61.8-1.fc21,anaconda-21.48.13-1.fc21 anaconda-21.48.12-1.fc21, python-blivet-0.61.7-1.fc21 has been pushed to the Fedora 21 stable repository. If problems still persist, please make note of it in this bug report. Re-opening as noted in c#36, as what we did in .12 was not to fix this bug but to disable NTFS resizing as a workaround for it. I'll check that NTFS resizing is disabled in RC2 and drop the blocker status of the bug if it is. I confirmed that Beta RC4 does not allow NTFS resizing, which is sufficient to clear the blocker status of this bug. You cannot resize NTFS partitions through guided or custom partitioning in Beta RC4. (for the record, we plan to fix this bug properly and re-allow NTFS resizing for Final, the disabling is a temporary measure for Beta.) anaconda-21.48.13-1.fc21, python-blivet-0.61.8-1.fc21 has been pushed to the Fedora 21 stable repository. If problems still persist, please make note of it in this bug report. Setting back to assigned, as the update did not fix this, only work around it by disabling NTFS resize. anaconda-21.48.14-1.fc21 has been submitted as an update for Fedora 21. https://admin.fedoraproject.org/updates/anaconda-21.48.14-1.fc21 Package anaconda-21.48.14-1.fc21, python-blivet-0.61.9-1.fc21: * should fix your issue, * was pushed to the Fedora 21 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing anaconda-21.48.14-1.fc21 python-blivet-0.61.9-1.fc21' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2014-14928/python-blivet-0.61.9-1.fc21,anaconda-21.48.14-1.fc21 then log in and leave karma (feedback). anaconda-21.48.14-1.fc21, python-blivet-0.61.9-1.fc21 has been pushed to the Fedora 21 stable repository. If problems still persist, please make note of it in this bug report. anaconda-21.48.15-1.fc21 has been submitted as an update for Fedora 21. https://admin.fedoraproject.org/updates/anaconda-21.48.15-1.fc21 python-blivet-0.61.10-1.fc21, anaconda-21.48.16-1.fc21 has been pushed to the Fedora 21 stable repository. If problems still persist, please make note of it in this bug report. |