Bug 1114786

Created attachment 913607 [details]
File: anaconda-tb

Created attachment 913608 [details]
File: anaconda.log

Created attachment 913609 [details]
File: environ

Created attachment 913610 [details]
File: journalctl

Created attachment 913611 [details]
File: lsblk_output

Created attachment 913612 [details]
File: nmcli_dev_list

Created attachment 913613 [details]
File: os_info

Created attachment 913614 [details]
File: program.log

Created attachment 913615 [details]
File: storage.log

Created attachment 913616 [details]
File: ifcfg.log

Still get this when reformatting an existing active swap partition, which is the only way I can see how to reuse an existing swap partition in the UI. I'd rather be allowed to just add the current swap leaving it active, rather than reformat it.

anaconda-21.48.4-1.fc21.i686
python-blivet-0.62-1.fc21.noarch

Proposed as a Blocker for 21-beta by Fedora user chrismurphy using the blocker tracking app because:

 When using the custom partitioning flow, the installer must be able to: Assign mount points to existing storage volumes.

Strictly speaking swap isn't a mount point, but we can take our pick. I should be able to either reuse an existing swap partition, or reformat it, and in any case not get a crash.

I'm not sure blivet could fix this. Existing swap in use can hardly be reformatted or deactivated by blivet. And using a swap device without reformatting for a new system can be quite dangerous as it can be being used by the other system at that point (hibernated system or anything else). Maybe anaconda could turn off those swap devices when starting? Or maybe we should disable automatic swap activation in the live system? The latter seems to me as the best option. Brian, what is your opinion on this?

The test system uses EFI/GPT, and swap partition has the linux swap partition type GUID set, therefore systemd auto-activates it even though it's not in the live install media fstab. This is probably not common yet, the common case is swap on LVM LV, which is not auto-activated. 

I'm not super familiar with the systemd autoactivate swap code, but it does seem to have a check whether there's a hibernation image in place on swap so I don't think you have to worry about this. In which case the right automatic thing to do is keep it active, and do nothing. It doesn't even need to be added to fstab.

So it's actually an interesting UI/UX issue. If this active swap is left alone, it will be used by the newly installed system via systemd autoactivation. So which is better for Manual Partitioning UI when the user clicks the blue link to create things automatically? a.) Don't include swap partitions with GPT partition type GUID, even though it is being used and will be used in the new install; b.) "fib" by showing it listed as part of the current installation but don't put it in fstab; c.) show it listed as part of the installation but do include it in fstab; d.) create a new swap partition.

I'd say d is not great. It wastes space. I'd opt for b or c, more likely b. It's more truth than fib, even though an fstab entry isn't being created.

First off, we shouldn't traceback so we should figure out a clean way to deal with it.

I don't think swap should be auto-activated by booting a live image -- we shouldn't be touching the user's disks until they tell us to. The live script should be disabling automounting, and if this also happens with non-live we should do the same in lorax.

(In reply to bcl from comment #15)
> First off, we shouldn't traceback so we should figure out a clean way to
> deal with it.
Anaconda shouldn't allow reformatting a device that is in use, resassigning back to anaconda.

> 
> I don't think swap should be auto-activated by booting a live image -- we
> shouldn't be touching the user's disks until they tell us to. The live
> script should be disabling automounting, and if this also happens with
> non-live we should do the same in lorax.
It happens only in live installations. What is the right component to clone this bug to?

(In reply to Vratislav Podzimek from comment #16)
> (In reply to bcl from comment #15)
> > First off, we shouldn't traceback so we should figure out a clean way to
> > deal with it.
> Anaconda shouldn't allow reformatting a device that is in use, resassigning
> back to anaconda.

But python-blivet really shouldn't traceback either.

> > I don't think swap should be auto-activated by booting a live image -- we
> > shouldn't be touching the user's disks until they tell us to. The live
> > script should be disabling automounting, and if this also happens with
> > non-live we should do the same in lorax.
> It happens only in live installations. What is the right component to clone
> this bug to?

spin-kickstarts

*** Bug 1131754 has been marked as a duplicate of this bug. ***

Live images actually *explicitly* mount swaps from the host system. It's in the 'livesys' init script.


swaps=\`blkid -t TYPE=swap -o device\`
if ! strstr "\`cat /proc/cmdline\`" noswap && [ -n "\$swaps" ] ; then
  for s in \$swaps ; do
    action "Enabling swap partition \$s" swapon \$s
  done
fi

i.e. this is entirely intended behaviour, and that's been there for years, and you're supposed to be able to stop it if you don't want it by passing 'noswap'.

I think there may be a misdiagnosis here - it seems like you may be thinking that it's systemd that's activating the swap partitions, and this is a new feature so live images have only recently started activating swaps from the disks present on the system, and this is causing anaconda problems.

I don't think that's correct. Whether systemd would do it anyway or not, that stuff's been in spin-kickstarts for donkey's years, so presumably anaconda has been able to cope with it up till now, and it's anaconda's behaviour that's changed. But that shouldn't be too hard to confirm by testing live installs with a few recent versions - 18, 19, 20 maybe - and checking whether the host swap is activated during bot, and whether anaconda copes with that somehow.

Per https://wiki.archlinux.org/index.php/swap#Activation_by_systemd , 'man systemd-gpt-auto-generator' and 'man systemd.swap' I don't think systemd's swap auto-activation would actually activate swaps from a host system disk when booting live, but again this should be reasonably easy to test - boot with 'noswap' and see if the swap gets activated. If not, it's fairly clear that it's the code in livesys that's activating it, not systemd (I don't think systemd parses 'noswap').

Adam is correct twice:

Without noswap:

[   13.042274] localhost livesys[726]: Enabling swap partition /dev/sda1 [  OK  ]
[   13.036309] localhost kernel: Adding 4194300k swap on /dev/sda1.  Priority:-1 extents:1 across:4194300k FS

With noswap this doesn't appear, nor does systemd-gpt-auto-generator automatically activate it. 'free' reports swap is not in use.

The easiest solution here is to filter out devices in use from anaconda. However, all "uninformed users" not knowing about the 'noswap' boot option will miss their swap partition in the partitioning GUI then.

Hiding the device is probably not an option. The device being in use could cause problems writing partitioning changes to the disk it is on. See blivet commit
6c6b2d4ea7189f2.

I don't know what blivet can do here except raise an exception. I think anaconda probably needs to catch that exception and handle it in a way appropriate to the UI.

Also, this:

22:27:13,599 INFO program: Running... swapoff /dev/sda6
22:27:13,611 DEBUG program: Return code: 0


Blivet deactivates all devices it found after it finishes detecting storage, to prevent weird bugs related to active devices. (Exactly like this bug.)

Ahha, and then systemd turns it back on.

Jun 30 22:27:54 rawhide.localdomain systemd[1]: Activating swap /dev/sda6...
Jun 30 22:27:54 rawhide.localdomain systemd[1]: Activating swap /dev/sda6...
Jun 30 22:27:54 rawhide.localdomain systemd[1]: Activating swap /dev/sda6...
Jun 30 22:27:54 rawhide.localdomain systemd[1]: Activating swap /dev/sda6...
Jun 30 22:27:54 rawhide.localdomain systemd[1]: Activating swap /dev/sda6...
Jun 30 22:27:54 rawhide.localdomain systemd[1]: Starting Getty on tty2...
Jun 30 22:27:54 rawhide.localdomain systemd[1]: Started Getty on tty2.
Jun 30 22:27:54 rawhide.localdomain kernel: Adding 511996k swap on /dev/sda6.  Priority:-1 extents:1 across:511996k SSFS
Jun 30 22:27:54 rawhide.localdomain systemd[1]: Activated swap /dev/disk/by-uuid/13b3debc-e6b8-4c3c-a650-f26cecfa04f5.
Jun 30 22:27:54 rawhide.localdomain systemd[1]: Activated swap /dev/disk/by-partuuid/dcc065fd-0b00-4b36-8668-d4b32c9f47be.
Jun 30 22:27:54 rawhide.localdomain systemd[1]: Activated swap /dev/disk/by-id/wwn-0x5002538043584d30-part6.
Jun 30 22:27:54 rawhide.localdomain systemd[1]: Activated swap /dev/disk/by-id/ata-SAMSUNG_SSD_830_Series_S0Z4NEAC933856-part6.
Jun 30 22:27:54 rawhide.localdomain systemd[1]: Activated swap /dev/sda6.

(In reply to bcl from comment #25)
Like I mentioned in c21, once noswap was passed, swap wasn't activated, even by systemd. man systemd-gpt-auto-generator says "It will only look for the other partitions on the same physical disk the root file system is located on." So what you've found would seem to be a bug, but it's not one I can reproduce with Fedora-Live-Workstation-x86_64-21_Alpha-TC7.iso I'll try RC1 in just a bit.

Boot Fedora-Live-Workstation-x86_64-21_Alpha-1.iso on BIOS+GPT and UEFI+GPT  using noswap boot param. i.e. I'm booting from /dev/sr0; and /dev/sda1 is a preexisting swap using the linux swap partitiontype GUID. The swap is not activated by systemd.

Hum, so I suppose what might be happening is that when livesys activates the swap, systemd starts tracking it, and then when blivet turns it off, systemd turns it back on again? that behaviour might conceivably have changed recently, we could look into that.

Discussed in 2014-09-24 Blocker Review meeting [1]. We decided to postpone the decision on this one. It looks worrying, but we need more details on what circumstances trigger the bug. Adam will look deeper into that. More debugging from anyone involved would be very helpful, of course.

[1] http://meetbot.fedoraproject.org/fedora-blocker-review/2014-09-24/

*** Bug 1146221 has been marked as a duplicate of this bug. ***

Re comment #28: systemd automatically adds a dependency of RequiredBy=swap.target to all swap units, even the ones created dynamically based on /proc/swaps. My guess is (based on comment #28) that when starting getty, systemd notices that it depends on basic.target, which depends on swap.target, which depends on the dynamically created .swap unit, which is inactive after the swapoff done by blivet, and starts the .swap unit to satisfy dependencies. The solution is probably to not make such dynamically created swap units RequiredBy=swap.target. I'll look into this from the systemd side if nobody beats me to it.

zbigniew: that sounds pretty plausible to me. It'd be good if something can be done in systemd - even though you can argue the behaviour of the livecd service in going out and turning on swaps like this is somewhat questionable, it's been doing it for years, and it's the kind of thing someone would probably complain about if we turned it off...I can see how it could be useful on low-RAM systems.

Discussed at 2014-10-01 blocker review meeting - http://meetbot.fedoraproject.org/fedora-blocker-review/2014-10-01/f21-blocker-review.2014-10-01-15.58.log.txt . As we understand it, and as some ad hoc testing indicated, in order to hit this, you need to:

* Do a UEFI/GPT install with swap *not* part of an LV
* Do a live install of F21 on top of that previous install

which just seems like slightly too unusual of a case to block on, especially given there is a simple workaround: boot with 'noswap'. We might consider this as a Final blocker if it is not fixed by then. (I tested that the bug doesn't seem to happen if swap is part of an LV in a UEFI/GPT install, or if swap is a regular partition but it's not a GPT disk).

Of course, just because it's not a Beta *blocker* doesn't mean it shouldn't be fixed, we'd certainly like to see a fix.

Please remember, if re-proposing as a Final blocker, to clear 'RejectedBlocker' from the whiteboard.

(In reply to Adam Williamson (Red Hat) from comment #33)

Ah, too bad. I have a fix ready and I'll push it to systemd git master, and then it'd be good to check if it fixes this problem.

> which just seems like slightly too unusual of a case to block on,

It's certainly not the most common scenario, but I don't see why it's unusual. If you have any previous Linux distribution installed before installing Fedora, you almost surely have a swap partition not part of a LV. And if you have a computer that's less than three years old, you likely also have a GPT partition table.

But I agree the workaround is sufficient for a beta release.

Proposed as a Blocker for 21-final by Fedora user catanzaro using the blocker tracking app because:

 "When using the dedicated installer images, the installer must be able to complete an installation using the text, graphical and VNC installation interfaces."

Chris's suggested custom partitioning criterion seems less appropriate to me, since the crash doesn't require you to enter custom partitioning.

Going through the discussion above I got lost a bit about what we have now and what's still needed to be done. Is there anything we should change in anaconda to fix this issue?

systemd side of the story should be fixed (swaps enabled externally do not become part of swap.target and systemd does forgets about them when they are deactivated). Please reopen if this fix does not suffice.

initscripts-9.56.1-1.fc21,systemd-216-2.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/initscripts-9.56.1-1.fc21,systemd-216-2.fc21

Re-opening for 21 just so we can make sure we don't forget about this.

Another user experienced a similar problem:

I have a broken Ubuntu installation with "/" on /dev/sda2 and swap on /dev/sda3.

I selected /dev/sda2 as root for the F21 installation (reformat ext4) and then clicked /dev/sda3 to choose it for swap (reformat). After clicking "Refresh" the installer crashed.

cmdline:        /usr/bin/python  /sbin/anaconda --liveinst --method=livecd:///dev/mapper/live-base
cmdline_file:   BOOT_IMAGE=/isolinux/vmlinuz0 root=live:LABEL=Fedora-Live-Workstation-x86_64-2 ro rd.live.image quiet rhgb nomodeset
hashmarkername: anaconda
kernel:         3.16.1-301.fc21.x86_64
other involved packages: python-blivet-0.62-1.fc21.noarch, anaconda-gui-21.48.6-1.fc21.x86_64
package:        anaconda-core-21.48.6-1.fc21.x86_64
packaging.log:  
product:        Fedora
reason:         DeviceError: ('cannot replace active format', 'sda3')
release:        Fedora release 21 (Twenty One)
version:        21

(In reply to Rolf Offermanns from comment #41)
> kernel:         3.16.1-301.fc21.x86_64
The systemd update for F21 was created this morning, so you are certainly using the previous version, so this is not unexpected.

Another user experienced a similar problem:

After selecting (existing) partitions for boot root and swap to format and an existing home for mount only

cmdline:        /usr/bin/python  /sbin/anaconda --liveinst --method=livecd:///dev/mapper/live-base
cmdline_file:   BOOT_IMAGE=/isolinux/vmlinuz0 root=live:LABEL=Fedora-Live-Workstation-x86_64-2 ro rd.live.image quiet rhgb
hashmarkername: anaconda
kernel:         3.16.1-301.fc21.x86_64
other involved packages: python-blivet-0.62-1.fc21.noarch, anaconda-gui-21.48.6-1.fc21.x86_64
package:        anaconda-core-21.48.6-1.fc21.x86_64
packaging.log:  
product:        Fedora
reason:         DeviceError: ('cannot replace active format', 'sda4')
release:        Fedora release 21 (Twenty One)
version:        21

Package initscripts-9.56.1-1.fc21, systemd-216-3.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing initscripts-9.56.1-1.fc21 systemd-216-3.fc21'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-12688/initscripts-9.56.1-1.fc21,systemd-216-3.fc21
then log in and leave karma (feedback).

*** Bug 1151763 has been marked as a duplicate of this bug. ***

*** Bug 1147115 has been marked as a duplicate of this bug. ***

*** Bug 1133358 has been marked as a duplicate of this bug. ***

*** Bug 998148 has been marked as a duplicate of this bug. ***

Package systemd-216-4.fc21, initscripts-9.56.1-2.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing systemd-216-4.fc21 initscripts-9.56.1-2.fc21'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-12688/initscripts-9.56.1-2.fc21,systemd-216-4.fc21
then log in and leave karma (feedback).

We're now frozen for Beta, so proposing as an FE so we can get the fix in.

(In reply to Adam Williamson (Red Hat) from comment #50)
> We're now frozen for Beta, so proposing as an FE so we can get the fix in.
To paraphrase my comment #37: can anybody please explain me what more is needed here to be done?

(In reply to Vratislav Podzimek from comment #51)
> (In reply to Adam Williamson (Red Hat) from comment #50)
> > We're now frozen for Beta, so proposing as an FE so we can get the fix in.
> To paraphrase my comment #37: can anybody please explain me what more is
> needed here to be done?
Just push the update.

vpodzime: I don't believe anaconda needs to do anything here, unless I've missed anything.

Can folks who usually vote on blocker/FE issues please file FE votes as comments? Thanks!

+1 FE, would be good to have this fix in Beta to make sure it gets tested properly and we don't have suprises in Final.

+1 FE

+1 FE

Marking as AcceptedFE. It'd be good if people could test and karma the update as it's quite a big one (it includes several other fixes that are probably FE-worthy on their own, but we should certainly test it fully).

Another user experienced a similar problem:

Formatted an existing swap partition.

cmdline:        /usr/bin/python  /sbin/anaconda --liveinst --method=livecd:///dev/mapper/live-base
cmdline_file:   BOOT_IMAGE=/isolinux/vmlinuz0 root=live:LABEL=Fedora-Live-Workstation-x86_64-2 ro rd.live.image quiet rhgb rd.live.check
hashmarkername: anaconda
kernel:         3.17.0-301.fc21.x86_64
other involved packages: anaconda-gui-21.48.9-1.fc21.x86_64, python-blivet-0.61.4-1.fc21.noarch
package:        anaconda-core-21.48.9-1.fc21.x86_64
packaging.log:  
product:        Fedora"
reason:         DeviceError: ('cannot replace active format', 'sda5')
release:        Fedora release 21 (Twenty One)
version:        Fedora

The fix for this should be in in TC4 for testing.

Package initscripts-9.56.1-2.fc21, systemd-216-5.fc21:
* should fix your issue,
* was pushed to the Fedora 21 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing initscripts-9.56.1-2.fc21 systemd-216-5.fc21'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2014-12688/initscripts-9.56.1-2.fc21,systemd-216-5.fc21
then log in and leave karma (feedback).

*** Bug 1155533 has been marked as a duplicate of this bug. ***

initscripts-9.56.1-2.fc21, systemd-216-5.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Another user experienced a similar problem:

I booted the nightly LiveCD (2014-10-29, Workstation, i686)
- created manual wired nethwork connection
- switched the language/input to german
- installed a network printer
- started the install-program to install the system to harddisk
- anaconda opened and in the partition step it crashed:
  - manual partition layout
  - used the existing partitions from a previous fedora (nightly) installation
  - all partitions are added for use (/boot, /, /home) and in the step to add the swap-partition I checked to format the partition and want to update the setting anacond crashed (report a bug)


cmdline:        /usr/bin/python  /sbin/anaconda --liveinst --method=livecd:///dev/mapper/live-base
cmdline_file:   BOOT_IMAGE=vmlinuz0 initrd=initrd0.img root=live:CDLABEL=Fedora-Live-Workstation-i686-21_ rootfstype=auto ro rd.live.image quiet  rhgb rd.luks=0 rd.md=0 rd.dm=0 
hashmarkername: anaconda
kernel:         3.17.1-302.fc21.i686
other involved packages: python-blivet-0.61.8-1.fc21.noarch, anaconda-gui-21.48.13-1.fc21.i686
package:        anaconda-core-21.48.13-1.fc21.i686
packaging.log:  
product:        Fedora"
reason:         DeviceError: ('cannot replace active format', 'sda5')
release:        Fedora release 21 (Twenty One)
version:        Fedora

(In reply to Rolle from comment #63)
> Another user experienced a similar problem:
> 
> I booted the nightly LiveCD (2014-10-29, Workstation, i686)
I it possible that that LiveCD is still using the old version of the package.

rolle: can you boot the live image and check the systemd / initscripts versions on it?

Another user experienced a similar problem:

Manual partition dialog. Crash after setting / partition.

addons:         com_redhat_kdump
cmdline:        /usr/bin/python  /sbin/anaconda
cmdline_file:   BOOT_IMAGE=/vmlinuz noipv6 selinux=0 biosdevname=0 net.ifnames=0 inst.sshd sshd vnc vncconnect=irina.phys.ntnu.no ip=129.241.48.123 netmask=255.255.254.0 gateway=129.241.48.1 dns=129.241.0.200 ks=http://bird.phys.ntnu.no/kickstart/eivind/21
hashmarkername: anaconda
kernel:         3.17.2-300.fc21.x86_64
package:        anaconda-21.48.13-1
product:        Fedora"
reason:         DeviceError: ('cannot replace active format', 'sda1')
release:        Cannot get release name.
version:        Fedora

Happened with install from latest F21 bits.

Partition table:

$ fdisk -l         

Disk /dev/sda: 500.1 GB, 500107862016 bytes, 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00025a5b

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048    51202047    25600000   83  Linux
/dev/sda2        51202048    61442047     5120000   82  Linux swap / Solaris
/dev/sda3        61442048    71682047     5120000   83  Linux
/dev/sda4        71682048   976773119   452545536    5  Extended
/dev/sda5        71684096   976773119   452544512   83  Linux


Tried setting /dev/sda1 as / partition in manual partition dialog -> crash.

Packages/ were current, however images etc was outdated, refresh resolved the issue, sorry about noise.

(In reply to Adam Williamson (Red Hat) from comment #65)
> rolle: can you boot the live image and check the systemd / initscripts
> versions on it?

Oh sorry. I didn't recognized that you need an info from me. I don't have this nightly LiveCD (2014-10-29, Workstation, i686) anymore. So I can't give you the right versions.
In newer iso-images the problem to crash during manual partitioning and configure the swap-partition don't occured for me. All newer attempts worked fine for me.
So for me it seems also that this problem has gone.