Bug 2406225 - System unusable after upgrade from F42 to F43
Summary: System unusable after upgrade from F42 to F43
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: authselect
Version: 43
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Pavel Březina
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: https://discussion.fedoraproject.org/...
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2025-10-24 16:35 UTC by Chris Irwin
Modified: 2025-11-10 12:02 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2025-11-10 12:02:07 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Pre-authselect nsswitch.conf (1.62 KB, text/plain)
2025-10-24 16:40 UTC, Chris Irwin
no flags Details
Pre-authselect pam.d/postlogin (300 bytes, text/plain)
2025-10-24 16:41 UTC, Chris Irwin
no flags Details
Pre-authselect pam.d/smartcard-auth (646 bytes, text/plain)
2025-10-24 16:41 UTC, Chris Irwin
no flags Details
Pre-authselect pam.d/fingerprint-auth (617 bytes, text/plain)
2025-10-24 16:41 UTC, Chris Irwin
no flags Details
Pre-authselect pam.d/password-auth (974 bytes, text/plain)
2025-10-24 16:42 UTC, Chris Irwin
no flags Details
Pre-authselect pam.d/system-auth (974 bytes, text/plain)
2025-10-24 16:42 UTC, Chris Irwin
no flags Details
All journalctl output of a reproduced example (2.05 MB, text/plain)
2025-10-24 19:35 UTC, Chris Irwin
no flags Details

Description Chris Irwin 2025-10-24 16:35:31 UTC
Description of problem:

System entirely unusable after upgrade from Fedora 42 to Fedora 43. GDM crashes repeatedly. systemd attempts to restart it so attempting to use a console for recovery does not work. System entirely unusable.

There are errors in dmesg about failures that appear to be regarding the new dynamic user assignments for gdm sessions.

Root cause seems be this system predates authselect, and authselect provides configuration required for GDM 49 to function. No authselect config was applied at any upgrades over the last few years.

gdm crashing renders the text-mode consoles unusable as it attempts to restart.

Version-Release number of selected component (if applicable):

gdm.x86_64 1:49.0.1-7.fc43
authselect.x86_64 1.6.2-1.fc43


Reproducible: Always

Steps to Reproduce:

1. Upgrade using `dnf system-upgrade`, starting from Fedora 25 or 26 in 2017, through to Fedora 43. Before authselect was used.

2. Reboot system after finishing upgrade to F43
Actual Results:
No graphics displayed, no consoles working

Expected Results:
GDM works, CTRL-ALT-F3 provides a working console if required

Additional Information:
This system was installed with Fedora 25 or 26 in 2017. It has been promptly upgraded to each new version (sometimes in beta).

Actual steps to resolve:

* authselect select local --force
* touch /.autorelabel

I had to perform these steps from a chroot on a livecd as the system was not usable after upgrade. I suspect if I did the authselect before upgrading, I would have no issues.

There were selinux issues after the authselect (likely due to the chroot stuff), which was resolved by the auto-relabel

Note that I have the updated version of authselect noted in #2397255, but authselect-apply-changes.service is disabled (preset: enabled, but it didn't actually get enabled in real life). I'm unsure if the `authselect apply-changes --upgrade` would be sufficient due needing to use `authselect local --force` to override pre-existing configuration first.

Comment 1 Chris Irwin 2025-10-24 16:40:20 UTC
Created attachment 2110724 [details]
Pre-authselect nsswitch.conf

Comment 2 Chris Irwin 2025-10-24 16:41:07 UTC
Created attachment 2110725 [details]
Pre-authselect pam.d/postlogin

Comment 3 Chris Irwin 2025-10-24 16:41:25 UTC
Created attachment 2110726 [details]
Pre-authselect pam.d/smartcard-auth

Comment 4 Chris Irwin 2025-10-24 16:41:48 UTC
Created attachment 2110727 [details]
Pre-authselect pam.d/fingerprint-auth

Comment 5 Chris Irwin 2025-10-24 16:42:07 UTC
Created attachment 2110728 [details]
Pre-authselect pam.d/password-auth

Comment 6 Chris Irwin 2025-10-24 16:42:25 UTC
Created attachment 2110729 [details]
Pre-authselect pam.d/system-auth

Comment 7 Chris Irwin 2025-10-24 16:43:18 UTC
I've attached the files that were replaced by authselect, from the backup it made in /var/lib/authselect/backups/2025-10-24-08-35-33.Y3Pc5E

Comment 8 Chris Irwin 2025-10-24 19:35:32 UTC
Created attachment 2110735 [details]
All journalctl output of a reproduced example

If you wish to reproduce the issue, you can use the following steps to place the pre-authselect configuration onto a current F43 system to replicate the post-upgrade experience for older systems:

1. Install Fedora 43 beta in a VM
2. Upgrade Fedora 43 beta to current
3. Copy my pre-authselect files from this ticket into /etc/authselect

   You could remove the symlinks and place the files in /etc/ and /etc/pam.d/ as appropriate instead. It wasn't necessary to demonstrate the error.

4. Reboot

Once the system comes up, you have a blank screen with a _ cursor in the top left. You can not activate a console to work on the issue.

I have attached the journalctl output for that entire boot.

Comment 9 Pavel Březina 2025-10-30 11:18:04 UTC
Hi,
the underlying cause is likely https://fedoraproject.org/wiki/Changes/Migrate_to_lastlog2 that replaces pam_lastlog with pam_lastlog2. 

(In reply to Chris Irwin from comment #7)
> I've attached the files that were replaced by authselect, from the backup it
> made in /var/lib/authselect/backups/2025-10-24-08-35-33.Y3Pc5E

I am confused here. I assume that this update was created when upgrading to F43, not before, right? So you had non-authselect configuration on F42? Is there any chance, you can attach the configuration that was *NOT* working?

Comment 10 Pavel Březina 2025-10-30 11:29:44 UTC


(In reply to Chris Irwin from comment #8)
> Created attachment 2110735 [details]
> All journalctl output of a reproduced example
> 
> If you wish to reproduce the issue, you can use the following steps to place
> the pre-authselect configuration onto a current F43 system to replicate the
> post-upgrade experience for older systems:
> 
> 1. Install Fedora 43 beta in a VM
> 2. Upgrade Fedora 43 beta to current
> 3. Copy my pre-authselect files from this ticket into /etc/authselect
> 
>    You could remove the symlinks and place the files in /etc/ and
> /etc/pam.d/ as appropriate instead. It wasn't necessary to demonstrate the
> error.
> 
> 4. Reboot

This actually means that you will boot up with the pre-authselect configuration that still has pam_lastlog.

> 
> Once the system comes up, you have a blank screen with a _ cursor in the top
> left. You can not activate a console to work on the issue.
> 
> I have attached the journalctl output for that entire boot.

I think the backup you have was created by calling `authselect select local --force` manually from the livecd, not by any automation from the scriptlet since I already removed forceful override of non-authselect configuration.

When you updated to F36, your non-authselect configuration should have been updated to authselect configuration to prevent this kind of issues. Maybe this did not work, or maybe you reverted it later.

I'm not sure why authselect-apply-changes.service is disabled, unless you manually changed it it should use the preset. I will update to F43 soon and will check. But this service is only important for Silverblue and rpm-ostree systems. It does not need to be enabled for Workstation.

Comment 11 Chris Irwin 2025-10-30 18:17:37 UTC
> I am confused here. I assume that this update was created when upgrading to F43, not before, right? So you had non-authselect configuration on F42? Is there any chance, you can attach the configuration that was *NOT* working?

When I manually ran authselect, it placed the old files (that it replaced) in a backup directory. The files I attached are the non-authselect files that were used by F42, I just retrieved them from that backup. I could retrieve them from a F42 btrfs snapshot, but they're the same files.

> This actually means that you will boot up with the pre-authselect configuration that still has pam_lastlog.

Correct. I was just trying to provide a steps to reproduce the state I was in, in case it helped for troubleshooting (I'm not a pam expert)

> When you updated to F36, your non-authselect configuration should have been updated to authselect configuration to prevent this kind of issues. Maybe this did not work, or maybe you reverted it later.

I think I removed fingerprint from sudo via pam at somepoint in the past. However, I've never messed with authselect previously, either enabling/disabling/bypassing. I'd never hard of authselect before this.

> I'm not sure why authselect-apply-changes.service is disabled, unless you manually changed it it should use the preset. I will update to F43 soon and will check. But this service is only important for Silverblue and rpm-ostree systems. It does not need to be enabled for Workstation.

I have not made any changes to the authselect services. Glad it isn't needed for Workstation, but not sure why it was disabled.

Comment 12 Pavel Březina 2025-11-04 12:25:11 UTC
Just to clarify, are you on Workstation or Silverblue?

Comment 13 Chris Irwin 2025-11-04 15:13:28 UTC
Workstation. I don't think Silverblue was available when I installed originally.

Comment 14 Petr Sklenar 2025-11-07 14:38:21 UTC
Documented as common issue: https://discussion.fedoraproject.org/t/171961

Comment 15 Pavel Březina 2025-11-10 12:02:07 UTC
Closing as won't fix as this is really not an authselect problem and it was resolved by documenting it as an common issue: https://discussion.fedoraproject.org/t/171961

Using authselect will prevent this kind of issues in the future.


Note You need to log in before you can comment on or make changes to this bug.