Bug 2005625 - gnome-keyring does not work correctly in gnome-initial-setup session (causes delay on user creation, incorrect login keyring password, failure to connect to wifi network...)
Summary: gnome-keyring does not work correctly in gnome-initial-setup session (causes ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: gnome-keyring
Version: 35
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Matthias Clasen
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedBlocker AcceptedFreezeException
: 2004565 (view as bug list)
Depends On:
Blocks: F35BetaBlocker F35BetaFreezeException F35FinalBlocker
TreeView+ depends on / blocked
 
Reported: 2021-09-18 19:03 UTC by Michael Catanzaro
Modified: 2021-09-24 21:19 UTC (History)
20 users (show)

Fixed In Version: gnome-keyring-40.0-3.fc35
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-09-24 00:31:13 UTC
Type: Bug


Attachments (Terms of Use)
video of GIS successful (1.10 MB, video/mp4)
2021-09-18 20:44 UTC, Geraldo Simião
no flags Details
entire journal content (506.42 KB, text/plain)
2021-09-21 17:48 UTC, Adam Williamson
no flags Details
third party repos screenshot (85.38 KB, image/png)
2021-09-23 09:54 UTC, Kamil Páral
no flags Details


Links
System ID Private Priority Status Summary Last Updated
GNOME Gitlab GNOME/glib - issues 2316 0 None None None 2021-09-21 21:54:49 UTC
GNOME Gitlab GNOME/glib - merge_requests 2212 0 None None None 2021-09-21 21:54:49 UTC
GNOME Gitlab GNOME/gnome-keyring - issues 77 0 None None None 2021-09-21 22:11:32 UTC
Red Hat Bugzilla 2004565 1 unspecified CLOSED The default keyring is created with an unknown password, can't be unlocked 2021-09-21 23:33:50 UTC
Red Hat Bugzilla 2006314 1 unspecified CLOSED Attempting to join wifi network in initial-setup fails 2021-09-24 21:32:13 UTC

Description Michael Catanzaro 2021-09-18 19:03:27 UTC
Description of problem: gnome-initial-setup hangs after passing


Version-Release number of selected component (if applicable): tested using Sep 18 F35 Workstation live nightly


How reproducible: not sure, only tested it once


Steps to Reproduce:
1. Install F35 Workstation live
2. Set initial user account password in gnome-initial-setup

Actual results: gnome-initial-setup hangs after I enter a password for my user account. I cannot continue.


Expected results: no hang.


Additional info: First noticed by another user at https://bugzilla.redhat.com/show_bug.cgi?id=1997310#c36, though I'm not sure if gnome-initial-setup remained unresponsive for him or not.

Comment 1 Fedora Blocker Bugs Application 2021-09-18 19:08:06 UTC
Proposed as a Blocker for 35-beta by Fedora user catanzaro using the blocker tracking app because:

 "A working mechanism to create a user account must be clearly presented during installation and/or first boot of the installed system."

If this is happening for everyone, I'd say it's an obvious blocker since it prevents successful completion of the initial setup process. If it happens more rarely, then maybe not.

Comment 2 Michael Catanzaro 2021-09-18 19:23:13 UTC
(In reply to Michael Catanzaro from comment #0)
> Description of problem: gnome-initial-setup hangs after passing

Er: it hangs after entering a password on the password page. mutter's unresponsive application dialog appears and does not disappear within a reasonable amount of time.

Comment 3 Geraldo Simião 2021-09-18 20:34:46 UTC
Allright, that's on me. I'll redo the test and see how many seconds I wait the dialog to disappear and the process to go on. :D
FYI I set up a VM using virt-manager, UEFI firmware (no secure boot), 6Gb RAM and 3 to 5 CPUs, and virtio as video.

Comment 4 Geraldo Simião 2021-09-18 20:44:39 UTC
Created attachment 1824289 [details]
video of GIS successful

The dialog stood only for a few seconds, I clicked on the "wait" button (two times) and it finished.

Comment 5 Alessio 2021-09-18 21:14:43 UTC
In my case, yes there is a delay after I enter a password and before the Start using Fedora Linux window; the delay lasts some seconds, but mutter's unresponsive application dialog doesn't appear.
(virt-manager, BIOS, 2 cpu, 4GB ram, video QXL)

Comment 6 Michael Catanzaro 2021-09-18 23:39:08 UTC
I tried again and this time it took about 30 seconds, with no unresponsive application dialog. Not the end of the world, but still pretty bad.

Comment 7 Michael Catanzaro 2021-09-18 23:44:33 UTC
Likely the same underlying cause as bug #2004565.

Comment 8 Kamil Páral 2021-09-20 11:12:03 UTC
I tested with Fedora-Workstation-Live-x86_64-35-20210919.n.0.iso. I can confirm clicking Next at the password step takes 20-25 seconds before it continues, and if you try to click on the form in the meantime, you can "Initial Setup is not responding. [Wait] [Kill]" dialog.

Comment 9 Geoffrey Marr 2021-09-20 17:36:58 UTC
Discussed during the 2021-09-20 blocker review meeting: [0]

The decision to classify this bug as both a "RejectedBlocker (Beta)", an "AcceptedBlocker (Final)" and an "AcceptedFreezeException (Beta)" was made as a conditional violation of the criterion:

"A working mechanism to create a user account must be clearly presented during installation and/or first boot of the installed system."

We don't think it's bad enough to block Beta, but it is bad enough to warrant a freeze exception

[0] https://meetbot.fedoraproject.org/fedora-blocker-review/2021-09-20/f35-blocker-review.2021-09-20-16.00.txt

Comment 10 Adam Williamson 2021-09-20 22:45:25 UTC
So I reproduced this and grabbed the logs. There are a ton of AVCs to do with watches for a zillion font paths and a few others that seem to be irrelevant. Filtering those out, I get this:

Sep 20 15:35:48 fedora usermod[1563]: change user 'test' password
Sep 20 15:35:48 fedora /usr/libexec/gdm-wayland-session[996]: dbus-daemon[996]: [session uid=983 pid=996] Activating service name='org.freedesktop.secrets' requested by ':1.64' (uid=983 pid=1381 comm="/usr/libexec/gnome-initial-setup " label="system_u:system_r:xdm_t:s0-s0:c0.c1023")
Sep 20 15:35:48 fedora gnome-keyring-daemon[1496]: The Secret Service was already initialized
Sep 20 15:35:48 fedora /usr/libexec/gdm-wayland-session[1571]: SSH_AUTH_SOCK=/run/user/983/keyring/ssh
Sep 20 15:36:13 fedora gnome-initial-s[1381]: Failed to get secret service: Error calling StartServiceByName for org.freedesktop.secrets: Timeout was reached
Sep 20 15:36:15 fedora gdm-password][1578]: gkr-pam: unable to locate daemon control file
Sep 20 15:36:15 fedora audit[1578]: USER_AUTH pid=1578 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 msg='op=PAM:authentication grantors=pam_usertype,pam_localuser,pam_unix,pam_gnome_keyring acct="test" exe="/usr/libexec/gdm-session-worker" hostname=fedora addr=? terminal=/dev/tty1 res=success'
Sep 20 15:36:15 fedora gdm-password][1578]: gkr-pam: stashed password to try later in open session
Sep 20 15:36:15 fedora audit[1578]: USER_ACCT pid=1578 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 msg='op=PAM:accounting grantors=pam_unix,pam_localuser acct="test" exe="/usr/libexec/gdm-session-worker" hostname=fedora addr=? terminal=/dev/tty1 res=success'
Sep 20 15:36:15 fedora audit[1578]: CRED_ACQ pid=1578 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_localuser,pam_unix,pam_gnome_keyring acct="test" exe="/usr/libexec/gdm-session-worker" hostname=fedora addr=? terminal=/dev/tty1 res=success'
Sep 20 15:36:15 fedora audit[1578]: USER_ROLE_CHANGE pid=1578 uid=0 auid=1000 ses=2 subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 msg='pam: default-context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 selected-context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 exe="/usr/libexec/gdm-session-worker" hostname=fedora addr=? terminal=/dev/tty2 res=success'

I also tested from the same snapshot but with enforcing=0; behaviour did not change, so SELinux is not the issue here.

Comment 11 Michael Catanzaro 2021-09-21 14:30:30 UTC
(In reply to Adam Williamson from comment #10)
> So I reproduced this and grabbed the logs. There are a ton of AVCs to do
> with watches for a zillion font paths and a few others that seem to be
> irrelevant.

Can you report new bugs for these, please? It seems lie the selinux policy for xdm_t has somehow become generally much too strict.

Comment 12 Adam Williamson 2021-09-21 15:29:57 UTC
I suspect https://bugzilla.redhat.com/show_bug.cgi?id=2006314 is yet another consequence of the same problem.

Comment 13 Adam Williamson 2021-09-21 17:10:37 UTC
There's an earlier error that might be significant:

Sep 20 15:34:40 fedora gnome-keyring-daemon[1496]: couldn't register in session: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name is not activatable

Comment 14 Adam Williamson 2021-09-21 17:48:40 UTC
Created attachment 1825028 [details]
entire journal content

Here's the full journal, just in case I missed anything else useful.

Comment 15 Adam Williamson 2021-09-21 19:03:02 UTC
So there's a long discussion about this in #fedora-desktop which boils down to 'we know what's going on and now we're debating what to do about it'. But sounds like we can come up with a quick hack for Beta at least.

Comment 16 Adam Williamson 2021-09-21 19:06:03 UTC
[benzea] this is glib refusing to use DBUS_SESSION_BUS_ADDRESS since quite recently when AT_SECURE is active

Comment 17 Adam Williamson 2021-09-21 21:54:49 UTC
sorry, there was further progress. this seems to be https://gitlab.gnome.org/GNOME/glib/-/merge_requests/2212 , basically - upstream reverted a workaround that we were still relying on. the easiest short-term fix should be just to revert the revert, and restore the workaround. I am currently testing a scratch build with that change.

Comment 18 Michael Catanzaro 2021-09-21 22:42:34 UTC
(In reply to Adam Williamson from comment #17)
> sorry, there was further progress. this seems to be
> https://gitlab.gnome.org/GNOME/glib/-/merge_requests/2212 , basically -
> upstream reverted a workaround that we were still relying on. the easiest
> short-term fix should be just to revert the revert, and restore the
> workaround. I am currently testing a scratch build with that change.

Please also test this scratch build, if you don't mind:

https://koji.fedoraproject.org/koji/taskinfo?taskID=76081174

(Without any changes in glib.)

This removes the capability to allocate extra secure memory. If you have a huge keyring, it should fall back to using normal malloc(), so users shouldn't notice any difference. Applications never lock the memory used to store the passwords after they're retrieved from D-Bus anyway, nor does D-Bus itself, so I'm not convinced the locking ever actually accomplished much good regardless.

Comment 19 Adam Williamson 2021-09-21 22:51:17 UTC
sure, I've triggered openQA on that scratch build, will grab and test the ISO when it's done. I am currently testing an install of the ISO built with my scratch build of glib.

Comment 20 Adam Williamson 2021-09-21 23:10:10 UTC
my glib scratch build does indeed avoid the issue, but will also test mcatanzaro's gnome-keyring scratch build before we do any real ones.

Comment 21 Adam Williamson 2021-09-21 23:33:51 UTC
*** Bug 2004565 has been marked as a duplicate of this bug. ***

Comment 22 Adam Williamson 2021-09-22 00:46:14 UTC
mcatanzaro's build doesn't seem to do the trick. the delay after creating a user is still there, the login keyring shows as locked. as we're short on time, I'll do a real build of glib and we'll go with that for the RC.

Comment 23 Fedora Update System 2021-09-22 01:10:15 UTC
FEDORA-2021-fa340d4bf0 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-fa340d4bf0

Comment 24 Benjamin Berg 2021-09-22 10:32:33 UTC
Not the best place to follow up. I am a bit confused by mcatanzaro's scratch build not working, not sure what the change was, but I still see:

  %attr(0755,root,root) %caps(cap_ipc_lock=ep) %{_bindir}/gnome-keyring-daemon

in the spec file. So maybe just the wrong .spec file got build?

The gnome-keyring-daemon is started in a weird way, IIRC. i.e. by the PAM module. I wonder if we could increase its ulimit somehow without affecting the rest of the session.

Some knobs we can modify are:
 1. user@.service LimitMEMLOCK= value; which should affect most of the session (but not e.g. the shell through SSH)
 2. DefaultLimitMEMLOCK= to decrease it again for most applications
 3. LimitMEMLOCK= for single units

But, they keyring is started from a PAM module and does seteuid/setegid to drop privileges.

Or, maybe a stupid idea. But could we possibly use socket-activation and write the password into a systemd managed socket to start the keyring? Then we have a systemd unit in the user session where we can play around with the options and the above knobs could be enough to do everything.

Comment 25 Michael Catanzaro 2021-09-22 13:39:28 UTC
(In reply to Benjamin Berg from comment #24)
> Not the best place to follow up. I am a bit confused by mcatanzaro's scratch
> build not working, not sure what the change was, but I still see:
> 
>   %attr(0755,root,root) %caps(cap_ipc_lock=ep)
> %{_bindir}/gnome-keyring-daemon
> 
> in the spec file. So maybe just the wrong .spec file got build?

Nah, I simply didn't notice that. I just added --without-libcap-ng to the configure line, so it no longer attempts to use the capability, but of course that does no good if the binary is still installed with the capability! I will do a new scratch build.

BTW Adam, if this new scratch build works, then I think we should prefer a gnome-keyring update rather than a glib update. (That said, of course it was good that you prepared the glib fix when you saw that my gnome-keyring fix didn't work.)

> The gnome-keyring-daemon is started in a weird way, IIRC. i.e. by the PAM
> module. I wonder if we could increase its ulimit somehow without affecting
> the rest of the session.
> 
> Some knobs we can modify are:
>  1. user@.service LimitMEMLOCK= value; which should affect most of the
> session (but not e.g. the shell through SSH)
>  2. DefaultLimitMEMLOCK= to decrease it again for most applications
>  3. LimitMEMLOCK= for single units
> 
> But, they keyring is started from a PAM module and does seteuid/setegid to
> drop privileges.

That's going to make it *real* hard to debug why gnomes-keyring behaves different when started via the command line.

> Or, maybe a stupid idea. But could we possibly use socket-activation and
> write the password into a systemd managed socket to start the keyring? Then
> we have a systemd unit in the user session where we can play around with the
> options and the above knobs could be enough to do everything.

Might still be confusing, but that seems a lot simpler. Anyway, it's beyond the scope of this issue IMO.

Comment 26 Michael Catanzaro 2021-09-22 13:51:08 UTC
Adam, here is a second scratch build to test:

https://koji.fedoraproject.org/koji/taskinfo?taskID=76116044

If that works, I'll prepare a real update.

Comment 27 Adam Williamson 2021-09-22 15:26:49 UTC
Thanks, Michael. We already built RC1 with the glib update in it. If the new scratch build works we can take the workaround back out of glib as a post-Beta update, though. I'll test it.

Comment 28 Adam Williamson 2021-09-22 17:00:17 UTC
Hah, so I tested that and it doesn't work in the built image...then I checked and found the built image doesn't have that build in it. You've built it as gnome-keyring-40.0-1golbat2.fc35 , but the current stable is 40.0-2.fc35, which is higher versioned. So that's what gets pulled in when we build the image, not the scratch build.

I'll re-run the test and pull in the scratch build manually after install but before running g-i-s, I guess.

Comment 29 Adam Williamson 2021-09-22 17:29:58 UTC
OK, with the scratch build forced in, it looks like it does work. No delay, login keyring is unlocked.

Comment 30 Michael Catanzaro 2021-09-22 17:39:57 UTC
(In reply to Adam Williamson from comment #28)
> Hah, so I tested that and it doesn't work in the built image...then I
> checked and found the built image doesn't have that build in it. You've
> built it as gnome-keyring-40.0-1golbat2.fc35 , but the current stable is
> 40.0-2.fc35, which is higher versioned. So that's what gets pulled in when
> we build the image, not the scratch build.

Sigh, sorry about that. The whole point of the weird custom NVR was to keep it higher than the current NVR but lower than the next real build. Oh well.

Anyway, since we have the glib workaround this is no longer urgent. Daiki, is it OK to build gnome-keyring using --without-libcap-ng and remove the capabilities from the binary? I can handle it if you agree.

Comment 31 Michael Catanzaro 2021-09-22 17:40:38 UTC
*** Bug 2006314 has been marked as a duplicate of this bug. ***

Comment 32 Adam Williamson 2021-09-22 17:48:38 UTC
yeah, 2golbat2 would've worked :D you may just have forgotten to 'git pull' before doing the scratch build, I guess.

I left the glib workaround in for Beta RC2 as it seemed safer. I think it's fine to go with that for Beta then we can get it flipped over with post-Beta updates.

Comment 33 Daiki Ueno 2021-09-22 18:01:17 UTC
(In reply to Michael Catanzaro from comment #30)
> (In reply to Adam Williamson from comment #28)
> > Hah, so I tested that and it doesn't work in the built image...then I
> > checked and found the built image doesn't have that build in it. You've
> > built it as gnome-keyring-40.0-1golbat2.fc35 , but the current stable is
> > 40.0-2.fc35, which is higher versioned. So that's what gets pulled in when
> > we build the image, not the scratch build.
> 
> Sigh, sorry about that. The whole point of the weird custom NVR was to keep
> it higher than the current NVR but lower than the next real build. Oh well.
> 
> Anyway, since we have the glib workaround this is no longer urgent. Daiki,
> is it OK to build gnome-keyring using --without-libcap-ng and remove the
> capabilities from the binary? I can handle it if you agree.

I guess I have to agree, sorry for not responding in upstream. Steve, do you have any strong objections? We could make it better after migrating to a new keyring file format...

Comment 34 Michael Catanzaro 2021-09-22 18:37:21 UTC
(In reply to Adam Williamson from comment #32)
> yeah, 2golbat2 would've worked :D you may just have forgotten to 'git pull'
> before doing the scratch build, I guess.

Nah, I just messed up and assumed the current version was -1. A look at the diff would have avoided that. Oh well.

> I left the glib workaround in for Beta RC2 as it seemed safer. I think it's
> fine to go with that for Beta then we can get it flipped over with post-Beta
> updates.

Agreed. For simplicity, I'm going to revert your glib change in rawhide now, but I'll leave it alone in F35 since there is no particularly urgent reason to drop your workaround.

(In reply to Daiki Ueno from comment #33)
> I guess I have to agree, sorry for not responding in upstream. Steve, do you
> have any strong objections? We could make it better after migrating to a new
> keyring file format...

I've gone ahead with the change in Fedora. Let's keep discussing in https://gitlab.gnome.org/GNOME/gnome-keyring/-/issues/77.

Comment 35 Fedora Update System 2021-09-22 18:47:25 UTC
FEDORA-2021-fa340d4bf0 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-fa340d4bf0`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-fa340d4bf0

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 36 Fedora Update System 2021-09-22 18:48:11 UTC
FEDORA-2021-824a0b6f04 has been submitted as an update to Fedora 35. https://bodhi.fedoraproject.org/updates/FEDORA-2021-824a0b6f04

Comment 37 Steve Grubb 2021-09-22 18:52:13 UTC
I suppose it's OK. I really have no idea how gnome-keyring has changed since the last time I looked at it.

Comment 38 Kamil Páral 2021-09-23 09:54:10 UTC
Created attachment 1825541 [details]
third party repos screenshot

I tested F35 RC2 with
glib2-2.70.0-2.fc35.x86_64
gnome-initial-setup-41~rc-3.fc35.x86_64

I can't test wifi configuration, but the other problems seem resolved. Location services and problem reporting toggles are honored. The login keyring is now functional.

I do have a question about third party repositories. Enabling them in gnome-initial-setup toggles "Enable New Repositories" under "Fedora Third Party Repositories", but doesn't actually enable any of those 4 listed (see the attached screenshot). So when searching for Chrome or Steam, there are no results. Is that expected/desired? Should I file a separate bug about it? (The flathub filtered repo is also nowhere to be found, expected?)

Comment 39 Geraldo Simião 2021-09-23 11:00:12 UTC
tested Fedora-Workstation-Live-x86_64-35_Beta-1.2.iso and it seems to me this bug is really fixed now.

Comment 40 Geraldo Simião 2021-09-23 11:04:16 UTC
Made a video from the sucessful first and second login: https://youtu.be/7S-bMIyJbwQ

Comment 41 Michael Catanzaro 2021-09-23 13:15:59 UTC
(In reply to Kamil Páral from comment #38)
> I do have a question about third party repositories. Enabling them in
> gnome-initial-setup toggles "Enable New Repositories" under "Fedora Third
> Party Repositories", but doesn't actually enable any of those 4 listed (see
> the attached screenshot).

That's all expected.

> So when searching for Chrome or Steam, there are
> no results. Is that expected/desired? Should I file a separate bug about it?

Not expected. The repo metadata should be enabled. Please report a bug against gnome-software.

> (The flathub filtered repo is also nowhere to be found, expected?)

Not sure. I would start with a bug against fedora-third-party.

Comment 42 Michael Catanzaro 2021-09-23 13:17:01 UTC
(In reply to Michael Catanzaro from comment #41)
> Not sure. I would start with a bug against fedora-third-party.

Actually I think that's just bug #2001837, which is not fixed yet.

Comment 43 Fedora Update System 2021-09-24 00:31:13 UTC
FEDORA-2021-fa340d4bf0 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 44 Fedora Update System 2021-09-24 02:51:15 UTC
FEDORA-2021-824a0b6f04 has been pushed to the Fedora 35 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-824a0b6f04`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-824a0b6f04

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 45 Fedora Update System 2021-09-24 20:32:40 UTC
FEDORA-2021-824a0b6f04 has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 46 Michael Catanzaro 2021-09-24 21:19:26 UTC
(In reply to Michael Catanzaro from comment #34)
> Agreed. For simplicity, I'm going to revert your glib change in rawhide now,
> but I'll leave it alone in F35 since there is no particularly urgent reason
> to drop your workaround.

Note: I'm going to revert the glib workaround in F35 now, since it should no longer be needed. If anything goes wrong we can always put it back, but I don't expect problems.


Note You need to log in before you can comment on or make changes to this bug.