Bug 1373217 - frozen screen on login to wayland session
Summary: frozen screen on login to wayland session
Keywords:
Status: CLOSED DUPLICATE of bug 1394755
Alias: None
Product: Fedora
Classification: Fedora
Component: gstreamer1-vaapi
Version: 25
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Simon Farnsworth
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: WaylandRelated
TreeView+ depends on / blocked
 
Reported: 2016-09-05 13:52 UTC by Kamil Páral
Modified: 2016-12-14 14:13 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-12-14 08:49:48 UTC


Attachments (Terms of Use)
system journal after hang happened (155.07 KB, text/plain)
2016-09-05 13:53 UTC, Kamil Páral
no flags Details
systemctl status --all during the hang (197.92 KB, text/plain)
2016-09-05 13:54 UTC, Kamil Páral
no flags Details
rpm -qa output (89.12 KB, text/plain)
2016-09-05 13:54 UTC, Kamil Páral
no flags Details
pstack on gsd process 1 (2.62 KB, text/plain)
2016-09-05 13:54 UTC, Kamil Páral
no flags Details
pstack on gsd process 2 (1.25 KB, text/plain)
2016-09-05 13:54 UTC, Kamil Páral
no flags Details
journal during frozen login (17.95 KB, text/plain)
2016-12-05 14:18 UTC, Kamil Páral
no flags Details
rpm -qa output (91.00 KB, text/plain)
2016-12-05 14:19 UTC, Kamil Páral
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1123536 None CLOSED Cannot play video with VA-API acceleration in a Wayland session 2019-02-22 14:55:53 UTC
Red Hat Bugzilla 1394755 None ASSIGNED screen recording freezes GNOME under Wayland when gstreamer cache is updated 2019-02-22 14:55:51 UTC

Internal Links: 1123536 1394755

Description Kamil Páral 2016-09-05 13:52:49 UTC
Description of problem:
After update to https://bodhi.fedoraproject.org/updates/FEDORA-2016-c565698917, I can't log in into Wayland session at all (X11 works). The screen stays gray, there's a frozen cursor in the middle (can't move it), and everything just hangs (I waited a minute or two). I can't switch to a different VT, but I can ssh in and reboot the machine.

The most suspicious thing I see in the journal:

...
Sep 05 15:31:19 dryad gnome-settings-[1641]: g_task_return_error: assertion 'error != NULL' failed
...
Sep 05 15:32:54 dryad gnome-session[1830]: gnome-session-binary[1830]: WARNING: Application 'gnome-settings-daemon.desktop' failed to register before timeout
Sep 05 15:32:54 dryad gnome-session-binary[1830]: WARNING: Application 'gnome-settings-daemon.desktop' failed to register before timeout
Sep 05 15:32:54 dryad gnome-session-binary[1830]: Unrecoverable failure in required component gnome-settings-daemon.desktop
...


Version-Release number of selected component (if applicable):
gnome-shell-3.21.91-1.fc25.x86_64
gnome-settings-daemon-3.21.90-1.fc25.x86_64
gnome-session-wayland-session-3.21.90-1.fc25.x86_64
libwayland-client-1.11.92-1.fc25.x86_64
libwayland-cursor-1.11.92-1.fc25.x86_64
libwayland-server-1.11.92-1.fc25.x86_64
mesa-libwayland-egl-12.0.1-2.fc25.x86_64
mesa-libwayland-egl-devel-12.0.1-2.fc25.x86_64
wayland-devel-1.11.92-1.fc25.x86_64
wayland-protocols-devel-1.7-1.fc25.noarch
xorg-x11-server-Xwayland-1.18.4-5.fc25.x86_64
mutter-3.21.91-1.fc25.x86_64

How reproducible:
100% atm

Steps to Reproduce:
1. boot
2. try to log in using wayland
3. see only gray screen and a cursor, frozen

Additional info:
Please note that I have already seen this or a similar bug when I upgraded from F24 last week. The symptoms were the same, but (I think) I could switch to a VT, and the problem went away soon for some unknown reason (either because using X11 for a while "fixed" it, or because I tried some rpm version shuffling). But this time, it doesn't go away. I can't use Wayland at all.

Comment 1 Kamil Páral 2016-09-05 13:53:52 UTC
Created attachment 1197959 [details]
system journal after hang happened

Comment 2 Kamil Páral 2016-09-05 13:54:05 UTC
Created attachment 1197960 [details]
systemctl status --all during the hang

Comment 3 Kamil Páral 2016-09-05 13:54:12 UTC
Created attachment 1197961 [details]
rpm -qa output

Comment 4 Kamil Páral 2016-09-05 13:54:23 UTC
Created attachment 1197962 [details]
pstack on gsd process 1

Comment 5 Kamil Páral 2016-09-05 13:54:34 UTC
Created attachment 1197963 [details]
pstack on gsd process 2

Comment 6 Christian Stadelmann 2016-09-05 14:04:00 UTC
How long did you wait to log in to your session, are you sure it doesn't just take very long?

I've recently seen an issue in g-s-d which causes login to hang for a very long time (like 30 seconds) until login finally succeeds. You might want to try the workaround: Downgrade cups-libs to 2.1.4 and try to login again.

Source and possible duplicate of this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1366775

Comment 7 Kamil Páral 2016-09-05 14:20:23 UTC
(In reply to Christian Stadelmann from comment #6)
> How long did you wait to log in to your session, are you sure it doesn't
> just take very long?

I waited for about 4 minutes.

> I've recently seen an issue in g-s-d which causes login to hang for a very
> long time (like 30 seconds) until login finally succeeds. You might want to
> try the workaround: Downgrade cups-libs to 2.1.4 and try to login again.

I masked cups.service and it didn't help. Seems to be a separate issue (unless masking is not supposed to avoid it).

Comment 8 Marek Kašík 2016-09-05 14:27:24 UTC
(In reply to Kamil Páral from comment #7)
> (In reply to Christian Stadelmann from comment #6)
> > I've recently seen an issue in g-s-d which causes login to hang for a very
> > long time (like 30 seconds) until login finally succeeds. You might want to
> > try the workaround: Downgrade cups-libs to 2.1.4 and try to login again.
> 
> I masked cups.service and it didn't help. Seems to be a separate issue
> (unless masking is not supposed to avoid it).

Masking the cups.service is not supposed to avoid the problem about which Christian is talking. Downgrading CUPS or explicit start of cups.service fixes the problem.

Comment 9 Kamil Páral 2016-09-05 15:03:37 UTC
After 2 hours of bisecting, I found out the culprit is gstreamer1-vaapi, not gnome components. This transaction breaks the desktop for me:

Command Line   : distro-sync gstreamer1-vaapi
Transaction performed with:
    Installed     dnf-1.1.10-1.fc25.noarch        @@commandline
    Installed     rpm-4.13.0-0.rc1.46.fc25.x86_64 @@commandline
Packages Altered:
    Upgraded gstreamer1-vaapi-1.9.1-1.fc25.x86_64 @fedora
    Upgrade                   1.9.2-1.fc25.x86_64 @updates-testing


My hardware is:
00:02.0 VGA compatible controller [0300]: Intel Corporation HD Graphics 5500 [8086:1616] (rev 09)

In case this is somehow related to vdpau (I think there's some relation between vaapi and vdpau), I have these packages installed:
libva-vdpau-driver-0.7.4-14.fc24.x86_64
libvdpau-1.1.1-3.fc24.x86_64
mesa-vdpau-drivers-12.0.1-2.fc25.x86_64
vdpauinfo-1.0-5.fc24.x86_64

Comment 10 Christian Stadelmann 2016-09-05 15:42:34 UTC
Afaik libva (=VAAPI) uses vdpau on Nvidia or some AMD GPU hardware if libva-vdpau-driver is installed. In case you have an intel hardware, the libva-vdpau-driver shouldn't be used at all.
I've had some issues with libva-intel-driver so I uninstalled it too (on F24).

See also: https://bugzilla.redhat.com/show_bug.cgi?id=1123536

Comment 11 Nicolas Chauvet (kwizart) 2016-09-05 15:51:38 UTC
Can we have any info about why gst is even run on the start of a wayland session ?

Can you reproduce with libva-vdpau-driver removed ?

Comment 12 Kamil Páral 2016-09-06 07:57:47 UTC
(In reply to Nicolas Chauvet (kwizart) from comment #11)
> Can you reproduce with libva-vdpau-driver removed ?

I tried removing libva-vdpau-driver, and then also installing libva-intel-driver. None of that helped, same issue.

Comment 13 Kamil Páral 2016-09-06 08:59:23 UTC
If I remove gstreamer1-vaapi and keep libva* packages installed, there's no problem during login.

Comment 14 Nicolas Chauvet (kwizart) 2016-09-06 09:59:24 UTC
Can you try to load a video with another player (such as vlc) using vaapi.
Then with gstreamer ?

Which component tries to load a video "at" login ?

Comment 15 Kamil Páral 2016-09-06 13:09:52 UTC
(In reply to Nicolas Chauvet (kwizart) from comment #14)
> Can you try to load a video with another player (such as vlc) using vaapi.
> Then with gstreamer ?

When using totem with gstreamer1-vaapi, I see only a black screen. Totem prints this:

$ totem sintel_trailer-480p.mp4 
libva info: VA-API version 0.39.2
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
libva info: VA-API version 0.39.2
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
libva info: VA-API version 0.39.2
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib64/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
... <unrelated output>


Vlc plays the video fine whichever output type I choose, but I suspect it does not use hardware acceleration in any of those cases. The output is always the same:

$ vlc -v sintel_trailer-480p.mp4 
VLC media player 3.0.0-git Vetinari (revision 2.2.0-git-8681-g749293f)
[000055a6a2aab1c8] core libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
[00007f6558c01728] mp4 stream warning: unknown box type ccpy (incompletely loaded)
[00007f6558c01728] mp4 stream warning: unknown box type desc (incompletely loaded)
[00007f6558c018f8] mp4 demux warning: elst box found
[00007f6558c018f8] mp4 demux warning: STTS table of 1 entries
[00007f6558c018f8] mp4 demux warning: CTTS table of 1059 entries
[00007f6558c018f8] mp4 demux warning: STTS table of 1 entries
[00007f6558cc15f8] avcodec decoder warning: thread type 1: disabling hardware acceleration
[00007f6558d7ae88] faad decoder warning: decoded zero sample
Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared object file: No such file or directory
[00007f6558cc15f8] avcodec decoder warning: plane 0 not aligned: disabling direct rendering
[000055a6a2b91c58] pulse audio output warning: starting late (-12197 us)

Comment 16 Christian Stadelmann 2016-09-28 19:01:37 UTC
(In reply to Kamil Páral from comment #7)
> (In reply to Christian Stadelmann from comment #6)
> > How long did you wait to log in to your session, are you sure it doesn't
> > just take very long?
> 
> I waited for about 4 minutes.

On an SSD or a rotating disk? I'm asking because of the access times. I am running into this freeze every time I log in for the first time after boot and on a SSD it takes me 5…30 seconds.

> > I've recently seen an issue in g-s-d which causes login to hang for a very
> > long time (like 30 seconds) until login finally succeeds. You might want to
> > try the workaround: Downgrade cups-libs to 2.1.4 and try to login again.
> 
> I masked cups.service and it didn't help. Seems to be a separate issue
> (unless masking is not supposed to avoid it).

Ok, the cups issue is completely unrelated, see the bug report there.

Comment 17 Kamil Páral 2016-10-03 12:38:39 UTC
Yes, this is about gstreamer1-vaapi, not cups.

Comment 18 Anass Ahmed 2016-10-21 15:06:43 UTC
I don't know if it's the same or not, but I've got the same effect right after upgrading from F24 to F25 Beta, but (I think) it was caused by GNOME Shell because after I logged into GNOME on Xorg, SELinux troubleshooter gave me a notification (I ignored it at first, but I couldn't log in to the wayland session without fixing it).

I reported the issue in Bug #1385090. and I fixed the issue with commands:
# ausearch -c 'gnome-shell' --raw | audit2allow -M my-gnomeshell
# semodule -X 300 -i my-gnomeshell.pp

Comment 19 Kamil Páral 2016-11-04 13:44:22 UTC
https://bodhi.fedoraproject.org/updates/FEDORA-2016-a3eee5e16a seems to fix this. No login issues and totem now correctly plays videos.

Comment 20 Kamil Páral 2016-12-05 09:40:15 UTC
I see this again with gstreamer1-vaapi-1.10.1-1.fc25 from https://bodhi.fedoraproject.org/updates/FEDORA-2016-344df1623c . This is in journal related to the frozen login:

Dec 05 10:14:12 dryad gst-plugin-scan[2074]: Couldn't g_module_open libpython. Reason: /usr/lib64/libpython3.5m.so: cannot open shared object file: No such file or directory
...
Dec 05 10:15:42 dryad gnome-session[1893]: gnome-session-binary[1893]: WARNING: Application 'gnome-settings-daemon.desktop' failed to register before timeout
Dec 05 10:15:42 dryad gnome-session-binary[1893]: WARNING: Application 'gnome-settings-daemon.desktop' failed to register before timeout
Dec 05 10:15:42 dryad gnome-session-binary[1893]: Unrecoverable failure in required component gnome-settings-daemon.desktop

Comment 21 Kamil Páral 2016-12-05 09:47:09 UTC
I asked for gnome-settings-daemon to be more resilient to gstreamer1-vaapi issues here:
https://bugzilla.gnome.org/show_bug.cgi?id=775619

Comment 22 Nicolas Chauvet (kwizart) 2016-12-05 11:33:02 UTC
Which component use system-python-libs (libpython3.5m.so) in gstreamer ? (not gstreamer1-vaapi at least).
I don't find it installed by default on my current system (f24)

Comment 23 Simon Farnsworth 2016-12-05 11:51:10 UTC
(In reply to Kamil Páral from comment #20)
> I see this again with gstreamer1-vaapi-1.10.1-1.fc25 from
> https://bodhi.fedoraproject.org/updates/FEDORA-2016-344df1623c . This is in
> journal related to the frozen login:
> 
> Dec 05 10:14:12 dryad gst-plugin-scan[2074]: Couldn't g_module_open
> libpython. Reason: /usr/lib64/libpython3.5m.so: cannot open shared object
> file: No such file or directory
> ...
> Dec 05 10:15:42 dryad gnome-session[1893]: gnome-session-binary[1893]:
> WARNING: Application 'gnome-settings-daemon.desktop' failed to register
> before timeout
> Dec 05 10:15:42 dryad gnome-session-binary[1893]: WARNING: Application
> 'gnome-settings-daemon.desktop' failed to register before timeout
> Dec 05 10:15:42 dryad gnome-session-binary[1893]: Unrecoverable failure in
> required component gnome-settings-daemon.desktop

Can you supply the snipped bit? There's a library in python3-gstreamer1 that would cause the error you're seeing, but it should fail fast if it can't find the binaries it's expecting.

Also, please retry without gstreamer1-vaapi installed, to confirm that it's gstreamer1-vaapi and not python3-gstreamer1 causing the bug. If the presence or absence of the bug is directly tied to gstreamer1-vaapi being installed, then we at least know it's not the message you're seeing that causes the freeze.

Comment 24 Kamil Páral 2016-12-05 14:18:09 UTC
So, after further debugging I found out this problem is related to EasyScreenCast extension [1]. If I disable it, the login works fine - probably because nothing touches vaapi during login. But if EasyScreenCast is enabled, the login is stuck. Either downgrading gstreamer1-vaapi to 1.10.0-1.fc25, or removing it completely solves the problem.

[1] https://extensions.gnome.org/extension/690/easyscreencast/

Comment 25 Kamil Páral 2016-12-05 14:18:45 UTC
Created attachment 1228071 [details]
journal during frozen login

Comment 26 Kamil Páral 2016-12-05 14:19:05 UTC
Created attachment 1228072 [details]
rpm -qa output

Comment 27 Kamil Páral 2016-12-05 14:55:11 UTC
This is getting ridiculous, but after experimenting a bit I can't login at all even with gstreamer1-vaapi-1.10.0-1.fc25, with EasyScreenCast enabled. I either need to disable the extension, use X11 or uninstall gstreamer1-vaapi. I think this got triggered by logging in once using X11, and since that time Wayland login stopped working even for gstreamer1-vaapi 1.10.0. It's a mess, and I'm getting a suspicion this could have the same root cause as bug 1394755. However, I experimented with ~/.cache/gstreamer-1.0/registry.x86_64.bin and still could not get it working.

Comment 28 Kamil Páral 2016-12-14 08:49:48 UTC
After reading https://bugzilla.gnome.org/show_bug.cgi?id=776041#c1 I believe this is essentially the same problem as bug 1394755. The gstreamer cache registry gets updated due to the a gstreamer plugin updated (in this case vaapi) and causes a deadlock in gnome-shell. Therefore probably not related to vaapi plugin at all. Closing as a duplicate.

*** This bug has been marked as a duplicate of bug 1394755 ***

Comment 29 Kamil Páral 2016-12-14 14:13:33 UTC
Confirmed with Rui that this is indeed a duplicate. During login, gst-plugin-scanner is running (waiting) with this stack trace:

Thread 1 (Thread 0x7f09333b8700 (LWP 3608)):
#0  0x00007f0931e31ff0 in __poll_nocancel () at /lib64/libc.so.6
#1  0x00007f092fabb7c9 in wl_display_poll () at /lib64/libwayland-client.so.0
#2  0x00007f092fabcdac in wl_display_dispatch_queue () at /lib64/libwayland-client.so.0
#3  0x00007f092fabd00b in wl_display_roundtrip_queue () at /lib64/libwayland-client.so.0
#4  0x00007f093137fcb2 in gst_vaapi_display_wayland_setup () at /usr/lib64/gstreamer-1.0/libgstvaapi.so
#5  0x00007f09313459d9 in gst_vaapi_display_new () at /usr/lib64/gstreamer-1.0/libgstvaapi.so
#6  0x00007f093132129a in gst_vaapi_create_test_display () at /usr/lib64/gstreamer-1.0/libgstvaapi.so
#7  0x00007f093131bd17 in plugin_init () at /usr/lib64/gstreamer-1.0/libgstvaapi.so
#8  0x00007f0932f0ec67 in gst_plugin_register_func () at /lib64/libgstreamer-1.0.so.0
#9  0x00007f0932f10bb6 in _priv_gst_plugin_load_file_for_registry () at /lib64/libgstreamer-1.0.so.0
#10 0x00007f0932f135d3 in exchange_packets () at /lib64/libgstreamer-1.0.so.0
#11 0x00007f0932f145e8 in _gst_plugin_loader_client_run () at /lib64/libgstreamer-1.0.so.0
#12 0x000055742258d914 in main ()
Detaching from program: /usr/libexec/gstreamer-1.0/gst-plugin-scanner, process 3608


Note You need to log in before you can comment on or make changes to this bug.