Bug 1806778 - Rapid fire crashing of virtualbox-guest-additions on Rawhide and fc32 beta
Summary: Rapid fire crashing of virtualbox-guest-additions on Rawhide and fc32 beta
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: virtualbox-guest-additions
Version: rawhide
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Hans de Goede
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1813060 1813840 1815417 1818792 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-02-25 02:16 UTC by Ian Laurie
Modified: 2020-03-30 13:01 UTC (History)
9 users (show)

Fixed In Version: virtualbox-guest-additions-6.1.4-2.fc32 virtualbox-guest-additions-6.1.4-4.fc32 virtualbox-guest-additions-6.1.4-3.fc30 virtualbox-guest-additions-6.1.4-3.fc31
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-03-16 20:20:46 UTC
Type: Bug


Attachments (Terms of Use)
Screen cap of abrt-gui (227.84 KB, image/png)
2020-02-25 02:16 UTC, Ian Laurie
no flags Details
vboxservice messages (4.67 KB, text/plain)
2020-02-25 03:45 UTC, Ian Laurie
no flags Details
dmesg messages (12.05 KB, text/plain)
2020-02-25 03:46 UTC, Ian Laurie
no flags Details
Display mode of running machine (81.70 KB, image/jpeg)
2020-02-29 01:12 UTC, Ian Laurie
no flags Details
VMSVGA mode crash (80.93 KB, image/jpeg)
2020-02-29 01:14 UTC, Ian Laurie
no flags Details
VMSVGA mode still crashing lsmod shown (197.31 KB, image/png)
2020-02-29 23:43 UTC, Ian Laurie
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1789545 None None None 2020-02-28 22:36:27 UTC

Description Ian Laurie 2020-02-25 02:16:39 UTC
Created attachment 1665544 [details]
Screen cap of abrt-gui

Description of problem:
Constant crashing of virtualbox-guest-additions and VBoxClient with rapid fire abort alerts.  Crashes every few seconds.

Version-Release number of selected component (if applicable):
virtualbox-guest-additions-6.1.4-1.fc33.x86_64
kernel-5.6.0-0.rc2.git3.1.fc33.x86_64

How reproducible:
Always on 2 virtual machines, one is GNOME the other is MATE.

Steps to Reproduce:
1. Apply latest updates to Rawhide
2.
3.

Additional info:
Booting from my previous kernel-5.6.0-0.rc2.git1.1.fc33.x86_64 doesn't fix the issue, seems to stick with latest virtualbox-guest-additions.

Comment 1 Sergio Basto 2020-02-25 03:02:29 UTC
Hi, 
Can you post latest lines of , journalctl -u vboxservice and dmesg ? please 

Thanks

Comment 2 Ian Laurie 2020-02-25 03:45:59 UTC
Created attachment 1665550 [details]
vboxservice messages

Comment 3 Ian Laurie 2020-02-25 03:46:25 UTC
Created attachment 1665551 [details]
dmesg messages

Comment 4 Ian Laurie 2020-02-25 03:47:39 UTC
Attached requested files.  Let me know if there is anything else I can do.

Comment 5 Ian Laurie 2020-02-25 03:54:16 UTC
Worth mentioning that I am not seeing this problem with the fc32 beta branch running the same version of virtualbox-guest-additions.

Comment 6 Sergio Basto 2020-02-25 04:01:24 UTC
hum , I'm checking , it can be old akmods, could you do `dnf remove akmod-VirtualBox` i.e. be sure that all akmods and kmod-VirtualBox are deleted . and reboot please

Comment 7 Ian Laurie 2020-02-25 04:10:50 UTC
Not sure I am following you... but:

rawhide$ rpm -qa | grep akmod
rawhide$ rpm -qa | grep kmod
kmod-26-5.fc32.x86_64
kmod-libs-27-1.fc33.x86_64
kmod-27-1.fc33.x86_64
rawhide$

Comment 8 Sergio Basto 2020-02-25 04:16:22 UTC
(In reply to Ian Laurie from comment #7)
> Not sure I am following you... but:
> 
> rawhide$ rpm -qa | grep akmod
> rawhide$ rpm -qa | grep kmod
> kmod-26-5.fc32.x86_64
> kmod-libs-27-1.fc33.x86_64
> kmod-27-1.fc33.x86_64
> rawhide$

ah OK , good , I though you may use akmod-VirtualBox from RPMFusion . 

ok I see you have "VBoxClient[8739]: segfault" 

by curiosity in what OS are VirtuaBox server , and his version ? 

and what kernel you use in F32 and in rawhide ? 

Thanks

Comment 9 Ian Laurie 2020-02-25 04:30:22 UTC
Having the issue on two hosts.  One is Windows 10 and the other is Windows 8.1, both systems are running VirtualBox 6.1.4 r136177 (latest version as of now).

Rawhide kernels showing the problem: kernel-5.6.0-0.rc2.git3.1.fc33.x86_64, kernel-5.6.0-0.rc2.git1.1.fc33.x86_64 and kernel-5.6.0-0.rc1.git2.1.fc33.x86_64 (all three show the problem).

fc32beta kernel (working ok) is kernel-5.6.0-0.rc2.git0.1.fc32 (which is the latest as of now for fc32 beta).

I never used VirtualBox drivers from RPMFusion.  I used the default kernel drivers and before 5.6.0 I was using the vboxsf driver from https://github.com/jwrdegoede/vboxsf/.

Comment 10 Ian Laurie 2020-02-25 07:41:44 UTC
I had another working Rawhide VM (with both Xfce and MATE) with an older version of the guest additions as follows:

rawhide$ uname -r
5.6.0-0.rc2.git2.1.fc33.x86_64
rawhide$ rpm -qa | grep virtualbox
virtualbox-guest-additions-6.1.2-2.fc32.x86_64
rawhide$ cat /etc/fedora-release
Fedora release 33 (Rawhide)

Note this was from when most of the Rawhide packages still had the ".fc32".  This VM worked as expected.

I then updated ONLY the virtualbox additions (NOT the kernel) to this:

rawhide$ rpm -qa | grep virtualbox
virtualbox-guest-additions-6.1.4-1.fc33.x86_64 

This VM now has the same problem.

I then updated everything (including the kernel) and of course the problem remains as per the other VMs.

Comment 11 Ian Laurie 2020-02-26 05:19:39 UTC
Not sure what's changed, but alarmingly I am now seeing this issue on the fc32 beta branch.

Comment 12 Villy Kruse 2020-02-26 09:23:12 UTC
My too:

Host:   fc31 xfce environment VirtualBox-6.1.4-1.fc31.x86_64

Client: fc32 xfce environment virtualbox-guest-additions-6.1.4-1.fc32.x86_64

Feb 25 08:12:28 luks systemd[1]: Started Process Core Dump (PID 1478/UID 0).
Feb 25 08:12:29 luks abrt-notification[1480]: Process 1093 (VBoxClient) crashed in XQueryExtension()
Feb 25 08:12:30 luks systemd-coredump[1479]: Process 1473 (VBoxClient) of user 1000 dumped core.
                                             
                                             Stack trace of thread 1473:
                                             #0  0x00007f99e89cc04f XQueryExtension (libX11.so.6 + 0x3d04f)
                                             #1  0x00007f99e89bf4a7 XInitExtension (libX11.so.6 + 0x304a7)
                                             #2  0x00007f99e890e6d1 XextAddDisplay (libXext.so.6 + 0xe6d1)
                                             #3  0x00007f99e8984a50 XRRFindDisplay.part.0 (libXrandr.so.2 + 0x2a50)
                                             #4  0x00007f99e89851d8 XRRQueryExtension (libXrandr.so.2 + 0x31d8)
                                             #5  0x00000000004090da _ZL3runPP11VBCLSERVICEb (VBoxClient + 0x90da)
                                             #6  0x00000000004076ec main (VBoxClient + 0x76ec)
                                             #7  0x00007f99e871c042 __libc_start_main (libc.so.6 + 0x27042)
                                             #8  0x00000000004078de _start (VBoxClient + 0x78de)
Feb 25 08:12:31 luks systemd[1]: systemd-coredump@5-1478-0.service: Succeeded.
Feb 25 08:12:31 luks systemd[1]: systemd-coredump@5-1478-0.service: Consumed 1.302s CPU time.

Comment 13 Sergio Basto 2020-02-27 05:19:53 UTC
I also see the problem with one F31 guest , after downgrade to  virtualbox-guest-additions-6.1.2 it fix the problem . wee need report upstream ...

Comment 14 Ian Laurie 2020-02-27 07:24:40 UTC
I thought of trying to downgrade the virtualbox-guest-additions but with the 5.6 kernel in Rawhide & fc32beta it is a bit tricky.  

Even compiling from VirtualBox source would be problematic because I don't believe anything officially released upstream is kernel 5.6 compatible yet, and the available test builds (just checked them) for the VirtualBox 6.1 stream are actually older than what has been officially released.  So until they update their test builds page we're out of luck with Rawhide/fc32beta.

Comment 15 Ian Laurie 2020-02-28 22:12:56 UTC
@Sergio upstream report is here https://www.virtualbox.org/ticket/19357

Comment 16 Sergio Basto 2020-02-28 22:41:14 UTC
as mention in bug #1789545, 

Settings -> display -> graphics controller 

on 6.1.2 No regressions with VBoxSVGA (guest resize still works as expected), but now with 6.1.4 I think it crash with deprecated VBoxSVGA .

while the new VMSVGA seems we got some resize problems .

So what "graphics controller" are you using ?

Comment 17 Ian Laurie 2020-02-28 23:10:23 UTC
I am using VBoxSVGA because I have never managed to get VMSVGA to work with Fedora, any version of Fedora, ever.  

VMSVGA seems to work OK for RHEL and CentOS, but it has never worked for me for Fedora (or any other distro I have tried it with for that matter other than RHEL/CentOS).  So I never understood why with 6.1.x we are being urged to use it (you get an error flag for any setting other than VMSVGA with a Linux guest).

In fact with VMSVGA I don't even have the Red Hat Graphical Boot working correctly (works fine with VBOXSVGA).

OK just tried VMSVGA with fc32beta... as completely expected old style text boot, then 800x600 display that won't scale.  However this is the behavior I always see with VMSVGA going as far back as I can remember, it has never worked for me with any version of any Linux except RHEL/CentOS.  So for me this isn't a "latest version thing".  However I am not getting the vbox driver crashes.  But it's unusable at 800x600 (or whatever it is, I didn't count the pixels).

Also tried VBoxVGA and I get the graphical boot (good), a full sized GUI (great) but the driver crashes same as VBoxSVGA.

I must emphasize that over many versions I have periodically tested VMSVGA with Fedora and [for me] it has never worked (and still doesn't).

Comment 18 Ian Laurie 2020-02-29 00:18:51 UTC
*sigh* I spoke too soon.  I am now seeing the VirtualBox driver crashing every few seconds with VMSVGA mode as well.

Comment 19 Sergio Basto 2020-02-29 00:30:02 UTC
(In reply to Ian Laurie from comment #18)
> *sigh* I spoke too soon.  I am now seeing the VirtualBox driver crashing
> every few seconds with VMSVGA mode as well.

I think not , VMSVGA does fix this bug ... 


@Hans
 
I think that changing video adapter to VMSVGA fixes crashing problem of this bug report or in another words old video adapters starting to crash with 6.1.4. 
But VMSVGA have the resize problem . Seems it need run VBoxClient --vmsvga , which is not in autostart and after restart X, resize start working. How VBoxClient --vmsvga may be automatically enable in other distros or configuration still a mystery .

But vboxvideo.ko from Fedora Kernel may need to be checked or updated , I don't know what is happen ... . On Centos 7 in which we use vboxvideo.ko from VirtualBox sources VMSVGA is working without problems . 


Thank you.

Comment 20 Ian Laurie 2020-02-29 01:12:59 UTC
Created attachment 1666512 [details]
Display mode of running machine

Screen cap of VirtualBox manager showing the settings of a running machine (Fedora 32 Beta Branch).

Comment 21 Ian Laurie 2020-02-29 01:14:08 UTC
Created attachment 1666513 [details]
VMSVGA mode crash

Screen cap showing VMSVGA mode still crashes the VirtualBox driver.

Comment 22 Ian Laurie 2020-02-29 01:17:36 UTC
Certainly it seems to happen less with the VMSVGA mode driver, and I have been running a Rawhide VM for some while now in VMSVGA mode without trouble (Rawhide was hard core crashing under VBoxSVGA mode).  

However it is still is happening as shown via the screen caps of my fc32beta VM.  Note I had to reboot about 10 times before it would show the crashing.  Bug is greatly influenced by the video mode but doesn't go away completely.

Comment 23 Ian Laurie 2020-02-29 01:23:19 UTC
Btw I still have that machine instance running, and I will be here for another hour or so.  If there is anything you need me to do that may help, please let me know. They problem may go away again for days, so lets leverage of the problem while I can reproduce it.

Clearing that error results in another crash alert within seconds.

Comment 24 Ian Laurie 2020-02-29 02:01:37 UTC
It's weird, I cannot get the problem to manifest itself at all now on Rawhide in VMSVGA mode, even though Rawhide was the worst in VBoxSVGA mode.  I went for about a week in VBoxSVGA mode on fc32beta not seeing the problem at all (see comment #5) and then all of a sudden it started happening all the time.

It's like a race condition that comes and goes with a 3rd variable we cannot see and have no control over.

Comment 25 Villy Kruse 2020-02-29 06:43:04 UTC
(In reply to Sergio Basto from comment #16)
> as mention in bug #1789545, 
> 
> Settings -> display -> graphics controller 
> 
> on 6.1.2 No regressions with VBoxSVGA (guest resize still works as
> expected), but now with 6.1.4 I think it crash with deprecated VBoxSVGA .
> 
> while the new VMSVGA seems we got some resize problems .
> 
> So what "graphics controller" are you using ?

As I understand VMSVGA will use the old video drivers made for VMware, whereas
the other two will use the videodriver that has been fixed by RedHat/Fedora.

The VMware driver seems to be stuck in the world using only 4:3 displays.

Comment 26 Ian Laurie 2020-02-29 23:43:46 UTC
Created attachment 1666686 [details]
VMSVGA mode still crashing lsmod shown

Regarding comment #22 this morning the very first reboot shows the problem.  I have shown the output of "lsmod | grep vmwgfx" showing that the VMSVGA mode is active, and the VirtualBox driver is still crashing.  As before, if I clear the error in abrt-gui there is another crash in a few seconds.

Comment 27 Sergio Basto 2020-03-01 07:38:43 UTC
(In reply to Villy Kruse from comment #25)
> (In reply to Sergio Basto from comment #16)
> > as mention in bug #1789545, 
> > 
> > Settings -> display -> graphics controller 
> > 
> > on 6.1.2 No regressions with VBoxSVGA (guest resize still works as
> > expected), but now with 6.1.4 I think it crash with deprecated VBoxSVGA .
> > 
> > while the new VMSVGA seems we got some resize problems .
> > 
> > So what "graphics controller" are you using ?
> 
> As I understand VMSVGA will use the old video drivers made for VMware,
> whereas
> the other two will use the videodriver that has been fixed by RedHat/Fedora.
> 
> The VMware driver seems to be stuck in the world using only 4:3 displays.

https://www.mesa3d.org/vmware-guest.html , doesn't seems to me stuck at 4:3 displays ...

Comment 28 Hans de Goede 2020-03-03 15:58:38 UTC
I've just debugged this and it clearly is a bug introduced in some of the changes in 6.1.4, the "VBoxClient --vmsvga-x11" helper process code now has this code:

VirtualBox-6.1.4/src/VBox/Additions/x11/VBoxClient/display-svga-x11.cpp: x11Connect():

    pContext->pDisplay = XOpenDisplay(NULL);
    if (pContext->pDisplay == NULL)
        return;

    if(!XQueryExtension(pContext->pDisplay, "VMWARE_CTRL",
                        &pContext->hVMWMajor, &dummy, &dummy))
    {
        XCloseDisplay(pContext->pDisplay);
        pContext->pDisplay = NULL;
    }

    if (!XRRQueryExtension(pContext->pDisplay, &pContext->hRandREventBase, &pCon
    {
        XCloseDisplay(pContext->pDisplay);
        pContext->pDisplay = NULL;
    }

Notice how the code happily continues after closing the display and setting pContext->pDisplay = NULL when the query for the "VMWARE_CTRL" extension fails; and this will fail when using VboxSVGA as virtual VGA adapter which was the default for a long time, as well as when running a Wayland based desktop and recent GNOME3 versions run as Wayland compositor (rather then on top of X11) by default now, so this is impacting a lot of users.

The fix is to add a return statement after all the pContext->pDisplay = NULL; lines in the x11Connect() function.

I've started builds of the Fedora virtualbox-guest-additions package with this fix added for F30+ and I will create an update with those builds at https://bodhi.fedoraproject.org/updates/ when the builds are finished.

Comment 29 Hans de Goede 2020-03-03 16:03:02 UTC
As for the resize issues, I suggest using VBoxSVGA as virtual graphics card type, then resizing should work fine.

The resize issue when using VMSVGA as virtual graphics card type is tracked in bug 1789545 and is unrelated to this bug (which is about the now fixed VBoxClient --vmsvga-x11 crash).

Comment 30 Fedora Update System 2020-03-03 16:06:33 UTC
FEDORA-2020-9be156194c has been submitted as an update to Fedora 32. https://bodhi.fedoraproject.org/updates/FEDORA-2020-9be156194c

Comment 31 Fedora Update System 2020-03-03 16:06:33 UTC
FEDORA-2020-8a8206dcae has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-8a8206dcae

Comment 32 Fedora Update System 2020-03-03 17:33:57 UTC
virtualbox-guest-additions-6.1.4-2.fc32 has been pushed to the Fedora 32 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-9be156194c

Comment 33 Fedora Update System 2020-03-03 20:28:27 UTC
virtualbox-guest-additions-6.1.4-2.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-3561d3ec62

Comment 34 Fedora Update System 2020-03-03 20:46:42 UTC
virtualbox-guest-additions-6.1.4-2.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-8a8206dcae

Comment 35 Ian Laurie 2020-03-04 00:22:09 UTC
I tested VBoxVGA, VBoxSVGA and VMSVGA.  In VMSVGA mode I tested with and without 3D acceleration and no crashing.  Issue is fixed for me, thanks guys.

Comment 36 Fedora Update System 2020-03-12 18:49:57 UTC
virtualbox-guest-additions-6.1.4-4.fc32 has been pushed to the Fedora 32 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-de54cbd4dd

Comment 37 Hans de Goede 2020-03-12 21:49:48 UTC
*** Bug 1813060 has been marked as a duplicate of this bug. ***

Comment 38 Hans de Goede 2020-03-16 10:12:35 UTC
*** Bug 1813840 has been marked as a duplicate of this bug. ***

Comment 39 Fedora Update System 2020-03-16 20:20:46 UTC
virtualbox-guest-additions-6.1.4-2.fc32 has been pushed to the Fedora 32 stable repository. If problems still persist, please make note of it in this bug report.

Comment 40 Fedora Update System 2020-03-16 20:32:14 UTC
virtualbox-guest-additions-6.1.4-2.fc32 has been pushed to the Fedora 32 stable repository. If problems still persist, please make note of it in this bug report.

Comment 41 Fedora Update System 2020-03-16 20:38:37 UTC
virtualbox-guest-additions-6.1.4-4.fc32 has been pushed to the Fedora 32 stable repository. If problems still persist, please make note of it in this bug report.

Comment 42 Fedora Update System 2020-03-20 01:39:36 UTC
virtualbox-guest-additions-6.1.4-3.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.

Comment 43 Fedora Update System 2020-03-20 01:47:55 UTC
virtualbox-guest-additions-6.1.4-3.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.

Comment 44 Hans de Goede 2020-03-20 09:03:39 UTC
*** Bug 1815417 has been marked as a duplicate of this bug. ***

Comment 45 Hans de Goede 2020-03-30 13:01:53 UTC
*** Bug 1818792 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.