Created attachment 1806163 [details]
Dist-Git patch to resolve this

I've attached a patch to resolve this. It can be applied to the RHEL Dist-Git with "git am -p2".

This impacts the CentOS Hyperscale Workstation spin with KDE. It also breaks VMware's autoresizing: https://pagure.io/centos-sig-hyperscale/spin-bugs/issue/2

Note that this will affect KDE Plasma and potentially other desktops in EPEL with the RHEL 8.5 rebase coming in the fall. It would be ideal to have this swiftly incorporated as an update to open-vm-tools targeting RHEL 8.5.

Hi Neal，
I am QE that test RHEL, I installed RHEL 8.5 on VMware Workstation 16, seems not have the auto resize issue, the RHEL 8.5 installed with open-vm-tools-desktop-11.2.5-2.el8.x86_64 and open-vm-tools-11.2.5-2.el8.x86_64, then I installed CentOS-Stream-8-x86_64-20210727-dvd1.iso on VMware Workstation 16, also not encounter auto resize issue, I use the default Gnome desktop, do you use KDE Plasma desktop with CentOS-Stream 8? do you have any other config to reproduce it?
I attach screenshot for the CentOS-Stream-8.

Thanks very much!

Lili Du

(In reply to ldu from comment #4)
> Hi Neal，
> I am QE that test RHEL, I installed RHEL 8.5 on VMware Workstation 16, seems
> not have the auto resize issue, the RHEL 8.5 installed with
> open-vm-tools-desktop-11.2.5-2.el8.x86_64 and
> open-vm-tools-11.2.5-2.el8.x86_64, then I installed
> CentOS-Stream-8-x86_64-20210727-dvd1.iso on VMware Workstation 16, also not
> encounter auto resize issue, I use the default Gnome desktop, do you use KDE
> Plasma desktop with CentOS-Stream 8? do you have any other config to
> reproduce it?
> I attach screenshot for the CentOS-Stream-8.
> 
> Thanks very much!
> 
> Lili Du

In GNOME, the only impact is that resolutions are not correctly enumerated from my host to the guest. In Plasma, this is true AND the resizing doesn't work.

You can easily test this with this KDE LiveDVD: https://d2r1r30d0ahkv5.cloudfront.net/CS8-HS-KDE-LIVE-20210724.iso

The SHA512 checksum for the DVD: bafdcb62009f102671e9ff6e410b7cebcfbfc1270ab8b82045edab151d4991f3c347d8b22bf9c71fafb84eb24135dd4a5f7d7b44802f3d55ac575833c4abd5c0

Hi Neal，
Thanks very much for your update!
I tried the KDE LiveDVD, indeed the resizing doesn't work.

@Mirek, could you help backport the patch Neal attached in Commnet 1 ?

Thanks,
Lili Du

It looks like the patch we are talking about is

commit ff5eb5f448c78448b96f9c3db957d19f0288a9b6
Author: John Wolfe <jwolfe>
Date:   Thu Feb 4 09:57:03 2021 -0800

     resolution: Fix kms autodetection

(In reply to Cathy Avery from comment #9)
> It looks like the patch we are talking about is
> 
> commit ff5eb5f448c78448b96f9c3db957d19f0288a9b6
> Author: John Wolfe <jwolfe>
> Date:   Thu Feb 4 09:57:03 2021 -0800
> 
>      resolution: Fix kms autodetection

Yes, this commit is included in the Dist-Git patch I provided in comment 1.

Hi John,

I have back-ported commit ff5eb5f448c78448b96f9c3db957d19f0288a9b6 'resolution: Fix kms autodetection' to RHEL 8.5 which is open-vm-tools 11.2.5.

Is there any reason why this commit might be a problem for 11.2.5?

Thanks,

Cathy

(In reply to Cathy Avery from comment #11)
> Hi John,
> 
> I have back-ported commit ff5eb5f448c78448b96f9c3db957d19f0288a9b6
> 'resolution: Fix kms autodetection' to RHEL 8.5 which is open-vm-tools
> 11.2.5.
> 
> Is there any reason why this commit might be a problem for 11.2.5?
> 
> Thanks,
> 
> Cathy

Hi Cathy,

There should be no problem taking that patch back to open-vm-tools 11.2.5.

This change went into 11.3.0.  It does anticipate a version of vmwgfx driver with DRM version 2.14 or above as stated in the change description which Zack Russin indicates has been upstream since 2017.

Hi Neal,
I have retest with the new open-vm-tools and open-vm-tools-desktop build that contain the patch on centos 8 with KDE desktop, seem not resolve the resize issue, after resize the windows to a small size, the console not auto resize to fit the client window, I attach the screenshot.
Do you want to test with the open-vm-tools package with the patch? I can upload to you on a ftp.

Thanks,
Lili Du

(In reply to ldu from comment #14)
> Hi Neal,
> I have retest with the new open-vm-tools and open-vm-tools-desktop build
> that contain the patch on centos 8 with KDE desktop, seem not resolve the
> resize issue, after resize the windows to a small size, the console not auto
> resize to fit the client window, I attach the screenshot.
> Do you want to test with the open-vm-tools package with the patch? I can
> upload to you on a ftp.
> 

Yes, sure I can test it. However, I don't see any screenshot attached that you mention here...

Created attachment 1811168 [details]
small_size

Created attachment 1811169 [details]
Bigger_size

(In reply to Neal Gompa from comment #17)
> (In reply to ldu from comment #14)
> > Hi Neal,
> > I have retest with the new open-vm-tools and open-vm-tools-desktop build
> > that contain the patch on centos 8 with KDE desktop, seem not resolve the
> > resize issue, after resize the windows to a small size, the console not auto
> > resize to fit the client window, I attach the screenshot.
> > Do you want to test with the open-vm-tools package with the patch? I can
> > upload to you on a ftp.
> > 
> 
> Yes, sure I can test it. However, I don't see any screenshot attached that
> you mention here...

Hi Neal,
Sorry for not unselect the private flag when upload the attachment, I have attach again. 
Also sorry for the ftp have some account problem, I will paste the ftp url when resolve it.

Thanks very much for your patience!

Lili Du

(In reply to ldu from comment #8)
> Hi Neal，
> Thanks very much for your update!
> I tried the KDE LiveDVD, indeed the resizing doesn't work.
> 
> @Mirek, could you help backport the patch Neal attached in Commnet 1 ?
> 
> Thanks,
> Lili Du

Cathy is working on the backport.

I double checked the build and the patch is in the brew build I gave to Lili.

commit ff5eb5f448c78448b96f9c3db957d19f0288a9b6
Author: John Wolfe <jwolfe>
Date:   Thu Feb 4 09:57:03 2021 -0800

     resolution: Fix kms autodetection

Neal has verified this is the patch that fixed the problem in fedora.

If Neal concurs that the brew build does not fix the issue we should try the rhel 9 open-vm-tools 1.3.0 version and go from there.

Hi Cathy, 
I also could reproduce it on Fedora, and indeed after install open-vm-tools 11.3.0 fix the auto resize issue on the fedora.

Could you help share open-vm-tools-11.2.5-2.el8.cavery202108030839.x86_64.rpm build to Neal via people.redhat.com ? Let Neal verified on KDE desktop.
My people.redhat.com account disabled and the password not work. I had send ticket to fix it, but no response yet.

Thanks,
Lili Du

Hi Cathy,

I have asked the video team what version(s) of the vmwgfx driver are required to work with this patch.  I will forward that information as soon as I get it.

Thanks,
John

Hi Lili,

This is a RHEL 9 open-vm-tools 11.3.0 build

https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=38802620

Could you try it on your Centos 8 with KDE Plasma setup? I am assuming that is the setup you used where my previous brew build of 11.2.5 with the patch did not work.

Thanks!

(In reply to Cathy Avery from comment #26)
> Hi Lili,
> 
> This is a RHEL 9 open-vm-tools 11.3.0 build
> 
> https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=38802620
> 
> Could you try it on your Centos 8 with KDE Plasma setup? I am assuming that
> is the setup you used where my previous brew build of 11.2.5 with the patch
> did not work.
> 
> Thanks!

Hi Cathy,
when install the RHEL 9 open-vm-tools 11.3.0 on centos 8 have some dependencies issue, I use rhel 9 repo to resolve it, after install the open-vm-tools-11.3.0-1.el9.cavery202108090947.x86_64.rpm and open-vm-tools-desktop-11.3.0-1.el9.cavery202108090947.x86_64.rpm on Centos 8 with KDE Plasma， the auto resize issue still exist. I attach the screenshot for KDE.

BRs,
Lili Du

Created attachment 1812731 [details]
resize_kde

(In reply to ldu from comment #27)
> (In reply to Cathy Avery from comment #26)
> > Hi Lili,
> > 
> > This is a RHEL 9 open-vm-tools 11.3.0 build
> > 
> > https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=38802620
> > 
> > Could you try it on your Centos 8 with KDE Plasma setup? I am assuming that
> > is the setup you used where my previous brew build of 11.2.5 with the patch
> > did not work.
> > 
> > Thanks!
> 
> Hi Cathy,
> when install the RHEL 9 open-vm-tools 11.3.0 on centos 8 have some
> dependencies issue, I use rhel 9 repo to resolve it, after install the
> open-vm-tools-11.3.0-1.el9.cavery202108090947.x86_64.rpm and
> open-vm-tools-desktop-11.3.0-1.el9.cavery202108090947.x86_64.rpm on Centos 8
> with KDE Plasma， the auto resize issue still exist. I attach the screenshot
> for KDE.
> 

Did you restart after doing that? My understanding is that it only takes effect after reboot.

And also note, you'll need KDE Plasma 5.22 from EPEL 8 Next on CentOS Stream 8 to effectively test this. EPEL 8 base still has Plasma 5.18 and the Wayland session there is quite broken.

The KDE issue is a little different. KDE had (it still might, as I haven't tested the latest kwin code) a bug in the kwin wayland code where it wouldn't correct resize the fb dimensions its rendering to after the resize, i.e. looking at https://www.kernel.org/doc/html/latest/gpu/drm-kms.html the drm_framebuffer objects used by kwin aren't correctly resized (recreated with the new dimensions) after udev sends a change event for the new mode on drm_crtc.

So while vmwgfx has set the new mode, the userspace is still rendering into old dimensioned framebuffer.

(In reply to Zack Rusin from comment #32)
> The KDE issue is a little different. KDE had (it still might, as I haven't
> tested the latest kwin code) a bug in the kwin wayland code where it
> wouldn't correct resize the fb dimensions its rendering to after the resize,
> i.e. looking at https://www.kernel.org/doc/html/latest/gpu/drm-kms.html the
> drm_framebuffer objects used by kwin aren't correctly resized (recreated
> with the new dimensions) after udev sends a change event for the new mode on
> drm_crtc.
> 
> So while vmwgfx has set the new mode, the userspace is still rendering into
> old dimensioned framebuffer.

If that was the case, then it wouldn't work in Fedora either, right? But it works there, and the version of Plasma in Fedora is the same version in EPEL 8 Next.

(In reply to Neal Gompa from comment #33)
> (In reply to Zack Rusin from comment #32)
> > The KDE issue is a little different. KDE had (it still might, as I haven't
> > tested the latest kwin code) a bug in the kwin wayland code where it
> > wouldn't correct resize the fb dimensions its rendering to after the resize,
> > i.e. looking at https://www.kernel.org/doc/html/latest/gpu/drm-kms.html the
> > drm_framebuffer objects used by kwin aren't correctly resized (recreated
> > with the new dimensions) after udev sends a change event for the new mode on
> > drm_crtc.
> > 
> > So while vmwgfx has set the new mode, the userspace is still rendering into
> > old dimensioned framebuffer.
> 
> If that was the case, then it wouldn't work in Fedora either, right? But it
> works there, and the version of Plasma in Fedora is the same version in EPEL
> 8 Next.

Yes, that should be the case (unless the config is running on top of X11, kwin's X11 code works correctly with external resizes).
A good check to see if the new open vm tools are working correctly in the guest is to do (inside the guest):
 $ sudo sh -c "echo 0x2 > /sys/module/drm/parameters/debug"
 $ journalctl -f -k
then resizing the window or setting the fullscreen mode should result in the kernel messages in the journalctl output of the form:
 kernel: [drm:vmw_kms_update_layout_ioctl [vmwgfx]] Layout count = 1
 kernel: [drm:vmw_kms_update_layout_ioctl [vmwgfx]]   x1 = 0 y1 = 0 x2 = 1920 y2 = 1080
(the coordinates will match the screen). If those messages don't show up then the kernel is not getting the set resolution messages from the guest tools and most likely it's running some old version of the resolution plugin.

Hi Neal,Zack,
I retried on Fedora with KDE Plasma(Wayland) desktop, the Fedora 34 installed open-vm-tools-11.3.0, the auto resize also not work(please check the attachment screenshot), that's same issue as centos 8 with KDE plasma, but when change to KDE plasma(X11）on Fedora, the auto resize works well.
I think that's maybe the issue Zack mentioned in comment 32.

Thanks,
Lili Du

Created attachment 1813036 [details]
Fedora34_kde_wayland

Hi Zack,
The journalctl -f -k result when set use KDE palsam wayland, resize the Workstation window size, the x2 y2 size not change follow the window size:

Aug 17 09:27:06 localhost.localdomain kernel: [drm:vmw_kms_update_layout_ioctl [vmwgfx]] Layout count = 1
Aug 17 09:27:06 localhost.localdomain kernel: [drm:vmw_kms_update_layout_ioctl [vmwgfx]]   x1 = 0 y1 = 0 x2 = 954 y2 = 683
Aug 17 09:27:10 localhost.localdomain kernel: [drm:vmw_kms_update_layout_ioctl [vmwgfx]] Layout count = 1
Aug 17 09:27:10 localhost.localdomain kernel: [drm:vmw_kms_update_layout_ioctl [vmwgfx]]   x1 = 0 y1 = 0 x2 = 954 y2 = 683
Aug 17 09:27:18 localhost.localdomain kernel: [drm:vmw_kms_update_layout_ioctl [vmwgfx]] Layout count = 1
Aug 17 09:27:18 localhost.localdomain kernel: [drm:vmw_kms_update_layout_ioctl [vmwgfx]]   x1 = 0 y1 = 0 x2 = 954 y2 = 683
Aug 17 09:27:20 localhost.localdomain kernel: [drm:vmw_kms_update_layout_ioctl [vmwgfx]] Layout count = 1
Aug 17 09:27:20 localhost.localdomain kernel: [drm:vmw_kms_update_layout_ioctl [vmwgfx]]   x1 = 0 y1 = 0 x2 = 954 y2 = 683

when change use the KDE palsam x11, then resize the Workstation window size, the x2 y2 size will change follow the window size.

Due to the tight schedule and the fact that RHEL 8.5 is not experiencing the problem I am deferring this BZ until RHEL 8.6 when we will be rebasing open-vm-tools to 11.3.0. This BZ will now be considered TestOnly.

Lili tells me she will be submitting a new bug to Fedora to track the KDE plasma wayland resize issue separately (comment 32).

Thanks,

Cathy

Hi,
I report a new bug 1994174 to track the KDE palsam wayland resize issue.

BRs,
Lili Du

Thank you. Yes, given that the vmwgfx seems to be getting the update layout callbacks there doesn't seem to be anything we can do. The userspace (in this case Plasma) has to resize its framebuffers. The new open-vm-tools has a utility called "vmwgfxctrl" that can be used to double check such issues as well. 
$ vmwgfxctrl --print-topology
will print the topology (among it the set resolution).
$ sudo vmwgfxctrl --set-topology 1024x768+0+0
will set resolution to 1024x768. That should help with fixing userspace issues related to incorrectly sized fb's.

After run sanity test for RHEL8.6 on VMware Workstation 16 with open-vm-tools-11.3.5, the auto resize works well, also test with vmwgfxctrl --set-topology xxx, then check with vmwgfxctrl --print-topology, all resize works well, so verified this bug.