Created attachment 1595114 [details] vdsm log from host showing the error Description of problem: When trying to start any VM customer is receiving a VNC error complaining about VNC and SASL. Version-Release number of selected component (if applicable): RHV 4.3.4 How reproducible: Every time Steps to Reproduce: 1. Create a VM and try to start it 2. Fails to start reporting a VNC error 3. Change VM to headless and it starts Actual results: Starting any VM reports: 2019-07-26 10:10:49,658-0500 WARN (jsonrpc/6) [vds] VNC not secure: passwdValidTo empty or missing and SASL not configured (graphics:342) 2019-07-26 10:10:49,658-0500 ERROR (jsonrpc/6) [vds] VNC does not seem to be secure: {u'xml': u'<?xml version="1.0" encoding="UTF-8"?><domain type="kvm" xmlns:ovirt-tune="http://ovirt.org/vm/tune/1.0" xmlns:ovirt-vm="http://ovirt.org/vm/1.0"><name>gvllx977</name><uuid>ea2bb5a7-9d5f-43dd-8c31-407588a754b7</uuid><memory>2097152</memory><currentMemory>2097152</currentMemory><iothreads>1</iothreads><maxMemory slots="16">8388608</maxMemory><vcpu current="2">16</vcpu><clock offset="variable" adjustment="0"><timer name="rtc" tickpolicy="catchup"></timer><timer name="pit" tickpolicy="delay"></timer></clock><cpu mode="host-model"><model>power9</model><topology cores="1" threads="1" sockets="16"></topology><numa><cell id="0" cpus="0,1" memory="2097152"></cell></numa></cpu><cputune></cputune><devices><input type="tablet" bus="usb"></input><channel type="unix"><target type="virtio" name="ovirt-guest-agent.0"></target><source mode="bind" path="/var/lib/libvirt/qemu/channels/ea2bb5a7-9d5f-43dd-8c31-407588a754b7.ovirt-guest-agent.0"></source></channel><channel type="unix"><target type="virtio" name="org.qemu.guest_agent.0"></target><source mode="bind" path="/var/lib/libvirt/qemu/channels/ea2bb5a7-9d5f-43dd-8c31-407588a754b7.org.qemu.guest_agent.0"></source></channel><emulator text="/usr/bin/qemu-system-ppc64"></emulator><video><model type="vga" vram="16384" heads="1"></model><alias name="ua-1d1801a0-1d63-4fdb-a361-5a1784855d82"></alias></video><graphics type="vnc" port="-1" autoport="yes" keymap="en-us"><listen type="network" network="vdsm-ovirtmgmt"></listen></graphics><controller type="scsi" model="ibmvscsi" index="0"><address type="spapr-vio"></address></controller><controller type="usb" model="nec-xhci" index="0"><address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci"></address></controller><rng model="virtio"><backend model="random">/dev/urandom</backend><alias name="ua-980baf4c-398e-4ac0-b08b-a43b678dccfc"></alias></rng><controller type="scsi" model="virtio-scsi" index="1"><driver iothread="1"></driver><alias name="ua-a878db22-aa8a-4fbb-af8b-5da205a15e99"></alias></controller><controller type="virtio-serial" index="0" ports="16"><alias name="ua-aa3099a3-5a96-4cac-83da-3500b6e3280e"></alias><address bus="0x00" domain="0x0000" function="0x0" slot="0x04" type="pci"></address></controller><memballoon model="virtio"><stats period="5"></stats><alias name="ua-ebc7c455-ba81-49d2-97c6-cad58f646fd8"></alias><address bus="0x00" domain="0x0000" function="0x0" slot="0x05" type="pci"></address></memballoon><interface type="bridge"><model type="virtio"></model><link state="up"></link><source bridge="prod_2033"></source><driver queues="2" name="vhost"></driver><alias name="ua-be4c6645-73ac-4478-afc8-0aa4d51f20eb"></alias><address bus="0x00" domain="0x0000" function="0x0" slot="0x01" type="pci"></address><mac address="56:6f:c1:e3:00:03"></mac><mtu size="9000"></mtu><filterref filter="vdsm-no-mac-spoofing"></filterref><bandwidth></bandwidth></interface><disk type="file" device="cdrom" snapshot="no"><driver name="qemu" type="raw" error_policy="report"></driver><source file="" startupPolicy="optional"><seclabel model="dac" type="none" relabel="no"></seclabel></source><target dev="sdc" bus="scsi"></target><readonly></readonly><alias name="ua-b923d219-3fe4-4bfb-8297-f099183c164f"></alias><address bus="0" controller="0" unit="2" type="drive" target="0"></address></disk><disk snapshot="no" type="block" device="disk"><target dev="sda" bus="scsi"></target><source dev="/rhev/data-center/mnt/blockSD/37d347f4-9803-4823-af71-c729c50ab1d5/images/938e9318-2c61-4dae-bd1f-2af1a0653294/53734d29-6052-400e-8b80-cebab865822f"><seclabel model="dac" type="none" relabel="no"></seclabel></source><driver name="qemu" io="native" type="raw" error_policy="stop" cache="none"></driver><alias name="ua-938e9318-2c61-4dae-bd1f-2af1a0653294"></alias><address bus="0" controller="1" unit="0" type="drive" target="0"></address><boot order="1"></boot><serial>938e9318-2c61-4dae-bd1f-2af1a0653294</serial></disk></devices><os><type arch="ppc64" machine="pseries-rhel7.6.0-sxxm">hvm</type></os><metadata><ovirt-tune:qos></ovirt-tune:qos><ovirt-vm:vm><ovirt-vm:minGuaranteedMemoryMb type="int">2048</ovirt-vm:minGuaranteedMemoryMb><ovirt-vm:clusterVersion>4.3</ovirt-vm:clusterVersion><ovirt-vm:custom></ovirt-vm:custom><ovirt-vm:device mac_address="56:6f:c1:e3:00:03"><ovirt-vm:custom></ovirt-vm:custom></ovirt-vm:device><ovirt-vm:device devtype="disk" name="sda"><ovirt-vm:poolID>a4bf8ca6-aa41-11e9-9c3f-001a4a16017c</ovirt-vm:poolID><ovirt-vm:volumeID>53734d29-6052-400e-8b80-cebab865822f</ovirt-vm:volumeID><ovirt-vm:imageID>938e9318-2c61-4dae-bd1f-2af1a0653294</ovirt-vm:imageID><ovirt-vm:domainID>37d347f4-9803-4823-af71-c729c50ab1d5</ovirt-vm:domainID></ovirt-vm:device><ovirt-vm:launchPaused>false</ovirt-vm:launchPaused><ovirt-vm:resumeBehavior>auto_resume</ovirt-vm:resumeBehavior></ovirt-vm:vm></metadata></domain>', 'vmId': None} (API:231) 2019-07-26 10:10:49,659-0500 ERROR (jsonrpc/6) [api] FINISH create error=Error creating the requested VM (api:131) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 124, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 234, in create "VNC is allowed to start VNC with no password auth and " CannotCreateVM: Error creating the requested VM 2019-07-26 10:10:49,659-0500 INFO (jsonrpc/6) [api.virt] FINISH create return={'status': {'message': 'Error creating the requested VM', 'code': 9}} from=::ffff:172.22.32.225,38762, flow_id=5f03f29b-6d26-4f1b-a969-b48c025cbb9d, vmId= (api:54) Expected results: Should be able to start VMs Additional info: According to the customer they originally tried to setup the hosts to have FIPS enabled. They could not get this to work correctly so the reinstalled all the hosts. I check dmesg to verify that fips was disabled in the host. I also checked that SASL was disabled in /etc/libvirt/qemu.conf so I am sure why VNC is trying to use SASL?
Tomas, any thoughts? I doubt if this behaves differently on POWER
it's probably still enabled/request in configuration to use secured VNC. But if they reinstalled hosts then it's no longer possible to run secured VNC. It needs to be disabled in webadmin as well in Cluster->Console setting The full error message is "A VM is not secure: VNC has no password and SASL authentication not configured. On hosts in FIPS mode VNC must use SASL." Toams, please improve the logging, it seems the message gets cut off when logged
(In reply to Michal Skrivanek from comment #3) > Toams, please improve the logging, it seems the message gets cut off when > logged *Tomas, sorry:) - and please also check the tooltip in Enable VNC Encryption setting, seems empty to me.
Yes, Michal is right, this looks like a misconfiguration. Also, for FIPS+Encypted VNC, SASL need to be correctly configured, preferably with the provided ansible role ('ovirt-host-setup-vnc-sasl', example playbook is 'ovirt-vnc-sasl.yml'). Putting it somewhere in the documentation is not the most user-friendly way to make users aware. Should the hint about it be included in the error message, maybe?
There is also a real bug on POWER. The kernel cmdline is not editable at all beacause originally it was only for hostdev passthrough not relevant on POWER. But now the FIPS and NOSMT should be available, all the others should still be disabled. Also the detection is probably wrong...for the disabled ones we probably should not initialize the json fields. example: {"current":"fips=0 nosmt","parsable":false,"blacklistNouveau":true,"iommu":true,"kvmNested":true,"unsafeInterrupts":true,"pciRealloc":true,"fips":true,"smtDisabled":true} which shows up in UI as checked greyed out. Problem is that they are still interpreted - fips config is being sent.
actually, we will need a doc text for the actual fix of wrongly grayed out options on POWER
sync2jira
The documentation text flag should only be set after 'doc text' field is provided. Please provide the documentation text and set the flag to '?' again.
WARN: Bug status (ON_QA) wasn't changed but the folowing should be fixed: [Found non-acked flags: '{}', ] For more info please contact: rhv-devops: Bug status (ON_QA) wasn't changed but the folowing should be fixed: [Found non-acked flags: '{}', ] For more info please contact: rhv-devops
Verified with: - RHV 4.4.0-0.26.master.el8ev - vdsm-4.40.7-1.el8ev Verification steps: 1. Add a POWER9 host (PowerNV 9006-12P) to a PPC64 cluster - make sure the PIPS checkbox (under kernel tab) is un-checked. 2. Reboot the host and make sure PIPS is disabled Result: - The host was added successfully and PIPS remain disabled after reboot. - The checkboxes for FIPS & SMT are available. - The tooltip for VNC encryption is updated with "Enabling VNC Encryption will enforce VNC communication over TLS (using X509Vnc VeNCrypt)".
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: RHV Manager (ovirt-engine) 4.4 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:3247