Bug 2035842

Summary: [RHEL9] Creation of VM failed if not choose Immediately start VM option in cockpit
Product: Red Hat Enterprise Linux 9 Reporter: Kai <wcheng>
Component: cockpit-machinesAssignee: Martin Pitt <mpitt>
Status: CLOSED DUPLICATE QA Contact: Jan Ščotka <jscotka>
Severity: low Docs Contact:
Priority: unspecified    
Version: 9.0CC: coli, virt-maint, yunyang, zixchen
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-02-09 01:27:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kai 2021-12-28 02:29:38 UTC
Description of problem:
RHEL9 Creation of VM failed if not choose Immediately start VM option in cockpit

Version-Release number of selected component (if applicable):
RHEL9 beta

How reproducible:
Login cockpit - create VM without enable "Immediately start VM"

Steps to Reproduce:
1. Login cockpit
2. create VM and remove "Immediately start VM" default option. 

Actual results:
Danger alert: Creation of VM test1 failed 

Expected results:
Workaround: Create VM with select "Immediately start VM" and poweroff it after create for change default network.

Additional info:
... failed VM log
char device redirected to /dev/pts/1 (label charserial0)
2021-12-25T01:24:12.095676Z qemu-kvm: terminating on signal 15 from pid 2169 (<unknown process>)
2021-12-25 01:24:12.296+0000: shutting down, reason=destroyed
...success vm log
char device redirected to /dev/pts/2 (label charserial0)
[last line of the log]

Comment 6 John Ferlan 2022-01-04 17:19:23 UTC
Can you find and attach the XML for the VM?  For "permanent" VM's that'd be /etc/libvirt/qemu/{VM}.xml... You may find something at /var/run/libvirt/qemu/{VM}.* too. 

Makes it easier to reproduce since devs may not use cockpit normally.

Perhaps some more details from /var/log/libvirt/qemu/{VM}.log?  Getting a process dump/trace of why qemu-kvm failed will help too.

I'm a bit rusty on the common debugging/logging needs for libvirt to get qemu data.

Comment 8 Kai 2022-01-14 07:30:23 UTC
Sorry for late. Here enabled debug log for libvritd. See if it help or guide me to collect the right logs. Many thanks.
https://drive.google.com/file/d/1frTLK6CA3I2fu_3AU2zzy8mSDeOTL9iu/view?usp=sharing

Comment 9 zixchen 2022-01-14 09:57:38 UTC
Hello Kai,

I test with rhel9 but I couldn't reproduce the issue, could you help to check my steps?

Version:
cockpit-259-1.el9.x86_64
qemu-kvm-6.2.0-1.el9.x86_64
libvirt-7.10.0-1.el9.x86_64
kernel-5.14.0-39.el9.x86_64


Steps:
1. Login to cockpit
2. Create a virtual network with the default options
Now there are a Storage pool and a Network available.
3. Create a VM also with default options but remove "Immediately start VM"
Installation type tried local install media and URL.
4. New VM is created, click "install" to start the installation

Results:
I created 4 VMs successfully.

Comment 10 Kai 2022-01-18 09:11:45 UTC
Dear Zixi,
Yes, i follow your step and can create VM.
Seem like issue only happen if Install type choose "Download an OS". See if you got same result. Many thanks.
Best Regards,
KAI

Comment 11 zixchen 2022-01-21 08:02:55 UTC
Thanks, KAI. I can reproduce this issue. There are two issues, one is SELinux preventing /usr/bin/ps from sys_ptrace access on the cap_userns labeled ksmtuned_t, the other is after disabling Selinux, still fail to create VM. 

Steps:
1. login to cockpit
2. create a VM with Install type choose "Download an OS" and remove the "Immediately start VM" default option, the other options just use the default value.

Results:
Cockpit reports VM creation failed, check 'journalctl' log, it shows:
setroubleshoot[8961]: SELinux is preventing /usr/bin/ps from sys_ptrace access on the cap_userns labeled ksmtuned_t. For complete SELinux messages>
setroubleshoot[8961]: SELinux is preventing /usr/bin/ps from sys_ptrace access on the cap_userns labeled ksmtuned_t.
                                                                              
                                                                              *****  Plugin catchall (100. confidence) suggests   **************************
                                                                              
                                                                              If you believe that ps should be allowed sys_ptrace access on cap_userns labeled ksmtuned_t by default.
                                                                              Then you should report this as a bug.
                                                                              You can generate a local policy module to allow this access.
                                                                              Do
                                                                              allow this access for now by executing:
                                                                              # ausearch -c 'ps' --raw | audit2allow -M my-ps
                                                                              # semodule -X 300 -i my-ps.pp
There is a fixed Fedora bug reported the same issue, but no bug for rhel8/9. 

If disable SELinux, the VM creation still failed, no log found under /var/log/libvirt/qemu/ directory, nor any log in journalctl, but I can see the image is created in /var/lib/libvirt/images/test.qcow2.

Comment 12 YunmingYang 2022-02-08 10:14:18 UTC
HI KAI, It seems is the same with this issue https://bugzilla.redhat.com/show_bug.cgi?id=2032462. Could you please check whether there is an error like follow in web console(F12 -> Console)
"Traceback (most recent call last):
  File "<stdin>", line 356, in <module>
  File "<stdin>", line 273, in create_vm
  File "<stdin>", line 310, in inject_metadata
  File "/usr/lib64/python3.9/xml/etree/ElementTree.py", line 1347, in XML
    parser.feed(text)
xml.etree.ElementTree.ParseError: junk after document element: line 61, column 0"

Also, could you please add the version of cockpit-machines? Thanks

Comment 13 Kai 2022-02-08 12:15:14 UTC
Dear Yunming,

Yes, same error in browser console.

Traceback (most recent call last):
  File "<stdin>", line 356, in <module>
  File "<stdin>", line 273, in create_vm
  File "<stdin>", line 310, in inject_metadata
  File "/usr/lib64/python3.9/xml/etree/ElementTree.py", line 1347, in XML
    parser.feed(text)
xml.etree.ElementTree.ParseError: junk after document element: line 61, column 0
cockpit.js:1:8157
spawn 'vm creation' returned error: "{"problem":null,"exit_status":1,"exit_signal":null,"message":"Traceback (most recent call last):\n  File \"<stdin>\", line 356, in <module>\n  File \"<stdin>\", line 273, in create_vm\n  File \"<stdin>\", line 310, in inject_metadata\n  File \"/usr/lib64/python3.9/xml/etree/ElementTree.py\", line 1347, in XML\n    parser.feed(text)\nxml.etree.ElementTree.ParseError: junk after document element: line 61, column 0"}"

version:
cockpit-machines-258.1-1.el9.noarch

Best Regards,
KAI

Comment 14 zixchen 2022-02-09 01:27:11 UTC
Thanks, Yunming and Kai, close this bug as a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=2032462.

*** This bug has been marked as a duplicate of bug 2032462 ***