Bug 2016601 - libvirtd.service does not start on Fedora 35
Summary: libvirtd.service does not start on Fedora 35
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: cockpit-machines
Version: 35
Hardware: All
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Katerina Koukiou
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 1980027
TreeView+ depends on / blocked
 
Reported: 2021-10-22 08:16 UTC by bengt
Modified: 2021-10-25 20:44 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-10-25 20:24:03 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description bengt 2021-10-22 08:16:45 UTC
Description of problem:
New install Fedora 35. Install cockpit-machines with default dependencies. When you navigate to cockpit-machines you get message "Virtualization service (libvirt) is not active" when you try to press start libvirtd, nothing happen. If you go to command line you can not start libvirtd manual.

All this is related to:
https://fedoraproject.org/wiki/Changes/LibvirtModularDaemons

If you upgrade from previous version, all is working as before. If you follow instruction about "Switching to modular daemons" it will break cockpit-machines.

Version-Release number of selected component (if applicable):
cockpit-machines-252-1.fc35

How reproducible:

Steps to Reproduce:
1. Fresh install 35
2. Install cockpit-machines
3. Navigate til cockpit machines

or
1. Upgrade to Fedora 35
2. Folow instructions for libvirtd "Switching to modular daemons"
3. Navigate to cockpit machines

Actual results:
Page go back to "Virtualization service (libvirt) is not active"

Expected results:
Page will show running virtual machines.
(virsh list --all show virtual machines)


Additional info:
This is related to https://fedoraproject.org/wiki/Changes/LibvirtModularDaemons

Comment 1 Martin Pitt 2021-10-25 06:10:44 UTC
I am starting from a Fedora 35 cloud image with `rpm -qa|grep libvirt` being empty. `dnf install cockpit-machines` pulls in 301 packages, amongst them these libvirt packages:

libvirt-client                             
libvirt-daemon                             
libvirt-daemon-driver-interface            
libvirt-daemon-driver-network              
libvirt-daemon-driver-nodedev              
libvirt-daemon-driver-nwfilter             
libvirt-daemon-driver-qemu                 
libvirt-daemon-driver-secret               
libvirt-daemon-driver-storage              
libvirt-daemon-driver-storage-core         
libvirt-daemon-driver-storage-disk         
libvirt-daemon-driver-storage-gluster      
libvirt-daemon-driver-storage-iscsi        
libvirt-daemon-driver-storage-iscsi-direct 
libvirt-daemon-driver-storage-logical      
libvirt-daemon-driver-storage-mpath        
libvirt-daemon-driver-storage-rbd          
libvirt-daemon-driver-storage-scsi         
libvirt-daemon-driver-storage-sheepdog     
libvirt-daemon-driver-storage-zfs          
libvirt-daemon-kvm                         
libvirt-dbus                               
libvirt-glib                               
libvirt-libs                               

After that, cockpit-machines indeed shows the "Virtualization service (libvirt) is not active" message, and the CLI fails as well:

# virsh list
error: failed to connect to the hypervisor
error: Failed to connect socket to '/var/run/libvirt/virtqemud-sock': Connection refused

# systemctl list-units --no-legend 'virt*'
  virtlockd.socket    loaded active listening Virtual machine lock manager socket
  virtlogd.socket     loaded active listening Virtual machine log manager socket
  virtnetworkd.socket loaded active listening Libvirt network local socket

It works better as user, `virsh list` succeeds, and it starts virtqemud (and a few others).


However, when I click the "Start" button in c-machines, I get the main view, and `systemctl status libvirtd` shows that it is running. That doesn't work for you, and that seems to correspond to your "If you go to command line you can not start libvirtd manual.".

What does "systemctl show libvirtd libvirtd.socket" say for you after trying to start it?

Comment 2 Daniel Berrangé 2021-10-25 08:43:33 UTC
(In reply to Martin Pitt from comment #1)
> I am starting from a Fedora 35 cloud image with `rpm -qa|grep libvirt` being
> empty. `dnf install cockpit-machines` pulls in 301 packages, amongst them
> these libvirt packages:

snip

> After that, cockpit-machines indeed shows the "Virtualization service
> (libvirt) is not active" message, and the CLI fails as well:
> 
> # virsh list
> error: failed to connect to the hypervisor
> error: Failed to connect socket to '/var/run/libvirt/virtqemud-sock':
> Connection refused
> 
> # systemctl list-units --no-legend 'virt*'
>   virtlockd.socket    loaded active listening Virtual machine lock manager
> socket
>   virtlogd.socket     loaded active listening Virtual machine log manager
> socket
>   virtnetworkd.socket loaded active listening Libvirt network local socket

Did you restart the VM after running dnf install ?

RPM packages cannot start the systemd units in their %post script, they can only register that they exist. 

The Fedora systemd preset policy sets virtqemud.service to be enabled by default, and if I reboot the cloud image after installing the packages, this works.

Comment 3 bengt 2021-10-25 12:50:44 UTC
I see my description was not clear.

The issue here is cockpit-machines. Looks like it do not support "Libvirt Modular Daemons".

virsh, virt-install and autostart of vm on boot works fine, but cockpit-machines will look if libvirtd.service is running. In Fedora 35 libvirtd.service is not supposed to run because modular daemons is used. You can revert back to monolithic daemon and cockpit-machines will work again.

cockpit-machines need to change to support modular daemons to work on Fedora 35. Is it to early to use modular daemons in Fedora as default? Or can cockpit-machines rpm revert back to monolithic daemon on install? Maybe documetation of how to revert until cockpit-machines supports modular daemons.

Comment 4 Daniel Berrangé 2021-10-25 13:03:52 UTC
This was addressed upstream in:

commit b59da0d7b1d0dedefab7f0d6ab6678aa7547386a
Author: Katerina Koukiou <kkoukiou>
Date:   Mon Sep 20 13:14:49 2021 +0200

    Stop trying to guess the name of the libvirt service/socket
    
    And leave the service activation to libvirt itself.
    
    In all our supported distros the libvirt service is socket activated.
    The socket is running by default, from the very first moment the user
    installs the package. A stopped libvirt socket means that the user
    manually changed the default state, and should be left alone to do so.
    
    In addition to that, the libvirt daemon changed recently to a modular
    achitecture, so instead of a single monolithic libvirtd daemon we now
    get one for each driver (domain, storage, network, interfaces etc). Some distros already
    opted in the new modular daemons, (see https://www.mail-archive.com/libvir-list@redhat.com/msg219931.html)
    but many keep using the old monolithic libvirtd daemon.
    
    As now we will have many sockets to check for their state, we decided to get
    rid of any libvirt service/socket name detection logic from
    cockpit-machines.
    This logic was previously used to detect if the service was running
    and if not we showed the user an empty state screen.
    That also helped us to listen to socket/service changes and update
    the UI accordingly.
    
    This commit replaces this logic will a simple API call in the initial
    render of this app which allows us to check if a connection type is
    available; if that fails for any of the system/session connections, that means that the libvirt service is not available for the connection.
    
    For the same reason of the name detection logic being problematic, let's
    stop providing to the cockpit-machines users the ability to start/enable
    the libvirt service/socket from the cockpit-machines UI directly.
    
    This now allows us to make cockpit-machines work on fedora-35.


which is part of cockpit-machines >= 253 iiuc, so your version is just slightly out of date.

Comment 5 Martin Pitt 2021-10-25 13:25:28 UTC
Right, current version is 254, which is supposed to address that: https://bodhi.fedoraproject.org/updates/FEDORA-2021-7ccf08dbe0

This just assumes that the OS will handle it, as everything should be socket activated. Fedora packages not starting enabled services upon installation is slightly irritating, Daniel already explained that.

The main thing that tipped me off was your "If you go to command line you can not start libvirtd manual.". That is the bit that I can't reproduce/explain, and need more information about (it's also not directly related to cockpit-machines, but I'm interested anyway)

Comment 6 bengt 2021-10-25 20:24:03 UTC
Upgraded to cockpit-machines-254-1.fc35.noarch.

It was strange because all other cockpit was 254:
cockpit.x86_64                                    254-1.fc35                           @fedora  
cockpit-bridge.x86_64                             254-1.fc35                           @fedora  
cockpit-machines.noarch                           252-1.fc35                           @fedora  
cockpit-networkmanager.noarch                     254-1.fc35                           @fedora  
cockpit-packagekit.noarch                         254-1.fc35                           @fedora  
cockpit-pcp.x86_64                                254-1.fc35                           @fedora  
cockpit-selinux.noarch                            254-1.fc35                           @fedora  
cockpit-storaged.noarch                           254-1.fc35                           @fedora  
cockpit-system.noarch                             254-1.fc35                           @fedora  
cockpit-ws.x86_64                                 254-1.fc35                           @fedora

Upgraded to 254 from koji.

Comment 7 bengt 2021-10-25 20:27:04 UTC
All working as expected with cockpit-machines-254-1.fc35.noarch.

Comment 8 Martin Pitt 2021-10-25 20:44:43 UTC
Thanks for confirming! Note that cockpit-machines is a separate upstream project/Fedora source package, so its version number does not align with cockpit's. (Same with cockpit-{podman,ostree,subscription-manager,session-recording})


Note You need to log in before you can comment on or make changes to this bug.