Bug 1334511 - xenstored.service fails during boot
Summary: xenstored.service fails during boot
Keywords:
Status: CLOSED DUPLICATE of bug 1341317
Alias: None
Product: Fedora
Classification: Fedora
Component: xen
Version: 26
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Michael Young
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-05-09 19:47 UTC by Konrad Rzeszutek Wilk
Modified: 2017-04-18 07:24 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-04-18 07:24:21 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Konrad Rzeszutek Wilk 2016-05-09 19:47:51 UTC
Description of problem:

I see this during bootup after installing Xen:

nconsoled - handles loggi...m guest consoles and hypervisor.
xenconsoled: Unable to increase fd limit from {1024, 4096} to {132008, 132008}: (Operation not permitted) - May run out with lot[FAILED] Failed to start The Xen xenstore.
See 'systemctl status xenstored.service' for details.
         Sta[FAILED] Failed to start The Xen xenstore.
See 'systemctl status xenstored.service' for details.
         Sta[FAILED] Failed to start The Xen xenstore.
See 'systemctl status xenstored.service' for details.
         Starting The Xen xenstore...
[FAILED] Failed to start The Xen xenstore.
See 'systemctl status xenstored.service' for details.
[FA[  OK  ] Started GSSAPI Proxy Daemon.
[  OK  ] Reached target NFS client services.
[  OK  ] Reached target Remote File Systems (Pre).
[  OK  ] Reached target Remote File Systems.

Version-Release number of selected component (if applicable):

Fedora-Server-dvd-x86_64-24_Alpha-1.7.iso 

How reproducible:

100%

Steps to Reproduce:
1. Install Fedora 24
2. edit /etc/default/grub to have GRUB_CMDLINE_SAVED="Fedora, with Xen hypervisor"
3.  dnf install xen
4. reboot

Actual results:
nconsoled - handles loggi...m guest consoles and hypervisor.
xenconsoled: Unable to increase fd limit from {1024, 4096} to {132008, 132008}: (Operation not permitted) - May run out with lot[FAILED] Failed to start The Xen xenstore.
See 'systemctl status xenstored.service' for details.
         Sta[FAILED] Failed to start The Xen xenstore.
See 'systemctl status xenstored.service' for details.
         Sta[FAILED] Failed to start The Xen xenstore.
See 'systemctl status xenstored.service' for details.
         Starting The Xen xenstore...
[FAILED] Failed to start The Xen xenstore.
See 'systemctl status xenstored.service' for details.
[FA[  OK  ] Started GSSAPI Proxy Daemon.
[  OK  ] Reached target NFS client services.
[  OK  ] Reached target Remote File Systems (Pre).
[  OK  ] Reached target Remote File Systems.

[root@tst032 ~]# systemctl status xenstored.service
 xenstored.service - The Xen xenstore
   Loaded: loaded (/usr/lib/systemd/system/xenstored.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Mon 2016-05-09 15:25:51 EDT; 1min 11s ago
  Process: 929 ExecStart=/usr/sbin/xenstored --no-fork $XENSTORED_ARGS (code=exited, status=1/FAILURE)
  Process: 925 ExecStartPre=/bin/mkdir -p /var/run/xen (code=exited, status=0/SUCCESS)
  Process: 920 ExecStartPre=/bin/rm -f /var/lib/xenstored/tdb* (code=exited, status=0/SUCCESS)
  Process: 917 ExecStartPre=/bin/grep -q control_d /proc/xen/capabilities (code=exited, status=0/SUCCESS)
 Main PID: 929 (code=exited, status=1/FAILURE)

May 09 15:25:51 tst032.dumpdata.com xenstored[929]: Checking store ...
May 09 15:25:51 tst032.dumpdata.com xenstored[929]: Checking store complete.
May 09 15:25:51 tst032.dumpdata.com xenstored[929]: WARNING: Failed to open connection to gnttab
May 09 15:25:51 tst032.dumpdata.com xenstored[929]: FATAL: Failed to open evtchn device: No such file or directory
May 09 15:25:51 tst032.dumpdata.com systemd[1]: xenstored.service: Main process exited, code=exited, status=1/FAILURE
May 09 15:25:51 tst032.dumpdata.com systemd[1]: Failed to start The Xen xenstore.
May 09 15:25:51 tst032.dumpdata.com systemd[1]: xenstored.service: Unit entered failed state.
May 09 15:25:51 tst032.dumpdata.com systemd[1]: xenstored.service: Failed with result 'exit-code'.
May 09 15:25:51 tst032.dumpdata.com systemd[1]: xenstored.service: Start request repeated too quickly.
May 09 15:25:51 tst032.dumpdata.com systemd[1]: Failed to start The Xen xenstore.

Expected results:

No failures.

Additional info:

The xenstored.service shows to be:

[root@tst032 system]# more xenstored.service 
[Unit]
Description=The Xen xenstore
Requires=xenstored_ro.socket xenstored.socket proc-xen.mount var-lib-xenstored.mount
After=proc-xen.mount var-lib-xenstored.mount
Before=libvirtd.service libvirt-guests.service
RefuseManualStop=true
ConditionPathExists=/proc/xen/capabilities

[Service]
Type=notify
KillMode=none
Sockets=xenstored.socket xenstored_ro.socket
Environment=XENSTORED_ARGS=
EnvironmentFile=-/etc/sysconfig/xencommons
ExecStartPre=/bin/grep -q control_d /proc/xen/capabilities
ExecStartPre=-/bin/rm -f /var/lib/xenstored/tdb*
ExecStartPre=/bin/mkdir -p /var/run/xen
ExecStart=/usr/sbin/xenstored --no-fork $XENSTORED_ARGS
ExecStartPost=-/usr/lib/xen/bin/xen-init-dom0

[Install]
WantedBy=multi-user.target
Also=xenstored_ro.socket xenstored.socket
Also=proc-xen.mount
Also=var-lib-xenstored.mount

[root@tst032 system]# cat /boot/config-4.5.3-300.fc24.x86_64 |grep XEN | grep EVTCHN
CONFIG_XEN_DEV_EVTCHN=m

So
[root@tst032 system]# lsmod | grep evt
[root@tst032 system]# ls -al /dev/xen
total 0
drwxr-xr-x.  2 root root    100 May  9 15:25 .
drwxr-xr-x. 20 root root   3820 May  9 15:25 ..
crw-------.  1 root root 10, 56 May  9 15:25 privcmd
crw-------.  1 root root 10, 62 May  9 15:25 xenbus
crw-------.  1 root root 10, 61 May  9 15:25 xenbus_backend
[root@tst032 system]# modprobe xen-evtchn
[root@tst032 system]# ls -al /dev/xen
total 0
drwxr-xr-x.  2 root root    120 May  9 15:44 .
drwxr-xr-x. 20 root root   3820 May  9 15:25 ..
crw-------.  1 root root 10, 55 May  9 15:44 evtchn
crw-------.  1 root root 10, 56 May  9 15:25 privcmd
crw-------.  1 root root 10, 62 May  9 15:25 xenbus
crw-------.  1 root root 10, 61 May  9 15:25 xenbus_backend

And then starting it again made it work.


Soo how come xen-evtchn is not loaded ?

Comment 1 Michael Young 2016-05-09 20:06:55 UTC
Try with selinux set to permissive mode. It sounds like Bug 1322625 which means modules listed in /usr/lib/modules-load.d aren't loaded on boot. 
Incidentally I seem to remember that the line
xenconsoled: Unable to increase fd limit from {1024, 4096} to {132008, 132008}
is also due to selinux permissions but isn't a problem in most use cases so I haven't got around to trying to get it fixed.

Comment 2 Konrad Rzeszutek Wilk 2016-05-10 14:52:25 UTC
That did it. I booted with selinux=0 and it worked.

Albeit I now I see:
[root@tst032 ~]# systemctl status  systemd-modules-load.service 
● systemd-modules-load.service - Load Kernel Modules
   Loaded: loaded (/usr/lib/systemd/system/systemd-modules-load.service; static; vendor preset: disabled)
   Active: failed (Result: exit-code) since Tue 2016-05-10 10:25:38 EDT; 23min ago
     Docs: man:systemd-modules-load.service(8)
           man:modules-load.d(5)
  Process: 630 ExecStart=/usr/lib/systemd/systemd-modules-load (code=exited, status=1/FAILURE)
 Main PID: 630 (code=exited, status=1/FAILURE)

May 10 10:25:38 tst032.dumpdata.com systemd-modules-load[630]: Failed to find module 'gntdev'
May 10 10:25:38 tst032.dumpdata.com systemd-modules-load[630]: Failed to find module 'netbk'
May 10 10:25:38 tst032.dumpdata.com systemd-modules-load[630]: Failed to find module 'blkbk'
May 10 10:25:38 tst032.dumpdata.com systemd-modules-load[630]: Failed to find module 'xen-scsibk'
May 10 10:25:38 tst032.dumpdata.com systemd-modules-load[630]: Failed to find module 'usbbk'
May 10 10:25:38 tst032.dumpdata.com systemd-modules-load[630]: Failed to find module 'pciback'
May 10 10:25:38 tst032.dumpdata.com systemd[1]: systemd-modules-load.service: Main process exited, code=exited, status=1/FAILURE
May 10 10:25:38 tst032.dumpdata.com systemd[1]: Failed to start Load Kernel Modules.
May 10 10:25:38 tst032.dumpdata.com systemd[1]: systemd-modules-load.service: Unit entered failed state.
May 10 10:25:38 tst032.dumpdata.com systemd[1]: systemd-modules-load.service: Failed with result 'exit-code'.

Any reason to have the "old" names in there?

Comment 3 Konrad Rzeszutek Wilk 2016-05-20 20:33:50 UTC
Would it make sense to modify the xen.spec to have:
--with-linux-backend-modules="xen-acpi-processor xen-evtchn xen-gntdev xen-gntalloc xen-blkback xen-netback xenfs xen_privcmd xen-scsiback"

so that the other ones (gnttab, etc) are not tried ?

Thanks.

Comment 4 Konrad Rzeszutek Wilk 2016-06-29 14:02:59 UTC
ping?

Comment 5 Robin Lee 2017-04-18 07:24:21 UTC

*** This bug has been marked as a duplicate of bug 1341317 ***


Note You need to log in before you can comment on or make changes to this bug.