Bug 806107

Doing the magic 'setenforce 0' and then starting xenstored makes it work. I still can't see the /proc/xen/xsd_port ioctl file thought.

Upstream discussion with patch:
https://lkml.org/lkml/2012/3/22/303

Changing component to "kernel".

I'm confused, bfcfaa77bdf0f775263e906015982a608df01c76 wasn't in 3.3
This shouldn't affect F17.

Sorry, I may have been confused by the coinciding symptoms.

Konrad, what do you think? Thanks.

Note: if you open the discussion in comment 2, it says:

"v3.3 with just that git commit"

and the kernel config file Konrad posted begins as:

#
# Automatically generated file; DO NOT EDIT.
# Linux/x86_64 3.3.0 Kernel Configuration
#

Thus what may have happened is: 3.3 was taken, upstream bfcfaa77 applied on it, the RPM rebuilt and used, and when it broke, the BZ was reported against 3.3 (not "3.3 + bfcfaa77"). In other words, this may be NOTABUG.

Konrad, please confirm. Thanks!

I can confirm the missing files really are missing with a regular F17 kernel running dom0.

I was also playing with my customised kernel on F16 (built to test a fix for Bug 804347 ) booting as a dom0 but without any xen processes, and I found by mounting and unmounting /proc/xen I could get either 3, 4 or 5 files in /proc/xen so it might be a periodic bug.

(In reply to comment #5)
> Note: if you open the discussion in comment 2, it says:
> 
> "v3.3 with just that git commit"
> 
> and the kernel config file Konrad posted begins as:
> 
> #
> # Automatically generated file; DO NOT EDIT.
> # Linux/x86_64 3.3.0 Kernel Configuration
> #
> 
> Thus what may have happened is: 3.3 was taken, upstream bfcfaa77 applied on it,
> the RPM rebuilt and used, and when it broke, the BZ was reported against 3.3
> (not "3.3 + bfcfaa77"). In other words, this may be NOTABUG.
> 
> Konrad, please confirm. Thanks!

Correct. It was the 3.3 post (or also known as 3.4-rc0)

Let me change the title back.

So to either clarify or muddy the waters - the issue I am seeing is against 3.3.0-1.fc17.i686.PAE. Which is a 3.3 kernel (right?), not a 3.4-rc0 type?

(In reply to comment #9)
> So to either clarify or muddy the waters - the issue I am seeing is against
> 3.3.0-1.fc17.i686.PAE. Which is a 3.3 kernel (right?), not a 3.4-rc0 type?

Correct.  The 3.4-rc0 type kernels will have 3.4.0-0.rcX.gitX.0.fc18 naming, and aren't in F17.

So this bug is really against rawhide from what I can tell.  What I can't tell is what the bug is really asking...

I'm guessing this is fixed by upstream commit:

commit f132c5be05e407a99cf582347a2ae0120acd3ad7
Author: Al Viro <viro.org.uk>
Date:   Thu Mar 22 21:59:52 2012 +0000

    Fix full_name_hash() behaviour when length is a multiple of 8
    
    We want it to match what hash_name() is doing, which means extra
    multiply by 9 in this case...
    
    Reported-and-Tested-by: Konrad Rzeszutek Wilk <konrad.wilk>
    Signed-off-by: Al Viro <viro.org.uk>
    Signed-off-by: Linus Torvalds <torvalds>

which means it should get rolled into rawhide with the next kernel build.

OK, except as Dave pointed out neither of those commits are in F17 3.3.0-1.fc17.

Konrad, if xenstored works on a stock Fedora 3.3.0-1.fc17 kernel with setenforce=0, is there something in the audit log before that?  Also, you said both of the files were needed for it to work but you only see one file with setenforce=0 but xenstored "works".

Confused.

The selinux problems could be a separate issue - I have just discovered that the permissions on the /var/lib/xenstored mount changed between F16 and F17 which was enough to stop guests booting with selinux in enforcing mode. It is fixed in the upcoming version of xen so it is probably worth retesting with the new version, though I can't see how this would relate to the missing files in /proc/xen .

I think the /proc/xen not seeing is b/c I am sudo-ing?

Either way, more than happy to test any new version - should I use xen-4.1.2-14.fc17 ?

So with that version I saw a missing permission file:

[root@tst013 konrad]# systemctl status xendomains.service
xendomains.service - Xendomains - start and stop guests on boot and shutdown
	  Loaded: loaded (/usr/lib/systemd/system/xendomains.service; enabled)
	  Active: failed (Result: exit-code) since Tue, 03 Apr 2012 17:15:49 -0400; 44s ago
	 Process: 1572 ExecStart=/usr/libexec/xendomains start (code=exited, status=203/EXEC)
	 Process: 1570 ExecStartPre=/bin/grep -q control_d /proc/xen/capabilities (code=exited, status=0/SUCCESS)
	  CGroup: name=systemd:/system/xendomains.service

Apr 03 17:15:49 tst013.dumpdata.com (ndomains)[1572]: Failed at step EXEC spawning /usr/libexec/xendomains: Permission denied

[root@tst013 konrad]#  /usr/libexec/xendomains 
bash: /usr/libexec/xendomains: Permission denied
[root@tst013 konrad]# chmod 755 /usr/libexec/xendomains 
[root@tst013 konrad]# systemctl start xendomains.service
[root@tst013 konrad]# xm list
Error: Unable to connect to xend: No such file or directory. Is xend running?
[root@tst013 konrad]# systemctl status xendomains.service
xendomains.service - Xendomains - start and stop guests on boot and shutdown
	  Loaded: loaded (/usr/lib/systemd/system/xendomains.service; enabled)
	  Active: active (exited) since Tue, 03 Apr 2012 17:17:26 -0400; 30s ago
	 Process: 1619 ExecStart=/usr/libexec/xendomains start (code=exited, status=0/SUCCESS)
	 Process: 1617 ExecStartPre=/bin/grep -q control_d /proc/xen/capabilities (code=exited, status=0/SUCCESS)
	  CGroup: name=systemd:/system/xendomains.service

and that the "xenconsoled.service" isn't started by default.

But I can't actually find anywhere the serice that would start xend?

In F17 I see:
root@tst013 konrad]# systemctl --all | grep xen
proc-xen.mount            loaded active   mounted       Mount /proc/xen files
var-lib-xenstored.mount   loaded active   mounted       mount xenstore file system
xenconsoled.service       loaded active   running       Xenconsoled - handles logging from guest consoles and hypervisor
xendomains.service        loaded active   exited        Xendomains - start and stop guests on boot and shutdown
xenstored.service         loaded active   running       Xenstored - daemon managing xenstore file system

While in F16 there was also the xend.service?

(In reply to comment #14)
> I think the /proc/xen not seeing is b/c I am sudo-ing?
> 
> Either way, more than happy to test any new version - should I use
> xen-4.1.2-14.fc17 ?

Yes, xen-4.1.2-14.fc17 fixes the permissions on /var/lib/xenstored though there may be other selinux issues. xend can be started in the normal systemd way, ie. systemctl start xend.service (systemctl enable xend.service will start it on boot). I believe service xend start still works as well.

Ah, hadn't realized that it was _not_ enabled by default. So this is what I get:
systemctl enable xend.service
ln -s '/usr/lib/systemd/system/xend.service' '/etc/systemd/system/multi-user.target.wants/xend.service'
[root@tst013 konrad]# systemctl start xend.service
Job failed. See system journal and 'systemctl status' for details.
[root@tst013 konrad]# systemctl status
Too few arguments.
[root@tst013 konrad]# systemctl status xend.service
xend.service - Xend - interface between hypervisor and some applications
	  Loaded: loaded (/usr/lib/systemd/system/xend.service; enabled)
	  Active: failed (Result: exit-code) since Wed, 04 Apr 2012 13:29:28 -0400; 10s ago
	 Process: 1552 ExecStart=/usr/sbin/xend (code=exited, status=1/FAILURE)
	 Process: 1550 ExecStartPre=/bin/grep -q control_d /proc/xen/capabilities (code=exited, status=0/SUCCESS)
	  CGroup: name=systemd:/system/xend.service

Apr 04 13:29:28 tst013.dumpdata.com xend[1552]: Traceback (most recent call last):
Apr 04 13:29:28 tst013.dumpdata.com xend[1552]: File "/usr/lib/python2.7/site.py", line 563, in <module>
Apr 04 13:29:28 tst013.dumpdata.com xend[1552]: main()



Amd the graphical tool tells me:
SELinux is preventing xend from read access on the file /etc/passwd.

which is odd. Why would Xend want to touch /etc/passwd?

Apr  4 13:29:19 tst013 systemd[1]: Reloading.
Apr  4 13:29:28 tst013 xend[1552]: Traceback (most recent call last):
Apr  4 13:29:28 tst013 xend[1552]: File "/usr/lib/python2.7/site.py", line 563, in <module>
Apr  4 13:29:28 tst013 xend[1552]: main()
Apr  4 13:29:28 tst013 xend[1552]: File "/usr/lib/python2.7/site.py", line 545, in main
Apr  4 13:29:28 tst013 xend[1552]: known_paths = addusersitepackages(known_paths)
Apr  4 13:29:28 tst013 xend[1552]: File "/usr/lib/python2.7/site.py", line 278, in addusersitepackages
Apr  4 13:29:28 tst013 xend[1552]: user_site = getusersitepackages()
Apr  4 13:29:28 tst013 xend[1552]: File "/usr/lib/python2.7/site.py", line 253, in getusersitepackages
Apr  4 13:29:28 tst013 xend[1552]: user_base = getuserbase() # this will also set USER_BASE
Apr  4 13:29:28 tst013 xend[1552]: File "/usr/lib/python2.7/site.py", line 243, in getuserbase
Apr  4 13:29:28 tst013 xend[1552]: USER_BASE = get_config_var('userbase')
Apr  4 13:29:28 tst013 xend[1552]: File "/usr/lib/python2.7/sysconfig.py", line 520, in get_config_var
Apr  4 13:29:28 tst013 xend[1552]: return get_config_vars().get(name)
Apr  4 13:29:28 tst013 xend[1552]: File "/usr/lib/python2.7/sysconfig.py", line 424, in get_config_vars
Apr  4 13:29:28 tst013 xend[1552]: _CONFIG_VARS['userbase'] = _getuserbase()
Apr  4 13:29:28 tst013 xend[1552]: File "/usr/lib/python2.7/sysconfig.py", line 182, in _getuserbase
Apr  4 13:29:28 tst013 xend[1552]: return env_base if env_base else joinuser("~", ".local")
Apr  4 13:29:28 tst013 xend[1552]: File "/usr/lib/python2.7/sysconfig.py", line 169, in joinuser
Apr  4 13:29:28 tst013 xend[1552]: return os.path.expanduser(os.path.join(*args))
Apr  4 13:29:28 tst013 xend[1552]: File "/usr/lib/python2.7/posixpath.py", line 260, in expanduser
Apr  4 13:29:28 tst013 xend[1552]: userhome = pwd.getpwuid(os.getuid()).pw_dir
Apr  4 13:29:28 tst013 xend[1552]: KeyError: 'getpwuid(): uid not found: 0'
Apr  4 13:29:28 tst013 systemd[1]: xend.service: control process exited, code=exited status=1
Apr  4 13:29:28 tst013 systemd[1]: Unit xend.service entered failed state.

which is all basic python related.

There is code in xend to run as a different user, but I suspect it would take a lot of work to actually use it. I suspect bits of this code are attempting to do uid related operations and thus trying to look at the passwd file.

To complicate things, if libvirtd is started when xend it isn't there you get a xend status process left lying around which systemd doesn't know about and that gets in the way if you try to start xend.

Related:  http://www.fedoraforum.org/forum/showthread.php?t=280766

Does this mean Xen is non-functional in F17  =  showstopper.

.

(In reply to comment #20)
> Related:  http://www.fedoraforum.org/forum/showthread.php?t=280766
> 
> Does this mean Xen is non-functional in F17  =  showstopper.
> 
> .

F17 already shipped, so obviously not a showstopper.

Konrad, Michael, now that F17 is on the 3.4.0 (really 3.4.1) kernel, are you still seeing the issues?  I seems that the kernel wasn't really the culprit anyway, and that a newer xen package was needed?

I haven't noticed any missing files in /proc/xen recently, but I haven't been looking for them so I might have missed them. I haven't had any problems using xen.

On the xend front I worked out why it wasn't starting via systemd when selinux was enabled - if python doesn't have the HOME environment variable set it tries to wrok out what it should be by looking in /etc/passwd before it starts parsing any of the code. This should be fixed in xen-4.1.2-19.fc17/18 .

I just tested xen-4.1.2-19.fc17 and it does seem to fix this problem. I was able to run my Xen/Dom0 stack with SELinux enforcing the targeted policy.

Moving to xen.  I think this can be closed now.

Now not seeing any Xen package listed for F17:

# yum list available xen
Error: No matching Packages to list

This should be fixed in xen-4.1.2-20.fc17 or later (xen-4.1.2-19.fc17 was obsoleted before it was released as an update).

(In reply to comment #25)
> Now not seeing any Xen package listed for F17:
> 
> # yum list available xen
> Error: No matching Packages to list

I assume that was just a repository glitch. It worked for me when I tried it just now.