Bug 845980 - vsftpd seccomp filter is too strict
vsftpd seccomp filter is too strict
Status: CLOSED CURRENTRELEASE
Product: Fedora
Classification: Fedora
Component: vsftpd (Show other bugs)
19
x86_64 Linux
unspecified Severity urgent
: ---
: ---
Assigned To: Jiri Skala
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-08-06 07:35 EDT by James Collins
Modified: 2016-05-30 07:36 EDT (History)
30 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-12-03 03:39:12 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description James Collins 2012-08-06 07:35:46 EDT
Description of problem:

After the Kernel upgrade to 3.5.0-2.fc17.x86_64 from 3.4.6-2.fc17.x86_64 vsftpd-3.0.0-2.fc17.x86_64 started giving what appeared to be random Fatal error: 500 OOPS: priv_sock_ aborts. With no explanation.

Further examination reveals the following: Under 3.5 vsftpd is unable to get directory information for a directory of 32 items or greater. Which means that neither directory listings nor file operations can be run. Reverting to an earlier Kernel restores normal vsftpd function. 

So although the problem is visible through vsftpd, I would suspect that it is a Kernel 3.5 X64 issue.

Kernel 3.5.0-2.fc17.i686 with vsftpd-3.0.0-2.fc17.i686 does not seem to have the same problems.


Version-Release number of selected component (if applicable):
vsftpd-3.0.0-2.fc17.x86_64
kernel-3.5.0-2.fc17.x86_64

How reproducible:
Easily. Boot with kernel-3.5.0-2.fc17.x86_64 and attempt file operations on that machine using vsftpd-3.0.0-2.fc17.x86_64

Steps to Reproduce:
1. Install and boot to kernel-3.5.0-2.fc17.x86_64
2. Install and system start vsftpd-3.0.0-2.fc17.x86_64
3. Attempt to either ftp or lftp to that machine from any other machine to a directory with 32 or more files in it.
  
Actual results:
Fatal error: 500 OOPS: priv_sock_get_cmd

Expected results:
Directory listing in this case.

Additional info:
Comment 1 James Collins 2012-08-06 07:38:30 EDT
It could be both a kernel and vsftpd issue actually. Some change in the Kernel that vsftpd needs to take into account? But it might well have wider implications, so both kernel and vsftpd developers may need to investigate. The number of files being 32 is an interesting number...
Comment 2 Brian Rademacher 2012-08-15 19:25:27 EDT
Interesting, I ran into this and hadn't attributed it to a kernel update, but...
I'm running kernel 3.5.1-1.fc17.x86_64

More info:
-I have selinux completely disabled from boot

-Setting the following allows it to work, but doesn't allow access to anything outside of a user's home dir:

chroot_local_user=YES
allow_writeable_chroot=YES

Using the chroot list to attempt to allow access for specific users fails with the same error.

Hope this gets fixed soon..It's rather crippling for FTP...
Comment 3 Brian Rademacher 2012-08-17 16:48:02 EDT
Continues to not work with kernel 3.5.2-1.fc17.x86_64...
Comment 4 Andras Kemeny 2012-08-18 06:05:25 EDT
confirmed here as well.

vsftpd.log shows nothing beyond "150 here comes the directory connection" even though verbose logging is turned on.

tried to use the hack above (chroot etc), does not work that way either.

tried both passive and active ftp connections, the "500 OOPS: priv_sock_get_cmd" error message pops up in passive mode only, active mode only returns nothing and on the next command attempt the connection is lost (seems like the vsftpd child process dies without a whimper).

please advise whether any work is commencing on fixing this bug - my clients are ready to tear me a new one. (i know, using sftp/scp could be an alternative, but there is a reason why ssh is disabled for most of the logins.)

ps: actually, starting to regret my decision to upgrade to fedora 17. php is now running without any accelerators (both apc and xcache fail in various ways with php 5.4.5, actually to the extent that the phpMyAdmin installation from the official rpm doesn't work), and now this. what happened?!
Comment 5 James Collins 2012-08-18 06:16:39 EDT
I changed the Component to Kernel, because I'm pretty sure that's where the problem lies. There's something dodgy with the buffers. But I haven't had time to work out what.

Incidentally, if you don't do a file list, you can still transfer files just fine...

James :) Collins.
Comment 6 Andras Kemeny 2012-08-18 06:47:29 EDT
(In reply to comment #5)
> Incidentally, if you don't do a file list, you can still transfer files just
> fine...

try telling that to clients using total commander. :/ (and actually, i'm using total commander myself as well sometimes. i now prefer winscp but due to security concerns i severely restrict who's allowed to ssh in, so that's not really a solution for "simple" users either.)

btw, i suspected that something may be off in the kernel, so i added this to the config:

use_sendfile=NO

but the bug remained still, so it's not a sendfile problem.
Comment 7 Dave Jones 2012-08-20 12:33:56 EDT
any selinux denials in /var/log/audit/ ?
Comment 8 Andras Kemeny 2012-08-20 15:04:43 EDT
(In reply to comment #7)
> any selinux denials in /var/log/audit/ ?

using SELINUXTYPE=DISABLED... :) it's not an SElinux issue, but besides, no log entries like that.
Comment 9 James Collins 2012-08-20 17:31:58 EDT
It seems that feature is usually disabled on systems exhibiting this behaviour, so it's not outside the realm of possibilities that it is part of the problem. Perhaps disabling SElinux is causing the errors? However it doesn't answer the genereal flakiness of machines which are run on 3.5 vs 3.4
Comment 10 John Decker 2012-09-03 03:59:01 EDT
Still NOT working on kernel 3.5.3-1.fc17.x86_64, I'm going back to 3.4.6, it works there.
Comment 11 John Decker 2012-09-03 04:00:21 EDT
Still NOT working on kernel 3.5.3-1.fc17.x86_64, I'm going back to 3.4.6, it works there.
Comment 12 Sergej Pupykin 2012-09-05 12:53:42 EDT
workaround:

seccomp_sandbox=NO
Comment 13 Andras Kemeny 2012-09-05 12:58:12 EDT
(In reply to comment #12)
> workaround:
> 
> seccomp_sandbox=NO

a resounding THANK YOU. finally seems to work. now if only apc would be fixed... but that's a different bugtrack number. :)
Comment 14 Brian Rademacher 2012-09-05 20:00:19 EDT
Excellent!!  Thank you very much...
Comment 15 Sergej Pupykin 2012-09-07 08:33:13 EDT
btw, vsftpd author (Chris Evans) is aware and promised to look which syscall is causing the trouble.
Comment 16 Kyle McMartin 2012-09-14 20:50:59 EDT
The problem is the seccomp filter doesn't include __NR_munmap in the default case, which glibc is using to back (FILE*) IO. (For the interested reader, qsort is being called by vsftpd, which results in glibc doing a sysconf(SC_PHYS_PAGES) to find out how much memory there is, which results in the mmap/munmap of /proc/meminfo.

http://kyle.fedorapeople.org/vsftpd-3.0.0-seccomp-fix-munmap-sigsys.patch

should fix the issue when seccomp_sandbox=YES.

[There's a secondary issue when using text_userdb_names, I'll address that separately.]

regards, Kyle
Comment 17 Jiri Skala 2012-09-17 04:47:54 EDT
Thanks Kyle,
this is fixed in the latest vsftpd-3.0.1 too. I plan to rebase rawhide as well as f17, f18.

Jiri
Comment 18 James Collins 2012-09-17 05:38:23 EDT
Thanks for the workaround Kyle and for the fix Jiri. All of us in the "Trenches" really appreciate your hard work!

James.
Comment 19 Kyle McMartin 2012-09-17 10:07:02 EDT
No problem, I'm not sure if Chris fixed the text_userdb_names issue as I've not had a chance to look at the 3.0.1 diff yet, but I'll do that today.

--Kyle
Comment 20 Sergej Pupykin 2012-09-19 03:13:26 EDT
And one more fix was needed, but it should be good in vsftpd-3.0.2

>> vsftpd-3.0.2 is released - the only noteworthy fixes are two seccomp sandbox policy tweaks which stops session crashes when listing large directories.
Comment 21 Fedora Update System 2012-09-19 07:36:22 EDT
vsftpd-3.0.2-1.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/vsftpd-3.0.2-1.fc17
Comment 22 Fedora Update System 2012-09-21 19:57:49 EDT
Package vsftpd-3.0.2-1.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing vsftpd-3.0.2-1.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-14450/vsftpd-3.0.2-1.fc17
then log in and leave karma (feedback).
Comment 23 Ralf Baechle 2012-10-01 13:02:21 EDT
With kernel-3.5.4-2.fc17.x86_64 and vsftpd-3.0.2-1.fc17.x86_64 I still seem to be hitting this issue:

ftp> dir
500 OOPS: priv_sock_get_int
Passive mode refused.
ftp> 

seccomp_sandbox=NO fixes this.  The interesting part of strace -ff (without seccomp_sandbox=NO) looks like:

[...]
"PASV\r\n", 4096, MSG_PEEK, NULL, NULL) = 6
[pid  4774] read(0, "PASV\r\n", 6)      = 6
[pid  4774] write(5, "\n", 1 <unfinished ...>
[pid  4772] <... read resumed> "\n", 1) = 1
[pid  4774] <... write resumed> )       = 1
[pid  4772] write(4, "\1", 1 <unfinished ...>
[pid  4774] read(5,  <unfinished ...>
[pid  4772] <... write resumed> )       = 1
[pid  4774] <... read resumed> "\1", 1) = 1
[pid  4772] read(4,  <unfinished ...>
[pid  4774] write(5, "\f", 1)           = 1
[pid  4772] <... read resumed> "\f", 1) = 1
[pid  4772] +++ killed by SIGSYS +++
[pid  4765] <... accept resumed> 0x7fff221031c0, [28]) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
[pid  4765] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=4772, si_status=SIGSYS, si_utime=0, si_stime=0} ---
[pid  4765] alarm(1)                    = 0
[pid  4765] rt_sigreturn()              = -1 EINTR (Interrupted system call)
[pid  4765] alarm(0)                    = 1
[pid  4765] wait4(-1, NULL, WNOHANG, NULL) = 4772
[pid  4765] wait4(-1, NULL, WNOHANG, NULL) = -1 ECHILD (No child processes)
[pid  4765] accept(3,  <unfinished ...>
[pid  4774] read(5, "", 4)              = 0
[pid  4774] fcntl(0, F_GETFL)           = 0x2 (flags O_RDWR)
[pid  4774] fcntl(0, F_SETFL, O_RDWR|O_NONBLOCK) = 0
[pid  4774] write(0, "500 OOPS: ", 10)  = 10
[pid  4774] write(0, "priv_sock_get_int", 17) = 17
[pid  4774] write(0, "\r\n", 2)         = 2
[pid  4774] exit_group(2)               = ?
[pid  4774] +++ exited with 2 +++
[...]

If I rerun the same job with seccomp_sandbox=NO I'm getting: the following syscalls after where above the SIGSYS struck:

[...]
[pid  4864] gettimeofday({1349110483, 633323}, NULL) = 0
[pid  4864] socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 5
[pid  4864] setsockopt(5, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0
[pid  4864] bind(5, {sa_family=AF_INET, sin_port=htons(59857), sin_addr=inet_addr("217.197.83.146")}, 16) = 0

I take it this was tested on bare metal on a modern machine where glibc avoids the gettimeofday(2) syscall but not on an old system without TSC or a VM where gettimeofday(2) needs to be called?  Something like that.
Comment 24 Kyle McMartin 2012-10-01 13:38:31 EDT
Hrm. Yeah, that does sound likely.
Comment 25 Andy Blanchard 2013-02-18 14:38:09 EST
Also seeing the same problem on Fedora 18 (just upgraded from Fedora 16) with v3.0.2-1 of VSFTPd.  Can get directory listings, but attempts at file transfers abort the connection without any data received.

Adding the line "seccomp_sandbox=NO" to vsftpd.conf resolves the problem in FC18 too.
Comment 26 Attila 2013-03-12 11:40:53 EDT
I can confirm this bug on Fedora 18, kernel 3.8.2-206.fc18.x86_64 and vsftpd-3.0.2-1.fc18.x86_64.

ftp> ls
500 OOPS: priv_sock_get_int
Passive mode refused.

seccomp_sandbox=NO resolves the problem.
Comment 27 Andras Kemeny 2013-03-12 11:49:16 EDT
i can also confirm the same thing after an update on Fedora 17, vsftpd version 3.0.0-2.fc17. the seccomp_sandbox=NO resolves the problem as before.
Comment 28 Kyle McMartin 2013-03-29 14:02:58 EDT
Looks like mmap is still being problematic, along with wait4 and alarm. I'll poke upstream about this.
Comment 29 Mike Iglesias 2013-05-21 19:01:55 EDT
I'm seeing this on a newly installed F17 system with all updates applied.  Kernel kernel-3.8.12-100.fc17.x86_64, vsftpd vsftpd-3.0.0-2.fc17.x86_64.  When I connect with a ftp client, I immediately get 

$ ftp xxx
Connected to xxx.
500 OOPS: child died
ftp> quit

Adding seccomp_sandbox=NO fixes this.
Comment 30 Fedora End Of Life 2013-07-04 01:57:16 EDT
This message is a reminder that Fedora 17 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 17. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '17'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 17's end of life.

Bug Reporter:  Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 17 is end of life. If you 
would still like  to see this bug fixed and are able to reproduce it 
against a later version  of Fedora, you are encouraged  change the 
'version' to a later Fedora version prior to Fedora 17's end of life.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.
Comment 31 David Strauss 2013-08-12 19:57:35 EDT
Confirming that this is still an issue on Fedora 19.
Comment 32 Kyle McMartin 2013-08-13 12:27:31 EDT
I'll look at it soon unless Jiri beats me.
Comment 33 Jiri Skala 2013-08-15 10:06:06 EDT
I don't see the issue on F19 with kernel 3.10.3-300.fc19.x86_64. Could someone confirm it?

Unfortunately the issue persists on F18 (kernel 3.10.6-100.fc18.x86_64). I'll investigate what's behind.
Comment 34 Jiri Skala 2013-09-10 03:38:22 EDT
I've changed temporarily default value of seccomp_sandbox to NO in f18 branch as a workaround together with number of other changes.
Comment 35 Jiri Skala 2013-12-03 03:39:12 EST
With respect to comments #33 and #34 inclusive the fact I didn't hear about the bug after commenting I'm going to close the bug.
Comment 36 Greg Scott 2015-02-26 10:38:01 EST
If anyone stumbles across this, even though this bugzilla is closed, it's still a bug on F21 with vsftpd-3.0.2-12.fc21.x86_64.  Putting seccomp_sandbox=NO into vstpd.conf still works around the problem.
Comment 37 Jalal Ahmed 2015-08-19 12:53:08 EDT
This problem still there in Fedora 22, it gets solve with adding
seccomp_sandbox=NO
into /etc/vsftpd/vsftpd.conf
as mentioned up.
Comment 38 Alexander Todorov 2016-03-02 10:05:52 EST
Jiri,
please see comment #37. I've also seen a similar (or same)  issue with Fedora 23.

kernel-4.4.2-301.fc23.x86_64
vsftpd-3.0.2-14.fc23.x86_64

echo 'seccomp_sandbox=NO' >> /etc/vsftpd/vsftpd.conf fixed it for me.

Perhaps you need a new bug open for that ?

Note You need to log in before you can comment on or make changes to this bug.