Bug 524752

Summary: udevd keeps CPU at 50% once running
Product: [Fedora] Fedora Reporter: David Cantrell <dcantrell>
Component: udevAssignee: Harald Hoyer <harald>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: high    
Version: rawhideCC: ddumas, eparis, harald, hpicht, maier
Target Milestone: ---   
Target Release: ---   
Hardware: s390   
OS: Linux   
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 525574 (view as bug list) Environment:
Last Closed: 2009-09-25 13:11:35 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On:    
Bug Blocks: 525574    
Description Flags
udevd.strace.xz none

Description David Cantrell 2009-09-21 23:41:02 EDT
On s390x, I'm seeing udevd stay at 50% of the CPU and continually spew the following message to the console:

udevd: error getting buffer for inotify

I am not sure what's going on, but if you want help debugging this, let me know.
Comment 1 Harald Hoyer 2009-09-23 16:17:25 EDT
this is a kernel bug, IIRC.. is this an old kernel?
Comment 2 Harald Hoyer 2009-09-23 16:28:15 EDT
can you strace udevd?

might be related to bug #499907

Eric, what do you think?
Comment 3 Denise Dumas 2009-09-23 16:33:04 EDT
btw, this is a HUGE problem for S390 debug effort - which is currently blocking rhel6 alpha2.  Help appreciated.
Comment 4 Eric Paris 2009-09-23 16:42:38 EDT
Let me see the strace of udevd and we'll see what's going on....
Comment 5 Eric Paris 2009-09-23 16:44:49 EDT
also let me know what the kernel is in question and i'll make sure that kernel doesn't have any of the obvious problems....
Comment 6 David Cantrell 2009-09-23 17:21:51 EDT
Created attachment 362355 [details]

Command used:
strace -f -F -v -o /udevd.strace udevd --debug --debug-trace
Comment 7 David Cantrell 2009-09-23 17:24:48 EDT
Kernel is 2.6.31-0.174.rc7.git2.fc12.s390x
Comment 8 Eric Paris 2009-09-23 17:44:50 EDT
Not the same problem....

389   ioctl(6, FIONREAD, [224])         = 0
389   mmap(NULL, 962072678400, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)

Looks like the ioctl gave something reasonable.  But udev is trying to allocate a buffer WAY WAY WAY to big....
Comment 9 Harald Hoyer 2009-09-23 18:04:19 EDT
This is the code in question:

        ssize_t nbytes;

        if ((ioctl(pfd[FD_INOTIFY].fd, FIONREAD, &nbytes) < 0) || (nbytes <= 0))
                return 0;

        buf = malloc(nbytes);
Comment 10 Eric Paris 2009-09-23 18:13:42 EDT
the kernel calls it a size_t rather than ssize_t

no idea how that could cause the problem in question.  But i'll check to see if this is a change.

If not, I think we need to find a libc person....
Comment 11 Eric Paris 2009-09-23 18:26:51 EDT
RHEL5 kernel called this as an unsigned int....

962072678400    = 0x000000E000001000
1168231108608   = 0x0000011000001000

So it looks like some of the higher bits of the nbytes is non-zero.

I admit I'm confused, I could see that happening on the old setup...
Comment 12 Eric Paris 2009-09-23 18:45:13 EDT
ok, I'm back.  RHEL5

struct inotify_device {
        unsigned int            queue_size;     /* size of the queue (bytes) */

ret = put_user(dev->queue_size, (int __user *) p);


size_t send_len = 0;

ret = put_user(send_len, (int __user *) p);

In both cases though we should only be writing an int's worth of data.  I believe that on s390, and x86_64 even:
size_t = long
ssize_t = unsigned long

but I'm only writing an int back from the kernel.  My guess is that those high bits are just stack leftovers.  I'm assuming that udev has just always gotten lucky and had 0 there and now we've hit a system where you don't.   I'd say either use an int, or explicitly 0 it out.....

Comment 14 Harald Hoyer 2009-09-25 13:11:35 EDT
should be fixed in udev-145-9.fc12