Description of problem: In _udev_notify_sem_create(), after dm creates the semaphore and sets it's value, it calls semctl(gen_semid, 0, GETVAL) to get the value. I don't know how likely this call is to ever fail, but if it does, _udev_notify_sem_create() sets the cookie pointer to 0, so the caller has no access to the semaphore, and returns an error, leaving the semaphore to just sit there. Conversely, if setting the value fails, dm destroys the semaphore before returning, so that everything is the same as before the call happened. There's a similar issue in _udev_notify_sem_inc, but in this case the cookie is returned along with the error. However, from the caller's perspective, the cookie is still in an indeterminate state. This is a trickier issue to deal with, since it's possible the cookie has already been used in prior commands at this point. Version-Release number of selected component (if applicable): In RHEL7 and the latest git code How reproducible: Not sure. It may be completely impossible for all I know. This bug is solely from code inspection.
Use 'dmsetup udevcookies' to see if any cookies are left behind, and use 'ipcs -s' to see all semaphores on the system, and look in /proc/sys/kernel/sem as the message says to see what the limits are.
(In reply to Ben Marzinski from comment #0) > Description of problem: > In _udev_notify_sem_create(), after dm creates the semaphore and sets > it's value, it calls > > semctl(gen_semid, 0, GETVAL) to get the value. I don't know how likely > this call is to ever fail, but if it does, _udev_notify_sem_create() > sets the cookie pointer to 0, so the caller has no access to the > semaphore, and returns an error, leaving the semaphore to just sit > there. > > Conversely, if setting the value fails, dm destroys the semaphore before > returning, so that everything is the same as before the call happened. > > There's a similar issue in _udev_notify_sem_inc, but in this case the > cookie is returned along with the error. However, from the caller's > perspective, the cookie is still in an indeterminate state. This is a > trickier issue to deal with, since it's possible the cookie has already been > used in prior commands at this point. In standard case the cookie handling application does not crash and should cleanup any holding semaphores on the exit path. In case of 'kill -9' or segfault it's admins task to run dmsetup udevcomplete_all [<age_in_minutes>] to drop any left cookies - but this should be a very rare situation. Also note we have this bug 1020150 - however the term of realization is unknown...
But in some of these cases, the library doesn't return the cookie, so the caller has no handle to it. The cookie handling application can't just go cleaning up all outstanding cookies, since they may belong to another application. Also, shouldn't the library be responsible for returning in a defined state? When you try to create a cookie and it fails, you have a right to expect that the library cleaned itself up, and you don't have to go poking around to see if there are any pointless cookies that the library left around.
The supported use-case by 'dmsetup' is: User creates a cookie and then runs multiple operations within the same 'cookie' sempahore (I.e. create 10 devices - and just does a final udev wait) So it's not bound to 'libdm' exit path. It's left to 'libdm' user to resolve what should happen. It's this app which knows the Cookie ID (not being stored anywhere inside libdm) libdm however does clean the cookie upon error path/failure of ioctl automatically - that's the only help we can provide here.