Bug 662223 - Can't build packages: fails adding 'mockbuild' group
Can't build packages: fails adding 'mockbuild' group
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: mock (Show other bugs)
14
Unspecified Unspecified
low Severity medium
: ---
: ---
Assigned To: David Cantrell
Fedora Extras Quality Assurance
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2010-12-10 17:14 EST by Alex Hudson (Fedora Address)
Modified: 2013-01-10 01:22 EST (History)
4 users (show)

See Also:
Fixed In Version: mock-1.1.10-1.el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-12-28 15:56:07 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
Proposed patch for early warning of incorrect uid and gid (2.11 KB, patch)
2010-12-13 13:34 EST, Clark Williams
no flags Details | Diff
root.log showing failure to run userdel, groupdel and groupadd (6.66 KB, text/plain)
2010-12-14 04:40 EST, Michael Hampton
no flags Details
mock -r fedora-14-i386 --trace --rebuild mock-1.1.6-1.fc14.src.rpm (6.72 KB, text/plain)
2010-12-15 12:56 EST, Michael Hampton
no flags Details
mock -r fedora-14-x86_64 --trace --rebuild mock-1.1.6-1.fc14.src.rpm (37.20 KB, text/plain)
2010-12-15 12:57 EST, Michael Hampton
no flags Details
Output from 'mock init -r fedora-14-x86_64 --trace' (37.49 KB, text/plain)
2010-12-15 13:16 EST, Alex Hudson (Fedora Address)
no flags Details
root.log from unsuccessful init (6.69 KB, text/plain)
2010-12-15 13:17 EST, Alex Hudson (Fedora Address)
no flags Details
Force userdel to run within chroot (814 bytes, patch)
2010-12-17 03:02 EST, Michael Hampton
no flags Details | Diff

  None (edit)
Description Alex Hudson (Fedora Address) 2010-12-10 17:14:17 EST
Description of problem:

Every time I'm attempting to build a package with mock, I'm getting an error saying that "/usr/sbin/groupadd -g 498 mockbuild" is failing. This isn't a huge surprise since gid 498 is "mock" here. I tried all sorts of poking in the config files and can't figure out why it's trying to do this or why it is so set on this gid.

Version-Release number of selected component (if applicable):

mock-1.1.6-1.fc14.noarch

How reproducible:

Happens every time. It's doing this on my main F14 machine. I also went to boot.fedoraproject.org and installed an entirely clean F14 virtual machine; it's doing exactly the same thing there.

Steps to Reproduce:
1. Any "mock --rebuild -r fedora-14-x86_64 <source.rpm>"
2.
3.
  
Actual results:

ERROR: Command failed. See logs for output.
 # ['/usr/sbin/groupadd', '-g', '498', 'mockbuild']
Comment 1 Jesse Keating 2010-12-10 17:25:59 EST
Are you running as a non-root user who's in the mock group?
Comment 2 Alex Hudson (Fedora Address) 2010-12-10 17:40:24 EST
Nope, running as root - checking the groups file, the mock group is empty anyway.
Comment 3 Jesse Keating 2010-12-10 17:52:09 EST
Yeah, stuff doesn't work right when you do it as root.  Please add a user to the mock group and run mock as that user.  We have plans to error out mock when it detects running as root.
Comment 4 Alex Hudson (Fedora Address) 2010-12-10 18:06:39 EST
Hrm, it's a bit annoying that a. there's not a message when run as root, but more b. when you run it as a user it asks to authenticate to root....

But anyway, that doesn't seem to do it either:

[alex@localhost ~]$ groups
alex mock
[alex@localhost ~]$ mock init -r fedora-14-x86_64
INFO: mock.py version 1.1.6 starting...
[...snip...]
State Changed: running yum
ERROR: Command failed. See logs for output.
 # ['/usr/sbin/groupadd', '-g', '498', 'mockbuild']

I've removed and re-installed mock, removed the previous fedora-14 cache, and even rebooted - no joy :(
Comment 5 Jesse Keating 2010-12-13 13:16:37 EST
Yeah, we know the lack of a warning sucks, and the ask for root password when ran as a user that isn't in the mock group needs to be removed.

More on your issue.  It's possible that one of the packages that is being installed into the chroot is taking group ID 498 when it shouldn't be.  Can you poke at the partially built chroot and see what group has 498 in the group file?  It could be a package that thinks it's grabbing a specific group ID to match it's user ID, but is failing.
Comment 6 Clark Williams 2010-12-13 13:34:59 EST
Created attachment 468450 [details]
Proposed patch for early warning of incorrect uid and gid

Attached is a commit I have in my local git tree. This should provide an early check on running from the root account and running from an account that is not in the mock group.
Comment 7 Michael Hampton 2010-12-14 04:40:40 EST
Created attachment 468567 [details]
root.log showing failure to run userdel, groupdel and groupadd

I am not running mock as root and I receive a very similar error, with the group id being 481. This gid happens to be assigned to the mock group on my system.

Attached is the result/root.log showing mock failing.
Comment 8 Fedora Update System 2010-12-14 11:10:52 EST
mock-1.1.7-1.fc13 has been submitted as an update for Fedora 13.
https://admin.fedoraproject.org/updates/mock-1.1.7-1.fc13
Comment 9 Fedora Update System 2010-12-14 11:11:05 EST
mock-1.1.7-1.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/mock-1.1.7-1.fc14
Comment 10 Fedora Update System 2010-12-14 11:13:42 EST
mock-1.0.14-1.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/mock-1.0.14-1.el5
Comment 11 Michael Hampton 2010-12-14 14:43:03 EST
I tried the submitted mock-1.1.7.noarch build downloaded from koji and the problem persists. Mock still fails with exactly the same errors when attempting to run userdel, groupdel or groupadd.
Comment 12 Fedora Update System 2010-12-15 04:02:23 EST
mock-1.1.7-1.fc14 has been pushed to the Fedora 14 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update mock'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/mock-1.1.7-1.fc14
Comment 13 Clark Williams 2010-12-15 09:24:29 EST
Michael, 

Are you running as the mockbuild user? It looks like the commands running in the chroot to delete and recreate the mock account and homedir are failing because the owner of those directories is logged in.

Mock presumes that you'll be running from a personal account that is a member of the mock group and that no one will be logged in as mockbuild.
Comment 14 Michael Hampton 2010-12-15 09:32:23 EST
Clark,

I am logged in as myself. It never occurred to me to login as any other user.

error@underground ~ $ id
uid=500(error) gid=500(error) groups=500(error),481(mock) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
Comment 15 Clark Williams 2010-12-15 09:58:01 EST
Very weird.

What I'm looking at in the log are the following lines:

DEBUG util.py:82:  remove tree: /var/lib/mock/fedora-14-i386/root/builddir
DEBUG util.py:301:  Executing command: ['/usr/sbin/userdel', '-r', 'mockbuild']
DEBUG util.py:267:  userdel: user mockbuild is currently logged in
DEBUG util.py:340:  Child returncode was: 8
DEBUG util.py:301:  Executing command: ['/usr/sbin/groupdel', 'mockbuild']
DEBUG util.py:267:  groupdel: cannot remove the primary group of user 'mockbuild'
DEBUG util.py:340:  Child returncode was: 8
DEBUG util.py:301:  Executing command: ['/usr/sbin/groupadd', '-g', '481', 'mockbuild']
DEBUG util.py:340:  Child returncode was: 9

For some reason the system thinks that 'mockbuild' is logged in and refuses to remove directories, uid/gid, etc.  

Don't think I've ever seen this before. Not surprising since I haven't updated my laptop to F14 yet. I'll go provision a box from the lab and see if I can duplicate it as well.
Comment 16 Clark Williams 2010-12-15 12:39:26 EST
I just did a clean install of x86_64 F14 on a box, created a user in the mock group and rebuilt the mock srpm for both x86_64 and i386 in F14 chroots. This was using the current released mock-1.1.6. I've also done the 'mock init -r fedora-14-x86_64' that was shown above as well as 'mock -r fedora-14-x86_64 shell' and poked around in the chroot. No failures.

So, we need to take a closer look at what's different on the failing build systems of both Michael and Alex. 

Could you guys do a 'mock -r fedora-14-x86_64 --trace --rebuild <some_srpm>' and post the output to this bz?
Comment 17 Michael Hampton 2010-12-15 12:56:25 EST
Created attachment 468921 [details]
mock -r fedora-14-i386 --trace --rebuild mock-1.1.6-1.fc14.src.rpm

Clark, it works fine for me with -r fedora-14-x86_64; it only fails with -r fedora-14-i386 (on x86_64). This is the failed build attempt; I'll also attach a successful one for comparison.
Comment 18 Michael Hampton 2010-12-15 12:57:08 EST
Created attachment 468922 [details]
mock -r fedora-14-x86_64 --trace --rebuild mock-1.1.6-1.fc14.src.rpm
Comment 19 Alex Hudson (Fedora Address) 2010-12-15 13:16:16 EST
Created attachment 468928 [details]
Output from 'mock init -r fedora-14-x86_64 --trace'
Comment 20 Alex Hudson (Fedora Address) 2010-12-15 13:17:47 EST
Created attachment 468930 [details]
root.log from unsuccessful init

I've attached the trace output and root.log. This isn't from a rebuild; I can't even get that far - mock init fails.

This is a clean F14 install, and it's 64-bit trying to mock 64-bit. I don't see how the 'mockbuild' user could be currently logged in.
Comment 21 Clark Williams 2010-12-15 14:30:45 EST
The failure of /usr/sbin/userdel to remove mockbuild seems to be what causes all of this. In both logs I see:

     userdel: user mockbuild is currently logged in

and I believe the groupdel failure is related to this as well. Obviously the groupadd failure is because mockbuild still exists. Now to try and figure out *why* mockbuild is supposedly logged in to the chroot. 

The chrootuid assigned to mockbuild in the chroot is the unprivileged uid that invoked mock. The chrootgid is the gid of the mock group. 

Alex and Michael, how are you logged into your build system? Are you running on a server that doesn't have graphics running or are you logging in graphically via GNOME/KDE/XFCE/etc?
Comment 22 Michael Hampton 2010-12-15 20:11:03 EST
Clark, this is my home computer and I'm using GNOME. But I tried it from a text console and got exactly the same problem.
Comment 23 Clark Williams 2010-12-16 10:18:18 EST
Ah well, another theory shot down.

I'm in the office today so I'll try the build from the console, just to see if there's something interesting there.
Comment 24 Fedora Update System 2010-12-17 01:51:45 EST
mock-1.0.14-1.el5 has been pushed to the Fedora EPEL 5 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update mock'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/mock-1.0.14-1.el5
Comment 25 Michael Hampton 2010-12-17 02:58:26 EST
OK, well I dug into this a bit further, and what I found is that /usr/sbin/userdel (part of shadow-utils) is reading directly from /proc to determine if a user is logged in, by whether the uid/gid has any running processes, and bailing if this is the case. This is definitely not the desired behavior in this chroot environment. In any case this can be overridden with the -f option to userdel. Indeed, once I made this change, in line 699 of /usr/lib/python2.7/site-packages/mock/backend.py, mock started building packages properly.
Comment 26 Michael Hampton 2010-12-17 03:02:28 EST
Created attachment 469308 [details]
Force userdel to run within chroot
Comment 27 Clark Williams 2010-12-17 10:24:26 EST
Ahh, that makes sense, although I'm not sure why we're not seeing this more often. Bound to be a race condition where the mockbuild uid hasn't been cleared out from /proc when a new command is being run. 

I'll queue up the above change for release in 1.1.8. 

Many thanks Michael.
Comment 28 Fedora Update System 2010-12-28 15:55:57 EST
mock-1.1.7-1.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 29 Fedora Update System 2011-01-18 14:58:13 EST
mock-1.1.8-1.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/mock-1.1.8-1.fc14
Comment 30 Fedora Update System 2011-01-18 14:58:54 EST
mock-1.1.8-1.fc13 has been submitted as an update for Fedora 13.
https://admin.fedoraproject.org/updates/mock-1.1.8-1.fc13
Comment 31 Fedora Update System 2011-01-18 15:03:46 EST
mock-1.0.15-1.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/mock-1.0.15-1.el5
Comment 32 Fedora Update System 2011-01-21 18:03:45 EST
mock-1.1.8-1.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 33 Fedora Update System 2011-02-19 21:25:56 EST
mock-1.1.9-1.fc13 has been submitted as an update for Fedora 13.
https://admin.fedoraproject.org/updates/mock-1.1.9-1.fc13
Comment 34 Fedora Update System 2011-02-19 21:29:09 EST
mock-1.0.16-1.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/mock-1.0.16-1.el5
Comment 35 Fedora Update System 2011-02-19 21:32:02 EST
mock-1.1.9-1.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/mock-1.1.9-1.el6
Comment 36 Fedora Update System 2011-02-19 21:34:55 EST
mock-1.1.9-1.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/mock-1.1.9-1.fc14
Comment 37 Fedora Update System 2011-03-03 03:24:28 EST
mock-1.1.9-1.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 38 Fedora Update System 2011-03-03 03:33:30 EST
mock-1.1.9-1.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 39 Fedora Update System 2011-05-13 16:33:42 EDT
mock-1.1.10-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/mock-1.1.10-1.fc15
Comment 40 Fedora Update System 2011-05-13 16:38:27 EDT
mock-1.1.10-1.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/mock-1.1.10-1.fc14
Comment 41 Fedora Update System 2011-05-13 16:42:45 EDT
mock-1.0.17-1.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/mock-1.0.17-1.el5
Comment 42 Fedora Update System 2011-05-13 16:47:03 EDT
mock-1.1.10-1.fc13 has been submitted as an update for Fedora 13.
https://admin.fedoraproject.org/updates/mock-1.1.10-1.fc13
Comment 43 Fedora Update System 2011-05-13 16:51:15 EDT
mock-1.1.10-1.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/mock-1.1.10-1.el6
Comment 44 Fedora Update System 2011-05-19 00:35:04 EDT
mock-1.1.10-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 45 Fedora Update System 2011-05-24 22:42:29 EDT
mock-1.1.10-1.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 46 Fedora Update System 2011-05-24 23:16:47 EDT
mock-1.1.10-1.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 47 Fedora Update System 2011-06-02 15:06:40 EDT
mock-1.0.17-1.el5 has been pushed to the Fedora EPEL 5 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 48 Fedora Update System 2011-06-02 15:16:39 EDT
mock-1.1.10-1.el6 has been pushed to the Fedora EPEL 6 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.