Bug 728004 - Using tmpfs Unable to use mock shell fails to launch
Summary: Using tmpfs Unable to use mock shell fails to launch
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: mock
Version: rawhide
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Clark Williams
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-08-03 21:59 UTC by Shawn Starr
Modified: 2012-08-18 15:15 UTC (History)
3 users (show)

Fixed In Version: mock-1.1.17-1.fc15
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-08-18 15:15:19 UTC


Attachments (Terms of Use)

Description Shawn Starr 2011-08-03 21:59:26 UTC
Description of problem:
Unable to use mock shell to examine if a build failed.

Version-Release number of selected component (if applicable):
1.1.11-1.fc16

How reproducible:
Always

Steps to Reproduce:
1. Set mock to use tmpfs
2. Do a build, if a build has a failure try to go to shell
3. run mock shell, unable to obtain chroot
  
Actual results:
Fails to mount filesystems in chroot, fails to give shell

Expected results:
mock-chroot> prompt

Comment 1 Clark Williams 2011-09-03 16:38:41 UTC
Would you include your setup for configuring to use tmpfs?

Comment 2 Yury V. Zaytsev 2011-09-24 21:49:58 UTC
Hi!

I can confirm this problem on RHEL6. I use stock *.cfg files and the following lines in site configuration file to enable the tmpfs plugin:

config_opts['plugin_conf']['tmpfs_enable'] = True    
config_opts['plugin_conf']['tmpfs_opts']['required_ram_mb'] = 5120
config_opts['plugin_conf']['tmpfs_opts']['max_fs_size'] = '4096m'

I have 6 GB of RAM allocated to this machine. I don't know if it matters, but SELinux is set to 'enforcing'.

It's interesting that I don't see /var/lib/mock/conf/root populated with files and devices during the build. It's really a mystery to me how come I don't see anything in there... but most surprisingly, the root.log seems to indicate that there exact directories are indeed used as the mount points for tmpfs:

INFO backend.py:837: Mock Version: 1.1.15 
DEBUG backend.py:274: rootdir = /var/lib/mock/epel-5-x86_64/root/ 
DEBUG backend.py:275: resultdir = /var/lib/mock/epel-5-x86_64/result 
DEBUG util.py:284: Executing command: ['mount', '-n', '-t', 'tmpfs', '-o', 'size=4096m', 'mock_chroot_tmpfs', '/var/lib/mock/epel-
5-x86_64/root/'] 
DEBUG util.py:323: Child returncode was: 0 
DEBUG util.py:57: ensuring that dir exists: /var/cache/mock/epel-5-x86_64/root_cache/ 
DEBUG util.py:284: Executing command: ['tar', '--use-compress-program', 'pigz', '-xf', '/var/cache/mock/epel-5-x86_64/root_cache/c
ache.tar.gz', '-C', '/var/lib/mock/epel-5-x86_64/root/'] 
DEBUG util.py:323: Child returncode was: 0 
DEBUG util.py:57: ensuring that dir exists: /var/lib/mock/epel-5-x86_64/root/./proc 
DEBUG util.py:60: creating dir: /var/lib/mock/epel-5-x86_64/root/./proc

I can provide any further information you might need to diagnose this issue.

Comment 3 Clark Williams 2011-09-25 15:07:41 UTC
This almost looks like the hooks aren't being called properly. 

If you look at the tmpfs plugin, you'll see that the mound command is setup as a 'preinit' hook, which should be call when the chroot object (in backend.py) does a self._callHooks('preinit') but I don't see any print saying 'mounting tmpfs'. 

I don't know if this is specific to tmpfs or there's something worse wrong in the hooks logic.

Comment 4 Clark Williams 2011-09-25 15:23:58 UTC
Ahh, I remember now. We don't create a Root object when we do a shell, due to the Root object not really being setup for interactive use. 

I'll have to look at this some more. Possibly we can create a Root object, do all the appropriate hooks, *then* kick off the interactive shell, then clean up the Root object after the shell exits.

Comment 5 Clark Williams 2011-10-12 22:31:14 UTC
I've modified the --shell and --chroot commands to do the same thing as a --rebuild. Haven't tested with tmpfs yet but I suspect it will work properly this time. 

These changes should appear in mock-1.1.16

Comment 6 Fedora Update System 2011-10-14 16:28:20 UTC
mock-1.1.16-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/mock-1.1.16-1.fc15

Comment 7 Fedora Update System 2011-10-14 16:29:25 UTC
mock-1.0.23-1.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/mock-1.0.23-1.el5

Comment 8 Fedora Update System 2011-10-14 16:30:27 UTC
mock-1.1.16-1.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/mock-1.1.16-1.fc14

Comment 9 Fedora Update System 2011-10-14 16:31:30 UTC
mock-1.1.16-1.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/mock-1.1.16-1.el6

Comment 10 Fedora Update System 2011-10-15 20:24:48 UTC
Package mock-1.1.16-1.fc14:
* should fix your issue,
* was pushed to the Fedora 14 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing mock-1.1.16-1.fc14'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2011-14382
then log in and leave karma (feedback).

Comment 11 Yury V. Zaytsev 2011-10-15 21:25:28 UTC
Hmmmm, unfortunately the update didn't seem to fix the issue for me. How about you, Shawn?

[zaytsev@locke SRPMS]$ mock --shell -r epel-6-x86_64
INFO: mock.py version 1.1.16 starting...
State Changed: init plugins
INFO: tmpfs initialized
INFO: selinux enabled
State Changed: start
State Changed: lock buildroot
INFO: mounting tmpfs at /var/lib/mock/epel-6-x86_64/root/.
State Changed: shell
Traceback (most recent call last):
  File "/usr/sbin/mock", line 842, in <module>
    main(retParams)
  File "/usr/sbin/mock", line 730, in main
    sys.exit(chroot.shell())
  File "/usr/lib/python2.6/site-packages/mockbuild/backend.py", line 646, in shell
    gid=self.chrootgid)
  File "/usr/lib/python2.6/site-packages/mockbuild/util.py", line 388, in doshell
    return subprocess.call(cmdstr, preexec_fn=preexec, env=environ, shell=True)
  File "/usr/lib64/python2.6/subprocess.py", line 478, in call
    p = Popen(*popenargs, **kwargs)
  File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.6/subprocess.py", line 1220, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Also, the lock is left behind...

What really amazes me is that the builds succeed, the directory /var/lib/mock/epel-6-x86_64/root/ exists, but when I am trying to ls it during the build I see *nothing* in there. Kind of black magic, huh...

drwxrwsr-x. 2 root    mock 4096 Oct 15 23:19 root

Comment 12 Fedora Update System 2011-10-24 22:59:14 UTC
mock-1.1.16-1.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 13 Yury V. Zaytsev 2011-10-24 23:03:32 UTC
Except that it doesn't solve the problem...

Comment 14 Fedora Update System 2011-10-24 23:04:41 UTC
mock-1.1.16-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 15 Fedora Update System 2011-11-01 15:40:15 UTC
mock-1.1.17-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/mock-1.1.17-1.fc15

Comment 16 Fedora Update System 2011-11-01 15:41:36 UTC
mock-1.1.17-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/mock-1.1.17-1.fc16

Comment 17 Fedora Update System 2011-11-01 15:42:50 UTC
mock-1.1.17-1.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/mock-1.1.17-1.fc14

Comment 18 Fedora Update System 2011-11-01 15:44:11 UTC
mock-1.0.24-1.el5 has been submitted as an update for Fedora EPEL 5.
https://admin.fedoraproject.org/updates/mock-1.0.24-1.el5

Comment 19 Fedora Update System 2011-11-01 15:45:35 UTC
mock-1.1.17-1.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/mock-1.1.17-1.el6

Comment 20 Yury V. Zaytsev 2011-11-01 19:48:01 UTC
Still doesn't work for me:

[zaytsev@locke SRPMS]$ mock --shell -r epel-6-x86_64
INFO: mock.py version 1.1.17 starting...
State Changed: init plugins
INFO: tmpfs initialized
INFO: selinux enabled
State Changed: start
State Changed: lock buildroot
INFO: mounting tmpfs at /var/lib/mock/epel-6-x86_64/root/.
State Changed: shell
Traceback (most recent call last):
  File "/usr/sbin/mock", line 849, in <module>
    main(retParams)
  File "/usr/sbin/mock", line 737, in main
    sys.exit(chroot.shell(options, cmd))
  File "/usr/lib/python2.6/site-packages/mockbuild/backend.py", line 655, in shell
    cmd=cmd)
  File "/usr/lib/python2.6/site-packages/mockbuild/util.py", line 391, in doshell
    return subprocess.call(cmdstr, preexec_fn=preexec, env=environ, shell=True)
  File "/usr/lib64/python2.6/subprocess.py", line 478, in call
    p = Popen(*popenargs, **kwargs)
  File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.6/subprocess.py", line 1220, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Comment 21 Yury V. Zaytsev 2011-11-01 19:49:25 UTC
In the mean time, building works fine:

[zaytsev@locke SRPMS]$ mock mock-1.1.17-1.el6.src.rpm -r epel-6-x86_64
INFO: mock.py version 1.1.17 starting...
State Changed: init plugins
INFO: tmpfs initialized
INFO: selinux enabled
State Changed: start
INFO: Start(mock-1.1.17-1.el6.src.rpm)  Config(epel-6-x86_64)
State Changed: lock buildroot
State Changed: clean
INFO: chroot (/var/lib/mock/epel-6-x86_64) unlocked and deleted
State Changed: unlock buildroot
State Changed: init
State Changed: lock buildroot
Mock Version: 1.1.17
INFO: Mock Version: 1.1.17
INFO: calling preinit hooks
INFO: mounting tmpfs at /var/lib/mock/epel-6-x86_64/root/.
INFO: enabled root cache
State Changed: unpacking root cache
State Changed: running yum
State Changed: unlock buildroot
State Changed: setup
State Changed: build
INFO: unmounting tmpfs.
INFO: Done(mock-1.1.17-1.el6.src.rpm) Config(epel-6-x86_64) 0 minutes 16 seconds
INFO: Results and/or logs in: /var/lib/mock/epel-6-x86_64/result
State Changed: end

Comment 22 Clark Williams 2011-11-01 20:42:23 UTC
(In reply to comment #20)
> Still doesn't work for me:
> 
> 

Yeah, I'm still not sure how to fix this. I haven't forgotten though...

Comment 23 Fedora Update System 2011-11-02 00:05:14 UTC
Package mock-1.0.24-1.el5:
* should fix your issue,
* was pushed to the Fedora EPEL 5 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=epel-testing mock-1.0.24-1.el5'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-EPEL-2011-4823
then log in and leave karma (feedback).

Comment 24 Fedora Update System 2011-11-11 01:22:34 UTC
mock-1.1.17-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 25 Fedora Update System 2011-11-26 19:23:39 UTC
mock-1.1.17-1.el6 has been pushed to the Fedora EPEL 6 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 26 Fedora Update System 2011-11-26 19:24:56 UTC
mock-1.0.24-1.el5 has been pushed to the Fedora EPEL 5 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 27 Fedora Update System 2011-11-26 23:02:12 UTC
mock-1.1.17-1.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 28 Fedora Update System 2011-11-26 23:06:28 UTC
mock-1.1.17-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 29 Yury V. Zaytsev 2012-01-12 16:27:17 UTC
mock-1.1.18: still the same...

Comment 30 Yury V. Zaytsev 2012-03-14 17:17:39 UTC
mock-1.1.21:

$ mock --shell -r epel-6-i386
INFO: mock.py version 1.1.21 starting...
State Changed: init plugins
INFO: tmpfs initialized
INFO: selinux enabled  
State Changed: start   
State Changed: lock buildroot
INFO: mounting tmpfs at /var/lib/mock/epel-6-i386/root/.
State Changed: shell   
Traceback (most recent call last):
  File "/usr/sbin/mock", line 881, in <module>
    main(retParams)
  File "/usr/sbin/mock", line 759, in main
    sys.exit(chroot.shell(options, cmd))
  File "/usr/lib/python2.6/site-packages/mockbuild/backend.py", line 682, in shell
    cmd=cmd)
  File "/usr/lib/python2.6/site-packages/mockbuild/util.py", line 402, in doshell
    return subprocess.call(cmdstr, preexec_fn=preexec, env=environ, shell=True)
  File "/usr/lib64/python2.6/subprocess.py", line 478, in call
    p = Popen(*popenargs, **kwargs)
  File "/usr/lib64/python2.6/subprocess.py", line 639, in __init__
    errread, errwrite) 
  File "/usr/lib64/python2.6/subprocess.py", line 1228, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Comment 31 Clark Williams 2012-03-16 17:50:50 UTC
Honestly, I'm not sure how to fix this (or if we really want to). 

The problem is that we mount/umount the tmpfs on mock startup/shutdown, because tmpfs is supposed to be *temporary*. I don't want to try and hack in some logic to mount the tmpfs and then leave it mounted when mock exits.  Possibly we could come up with a way to use an existing tmpfs that was mounted before mock starts up and leave it alone on exit. 

Does your failure only occur when using tmpfs?

Comment 32 Clark Williams 2012-08-18 15:15:19 UTC
I looked at this again today and honestly, I don't see a good way to keep a tmpfs build around after a build. 

Closing as not a bug.


Note You need to log in before you can comment on or make changes to this bug.