Created attachment 422218 [details] Spec file for python-mock Description of problem: This is for info on a name space collision for Red Hat or Fedora folks. In my adventures to put together a test passing rpm build of BuildBot 0.8.0, I cherry picked a couple of commits for the MailNotifier and its unit tests from the buildbot's upstream master branch. Those being: 5ff4d181d552f44b2c274a17a9a0b25d1915493f fix NameError by passing properties into createEmail e1cfc9d95d9df49e1dca449f3cf149862774f7f9 fix tests broken in 5ff4d181 The later commit pulls in an external dependency on Mock, a mocking and testing library from http://www.voidspace.org.uk/python/mock/ and available on PyPI at http://pypi.python.org/pypi/mock/. Now for the fun part, I packaged up mock for our internal package but also with the intention of getting it into Fedora or EPEL if possible. No probs here, one nicely packaged and installed python-mock rpm. No probs until I try running BuildBot's unit tests with the installed python-mock rpm. $ trial buildbot/test/unit/test_status_mail_MailNotifier.py /usr/lib64/python2.6/site-packages/twisted/mail/smtp.py:10: DeprecationWarning: the MimeWriter module is deprecated; use the email package instead import MimeWriter, tempfile, rfc822 buildbot.test.unit.test_status_mail_MailNotifier TestMailNotifier test_createEmail_message_with_patch_and_log_contains_unicode ... [ERROR] test_createEmail_message_without_patch_and_log_contains_unicode ... [ERROR] =============================================================================== [ERROR]: buildbot.test.unit.test_status_mail_MailNotifier.TestMailNotifier.test_createEmail_message_with_patch_and_log_contains_unicode Traceback (most recent call last): File "/home/gareth/WORK/git/buildbot/master/buildbot/test/unit/test_status_mail_MailNotifier.py", line 40, in test_createEmail_message_with_patch_and_log_contains_unicode build = self.make_build() File "/home/gareth/WORK/git/buildbot/master/buildbot/test/unit/test_status_mail_MailNotifier.py", line 26, in make_build b = mock.Mock() exceptions.AttributeError: 'module' object has no attribute 'Mock' =============================================================================== I even modified the test_status_mail_MailNotifier.py to use "from mock import Mock", but still no joy. [gareth@localhost] ~/WORK/git/test/buildbot/master $ trial buildbot/test/unit/test_status_mail_MailNotifier.py [ERROR] =============================================================================== [ERROR]: buildbot/test/unit/test_status_mail_MailNotifier.py Traceback (most recent call last): File "/usr/lib64/python2.6/site-packages/twisted/trial/runner.py", line 651, in loadByNames things.append(self.findByName(name)) File "/usr/lib64/python2.6/site-packages/twisted/trial/runner.py", line 460, in findByName return filenameToModule(name) File "/usr/lib64/python2.6/site-packages/twisted/trial/runner.py", line 95, in filenameToModule ret = reflect.namedAny(reflect.filenameToModuleName(fn)) File "/usr/lib64/python2.6/site-packages/twisted/python/reflect.py", line 464, in namedAny topLevelPackage = _importAndCheckStack(trialname) File "/home/gareth/WORK/git/test/buildbot/master/buildbot/test/unit/test_status_mail_MailNotifier.py", line 2, in <module> from mock import Mock exceptions.ImportError: cannot import name Mock ------------------------------------------------------------------------------- FAILED (errors=1) But python-mock was installed !!! rpm -ql python-mock . /usr/lib/python2.6/site-packages/mock.py . So to cut a long story short, the culprit is Fedora's "mock". In can also be argued that the python-mock is the culprit. Either way, we have a problem. rpm -ql mock . /usr/lib/python2.6/site-packages/mock /usr/lib/python2.6/site-packages/mock/__init__.py . I realized that building python-mock inside a mock chroot is not a problem but it is if both mock and python-mock are installed. Version-Release number of selected component (if applicable): mock-1.0.7-1.fc12.noarch How reproducible: Always. Steps to Reproduce: 1. Install mock.py to python site-packages on system either manually or with python-mock rpm 2. In python interpreter, type from mock import Mock Actual results: Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: cannot import name Mock Expected results: Expect to be able to import the Python mocking and testing library correctly. from mock import Mock Additional info: The python-mock rpm does not exist in Fedora or EPEL. Please find the spec file in attachement.
ummm Doesn't Python permit more detailed namespace qualification, and so by a simple addition of qualification, this may be solved within buildbot, rather than mungeing with Fedora's mock ? -- Russ herrold
import mock doesn't accept much qualification. The problem is that the two packages are trying to install two modules with exactly the same name - similar to two packages installing, say, /usr/lib/libmock.so. One of them needs to be renamed or moved to somewhere else in the hierarchy, and then everything *using* that module will need to be patched to find it in the new location. Since fedora-mock presumably has fewer dependents, it seems most appropriate to change that package. (there is a slight twist here in that the packages do not actually install files with the same pathname; to the level of detail needed here, though, mock/__init__.py and mock.py mean the same thing to Python)
My initial query shows that the following packages depend on fedora-mock's python packages: fedora-packager-0:0.4.2-1.fc13.noarch puritan-0:0.4-5.fc12.noarch revisor-mock-0:2.2-1.fc13.noarch
Of the 3 packages that are listed in Commment 3, only revisor-mock imports mock directly in /usr/lib/python2.6/site-packages/revisor/modmock/__init__.py from mock.backend import Root fedora-packager and puritan both use the `mock` CLI.
have you talked to upstream about renaming their package? The mock application has been around for a lot longer than python-mock: http://www.voidspace.org.uk/python/mock/changelog.html#version-0-1-0 http://cvs.fedoraproject.org/viewvc/mock/ChangeLog?revision=1.12&root=fedora&view=markup http://git.fedorahosted.org/git/?p=mock.git;a=log For Fedora, we'll need to rename one or the other of these packages. Since the mock application existed first, we'd normally lean towards letting that one keep the name. We like to get upstream buyin for someone to change rather than having to change something only locally in Fedora, though.
Note that if python-mock's upstream is unwilling to rename, you can try to contact the mock application upstream as well -- Some of them are CC'd on this bug already *ahem*. But others are in the recent git changelog.
Speaking as the mock upstream, I'm really not interested in renaming the package.
This message is a reminder that Fedora 12 is nearing its end of life. Approximately 30 (thirty) days from now Fedora will stop maintaining and issuing updates for Fedora 12. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '12'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 12's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 12 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora please change the 'version' of this bug to the applicable version. If you are unable to change the version, please add a comment here and someone will do it for you. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
In the Python world the important namespace is the PyPI namespace, for which my project is named "mock". As far as I can tell you're suggesting not just renaming the project but renaming the module. The latest release (0.7.0b3 - a beta) has had over 5300 downloads since its release on 18th September (about 3 weeks) so it is pretty popular and there is a lot of downside to renaming it. I appreciate that the various Linux distros all have difficulty with naming. In Debian mock is packaged as python-mock.
re-assigning to rawhide to avoid the bugbot. Sadly I really do think we need to change the python module name for the mock package.
I'm the buildbot maintainer and just realized this issue while trying to upgrade the package. However: (In reply to comment #5) > For Fedora, we'll need to rename one or the other of these packages. If I understand it correctly, we do not need to rename the "mock" project or RPM package, but just the mock module provided by it. If true, it looks like a much easier (and less disruptive) change.
A few notes: 1. Regarding Buildbot, the mock dependency is only needed for the unit tests. So at the cost of temporarily disabling those tests, there's no great hazard in shipping newer versions of Buildbot. 2. Most of the comments above just talk about 'mock', and I'm not sure *which* mock was intended. That said, comment 3 indicates that only three projects use fedora-mock, and only one uses the 'mock' Python namespace itself. As Michael mentioned in comment 9, the fedora-mock package is committing a Python faux pas by using a top-level package name different from its distribution name -- it should be using 'fedora_mock' or something like that instead. Compare that to python-mock, which is widely used and imported in many locations by the applications that use it. So it sounds like (a) fedora-mock is not playing nice in Python and (b) it'd be a lot easier to change the top-level Python package name that fedora-mock uses, rather than changing that of python-mock. (c) Presumably, with a name like fedora-mock, the package has no external upstream, so Fedora could make such a rename internally without affecting other distros.
I'm not sure what you're talking about with this "fedora_mock" thing. There is a package in Fedora, named 'mock', written in python, that has all it's modules in <pythondir>/site-packages/mock and imports them via constructs like: import mock.exceptions This package has been around since 2005 (the initial commit from Seth Vidal).
I'm not sure I should weigh in here, but in the broader cross-platform Python development community Michael Foord's "mock" package is a well-known and useful module for writing test cases (and I highly recommend the "Testing In Python" upstream python SIG; see http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy and http://lists.idyll.org/listinfo/testing-in-python ; hi Michael!) However, within the RPM-using community, the "mock" project is a very useful tool, and well known tool, for provisioning clean environments for performing builds. So with my "making Fedora a great Python development/deployment environment" hat on, IMHO "mock" within site-packages should go to Michael Foord's package, since that's what the broader Python community expects. It's only on the RPM-using platforms that "mock" has had a different meaning. So can I request that the mock chroot-provisioning tool changes the module import name? (and have a look at http://pycheesecake.org/wiki/PythonTestingToolsTaxonomy#MockTestingTools since there are quite a few similarly-named modules, alas). "mockbuild" perhaps? I appreciate that this will be a pain. "import mockbuild as mock" could be a way to ease the transition.
if the mock -> mockbuild namespace change is agreed upon, I could find some time to produce a patch
This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component.
It would be a very satisfying resolution of this if Dave Malcolm's suggestion to rename the chroot-provisioning tool to "mockbuild" were accepted and Gianluca Sforna's offer to implement it were accepted.
That sounds like the best plan. I Jesse told me over IRC that he and Clark talked about it and it sounds reasonable. However, the work will need to be done on more than just the mock package. Some of the packages that depend mock are importing the mock module, not just calling /usr/bin/mock. $ repoquery -q --whatrequires mock --alldeps mock-0:1.1.9-1.fc14.noarch fedora-packager-0:0.5.9.0-1.fc14.noarch koji-builder-0:1.6.0-1.fc14.noarch mock-rpmfusion-free-0:14.0-1.fc14.noarch mock-rpmfusion-nonfree-0:14.0-1.fc14.noarch plague-builder-0:0.4.5.7-9.20100505cvs.fc14.noarch puritan-0:0.4-6.fc14.noarch revisor-mock-0:2.2-2.fc14.noarch rpmfusion-packager-0:0.4-1.fc12.noarch Some of those may just be calling the /usr/bin/mock binary. * fedpkg (not listed here -- the dependency needs to be added to our fedpkg package) does that. * autoqa (not packaged for Fedora, just used by the QA SIG) does that as well. I would suggest grepping for mock in the .py files for those packages and then checking that none of the occurrences of mock are an import. This is also probably not a change that we'd be able to push to older Fedora (and possibly RHEL/EPEL) so we'll need to figure out how to deal with those separately. (Fedora, we can let expire on its own... RHEL will take some thought).
I'm currently working on the controlling tty issues and have a 1.1.10/1.0.17 release planned. Once that's out I'll accept patches for changing the directory of mock in site-packages to be mockbuild. Whoever does it will need to coordinate with the dependent packages listed above.
The Tahoe-LAFS project has a buildbot herd that includes a Fedora buildslave (operated by Ruben Kerkhof). That buildslave is currently red due to this issue, and will presumably go green as soon as Ruben deploys the fix to this issue onto his buildslave. So, if you want to confirm whether the fix works for Tahoe-LAFS's purposes, just watch this space: http://tahoe-lafs.org/buildbot/builders/Ruben%20Fedora Thanks!
I just finished checking the packages listed by repoquery: NEEDSWORK: * mock * revisor-mock OK, configuration files for mock * mock-rpmfusion-free-0:15.0-1.fc14.noarch * mock-rpmfusion-nonfree-0:15.0-1.fc14.noarch OK, no apparent mock usage (maybe called through another executable?) * rpmfusion-packager-0:0.4-1.fc12.noarch * fedora-packager-0:0.5.9.2-1.fc14.noarch OK, just calls the mock binary * puritan * plague-builder-0:0.4.5.7-9.20100505cvs.fc14.noarch * koji-builder-0:1.6.0-1.fc14.noarch
A patch for Fedora's mock was submitted: https://fedorahosted.org/mock/ticket/16
And here is the one for revisor: https://fedorahosted.org/revisor/ticket/373
Clark: ping? This is a good point in the cycle to get this applied in time for Fedora 17. I can even apply the patch and push a new rpm package to rawhide if you are ready to deal with fallout of that. Another option is to prep a new upstream release now and push it into Fedora once F16 is final -- that way we'd have more releng people who can help to deal with problems that may arise. If we do nothing, I fear that this will continue into the next release cycle... and the next.. and the[..] :-)
Pushing this with mock-1.1.13
mock-1.1.13-1.el6 has been submitted as an update for Fedora EPEL 6. https://admin.fedoraproject.org/updates/mock-1.1.13-1.el6
mock-1.0.20-1.el5 has been submitted as an update for Fedora EPEL 5. https://admin.fedoraproject.org/updates/mock-1.0.20-1.el5
mock-1.1.13-1.fc15 has been submitted as an update for Fedora 15. https://admin.fedoraproject.org/updates/mock-1.1.13-1.fc15
mock-1.1.13-1.fc14 has been submitted as an update for Fedora 14. https://admin.fedoraproject.org/updates/mock-1.1.13-1.fc14
Package mock-1.1.13-1.el6: * should fix your issue, * was pushed to the Fedora EPEL 6 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=epel-testing mock-1.1.13-1.el6' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/mock-1.1.13-1.el6 then log in and leave karma (feedback).
Hooray! Ruben's Fedora buildslave for Tahoe-LAFS is green now! http://tahoe-lafs.org/buildbot/waterfall?show=Ruben%20Fedora
> mock-1.1.13-1.el6 has been submitted as an update for Fedora EPEL 6. Unfortunately this is broken on RHEL 6: localhost:~> rpm -q mock mock-1.1.13-1.el6.noarch localhost:~> mock -r rhel-6-x86_64 /tmp/test.rpm Traceback (most recent call last): File "/usr/sbin/mock", line 64, in <module> import mockbuild.exception ImportError: No module named mockbuild.exception zsh: exit 1 mock -r rhel-6-x86_64 /tmp/test.rpm localhost:~>
(In reply to comment #32) > > mock-1.1.13-1.el6 has been submitted as an update for Fedora EPEL 6. > > Unfortunately this is broken on RHEL 6: > > localhost:~> rpm -q mock > mock-1.1.13-1.el6.noarch > localhost:~> mock -r rhel-6-x86_64 /tmp/test.rpm > Traceback (most recent call last): > File "/usr/sbin/mock", line 64, in <module> > import mockbuild.exception > ImportError: No module named mockbuild.exception > zsh: exit 1 mock -r rhel-6-x86_64 /tmp/test.rpm > localhost:~> Hmmm, is there a mockbuild directory in /usr/lib/python2.6/site-packages?
> Hmmm, is there a mockbuild directory in /usr/lib/python2.6/site-packages? Nope: localhost:~> rpm -V mock S.5....T. c /etc/mock/site-defaults.cfg zsh: exit 1 rpm -V mock localhost:~> rpm -ql mock | grep site-packages /usr/lib/python2.6/site-packages/mock /usr/lib/python2.6/site-packages/mock/__init__.py /usr/lib/python2.6/site-packages/mock/__init__.pyc /usr/lib/python2.6/site-packages/mock/__init__.pyo /usr/lib/python2.6/site-packages/mock/backend.py /usr/lib/python2.6/site-packages/mock/backend.pyc /usr/lib/python2.6/site-packages/mock/backend.pyo ...
I notice that this patch wasn't completely applied: https://fedorahosted.org/mock/attachment/ticket/16/0003-Update-install-paths.patch and that's where the module paths to be installed will be defined.
Created attachment 522368 [details] patch to install into the proper location Documentation on automake's pkgpython variable here: http://www.gnu.org/software/automake/manual/html_node/Python.html#Python I agree with gaillu's approach to this. Attaching a patch that applies his changes. This gets the mock module installed into mockbuild correctly. I'm preparing a patch for a second problem with default plugin paths still looking in the python_sitelib/mock/ directory.
Created attachment 522369 [details] Fix install path of mockbuild module and default path to module dir Updated the previous patch instead -- it turns out that the old path was being substituted via the Makefile.am. Updated that section to substitute the new path instead. Remember to run autoreconf -fi afterwards :-)
mock-1.1.14-1.fc14 has been submitted as an update for Fedora 14. https://admin.fedoraproject.org/updates/mock-1.1.14-1.fc14
mock-1.0.21-1.el5 has been submitted as an update for Fedora EPEL 5. https://admin.fedoraproject.org/updates/mock-1.0.21-1.el5
mock-1.1.14-1.fc15 has been submitted as an update for Fedora 15. https://admin.fedoraproject.org/updates/mock-1.1.14-1.fc15
mock-1.1.14-1.el6 has been submitted as an update for Fedora EPEL 6. https://admin.fedoraproject.org/updates/mock-1.1.14-1.el6
Package mock-1.1.14-1.el6: * should fix your issue, * was pushed to the Fedora EPEL 6 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=epel-testing mock-1.1.14-1.el6' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/mock-1.1.14-1.el6 then log in and leave karma (feedback).
> mock-1.1.14-1.el6 has been submitted as an update for Fedora EPEL 6. This one works as expected, thanks!
mock-1.1.14-1.fc15 has been pushed to the Fedora 15 stable repository. If problems still persist, please make note of it in this bug report.
mock-1.1.14-1.fc14 has been pushed to the Fedora 14 stable repository. If problems still persist, please make note of it in this bug report.
mock-1.1.15-1.fc15 has been submitted as an update for Fedora 15. https://admin.fedoraproject.org/updates/mock-1.1.15-1.fc15
mock-1.1.15-1.el6 has been submitted as an update for Fedora EPEL 6. https://admin.fedoraproject.org/updates/mock-1.1.15-1.el6
mock-1.0.22-1.el5 has been submitted as an update for Fedora EPEL 5. https://admin.fedoraproject.org/updates/mock-1.0.22-1.el5
mock-1.1.15-1.fc14 has been submitted as an update for Fedora 14. https://admin.fedoraproject.org/updates/mock-1.1.15-1.fc14
mock-1.1.15-1.fc15 has been pushed to the Fedora 15 stable repository. If problems still persist, please make note of it in this bug report.
mock-1.1.15-1.fc14 has been pushed to the Fedora 14 stable repository. If problems still persist, please make note of it in this bug report.
mock-1.0.22-1.el5 has been pushed to the Fedora EPEL 5 stable repository. If problems still persist, please make note of it in this bug report.
mock-1.1.15-1.el6 has been pushed to the Fedora EPEL 6 stable repository. If problems still persist, please make note of it in this bug report.