$ ls -R /usr/lib/python3.4/ /usr/lib/python3.4/: site-packages /usr/lib/python3.4/site-packages: dnf-plugins seobject /usr/lib/python3.4/site-packages/dnf-plugins: __pycache__ /usr/lib/python3.4/site-packages/dnf-plugins/__pycache__: builddep.cpython-34.pyc debuginfo-install.cpython-34.pyc needs_restarting.cpython-34.pyc repoquery.cpython-34.pyc config_manager.cpython-34.pyc download.cpython-34.pyc noroot.cpython-34.pyc reposync.cpython-34.pyc copr.cpython-34.pyc generate_completion_cache.cpython-34.pyc protected_packages.cpython-34.pyc /usr/lib/python3.4/site-packages/seobject: __pycache__ /usr/lib/python3.4/site-packages/seobject/__pycache__: $ ls -R /usr/lib/python3.5/ /usr/lib/python3.5/: site-packages /usr/lib/python3.5/site-packages: dnf-plugins isc /usr/lib/python3.5/site-packages/dnf-plugins: __pycache__ /usr/lib/python3.5/site-packages/dnf-plugins/__pycache__: builddep.cpython-35.pyc debuginfo-install.cpython-35.pyc needs_restarting.cpython-35.pyc repoquery.cpython-35.pyc config_manager.cpython-35.pyc download.cpython-35.pyc noroot.cpython-35.pyc reposync.cpython-35.pyc copr.cpython-35.pyc generate_completion_cache.cpython-35.pyc protected_packages.cpython-35.pyc /usr/lib/python3.5/site-packages/isc: __pycache__ /usr/lib/python3.5/site-packages/isc/__pycache__:
This is usually because some packages create files there that are not owned by those packages. See for example here: $ rpm -ql python3-bind /usr/lib/python3.6/site-packages/isc-2.0-py3.6.egg-info /usr/lib/python3.6/site-packages/isc/__init__.py /usr/lib/python3.6/site-packages/isc/__pycache__/__init__.cpython-36.opt-1.pyc /usr/lib/python3.6/site-packages/isc/__pycache__/__init__.cpython-36.pyc ... /usr/lib/python3.6/site-packages/isc/__pycache__/utils.cpython-36.opt-1.pyc /usr/lib/python3.6/site-packages/isc/__pycache__/utils.cpython-36.pyc /usr/lib/python3.6/site-packages/isc/checkds.py ... /usr/lib/python3.6/site-packages/isc/utils.py The package does not own either /usr/lib/python3.6/site-packages/isc or /usr/lib/python3.6/site-packages/isc/__pycache__. Now this is an error and it has to be fixed, but there is probably no way to fix it for the existing 3.4 and 3.5 files, because that would require issuing an update to already EOLed Fedora versions.
None of the files in my original bug report belong to any installed packages - I verified that using `rpm -qf` I do understand this bug report is impossible to fix for previous deprecated Python releases but it makes sense to resolve it for future Python versions.
Those are some directories (installed on my system) with similar issues: $ LANG=C.utf8 find /usr/lib/python3.6/site-packages/ -type d -exec rpm -qf {} \; | grep -v locale | grep "is not owned by any package" file /usr/lib/python3.6/site-packages/sphinxcontrib is not owned by any package file /usr/lib/python3.6/site-packages/nbconvert is not owned by any package file /usr/lib/python3.6/site-packages/initial_setup is not owned by any package file /usr/lib/python3.6/site-packages/initial_setup/gui is not owned by any package file /usr/lib/python3.6/site-packages/pykickstart is not owned by any package file /usr/lib/python3.6/site-packages/pykickstart/commands is not owned by any package file /usr/lib/python3.6/site-packages/pykickstart/handlers is not owned by any package file /usr/lib/python3.6/site-packages/isc is not owned by any package file /usr/lib/python3.6/site-packages/isc/__pycache__ is not owned by any package file /usr/lib/python3.6/site-packages/sepolicy is not owned by any package $ LANG=C.utf8 find /usr/lib64/python3.6/site-packages/ -type d -exec rpm -qf {} \; | grep -v locale | grep "is not owned by any package" file /usr/lib64/python3.6/site-packages/pyanaconda is not owned by any package file /usr/lib64/python3.6/site-packages/pyanaconda/__pycache__ is not owned by any package file /usr/lib64/python3.6/site-packages/pyanaconda/ui is not owned by any package file /usr/lib64/python3.6/site-packages/pyanaconda/ui/tui is not owned by any package file /usr/lib64/python3.6/site-packages/report is not owned by any package file /usr/lib64/python3.6/site-packages/reportclient is not owned by any package I think we might need to create a scrip that: * installs all python3 packages (to a container or mock, etc.) * does the above queries * figures what packages have files in those dirs CCing Python SIG. dnf-plugins pycs are now properly owned: $ rpm -qf /usr/lib/python3.6/site-packages/dnf-plugins/__pycache__ python3-dnf-plugins-extras-common-2.0.3-1.fc27.noarch python3-dnf-2.7.5-1.fc27.noarch $ rpm -qf /usr/lib/python3.6/site-packages/dnf-plugins/__pycache__/*.pyc python3-dnf-plugins-core-2.1.5-1.fc27.noarch ... python3-dnf-plugins-core-2.1.5-1.fc27.noarch python3-dnf-plugin-system-upgrade-2.0.3-1.fc27.noarch python3-dnf-plugin-system-upgrade-2.0.3-1.fc27.noarch I don't seem to be able to figure out where seobject comes from.
Not sure that script would work – are the files created by just installing the RPM? seobject is probably from SELinux, judging by the name.
(In reply to Petr Viktorin from comment #4) > Not sure that script would work – are the files created by just installing > the RPM? Yes. It's a directory where the files owned by the package are, i.e. see here from nbconvert %files section (irrelevant parts removed): %files -n python2-%{pypi_name} %{python2_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info %{python2_sitelib}/%{pypi_name}/* %files -n python3-%{pypi_name} %{python3_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info %{python3_sitelib}/%{pypi_name}/* The %{pythonX_sitelib}/%{pypi_name} directories are not owned by the packages, only their contents. RPM will create the directory when installing, but it will remain there when the package is removed.
I am installing all packages with python3 in the name and I'll provide a list of directories with the same problem.
For each such directory, you want to trigger rpm -qf for some file in it (i.e. for __init__.py or even for * if you sort and uniq the results). That will give you the actual package that should own the directory.
Created attachment 1376217 [details] directories not owned by package I finished the investigation. Procedure: $ find /usr/lib*/python3.6/site-packages/ -type d -exec rpm -qf {} \; | grep -v locale | grep "is not owned by any package" > not_owned.txt $ for folder in `cat not_owned.txt | cut -d " " -f 2`; do echo -n "$folder - "; rpm -qf `find $folder -type f | head -n 1` ; done The result of the second command is attached. It found 47 folders which need to be fixed. I also fixed one of them and fill a PR: https://src.fedoraproject.org/rpms/python-flask-babel/pull-request/1 A question is whether we want to fix them manually, create a script to fix them automatically or just open bugs or send an email to package owners.
I think that it'd be most efficient if you could file PRs for all of them. Just filing bugs is as much work, and with PRs we're almost done ;) I'd try to just "edit" the list into a series of sed -i invocations to mangle the spec files.
I've fixed all 47 directories in 26 packages, scratch-built them, tested a new version in mock and filled PRs. Some of PRs are merged already. I've also discovered some FTBFS bugs, so I'll try to fix them or at least create a bugs. Will we continue somehow and, for example, create some automatic check for this issue?
taskotron check that tests directory not owned by any package makes sense (it is not only Python related). maybe put it in rpmgrill (or not)? talk to Kamil Páral or Martin Křížek, they might have some idea where to implement this. CC'ed.
I think this check could be a good fit for either rpmlint or rpmgrill. I can't really tell which project is better for this, talk to the project owners or look how they're structured internally (I don't know.) We already execute these tasks in Taskotron, so finding out offending packages for this particular problem might be easy in the future (because some package maintainers will ignore the warnings).
Lumír, what's the status? Do you need any help?
There are still around 10 PR waiting for review and merge. We can wait one more week and then some proven packager could merge them. I didn't have enough time to take a look at rpmlint/rpmgrill but I'm going to do it. Does it have a high priority?
No, it's definitely not high priority. Just checking the status.
I've implemented the check in the rpmlint tool: https://github.com/rpm-software-management/rpmlint/pull/131 If/when it'll appear in upstream, I can modify taskotron to install packages before rpmlint check.
As you can see in the mentioned pull request, after some discussion the proposed check for directory ownership was rejected. The reason is that tests of rpmlint itself don't count with testing installed RPMs and it seems that nobody uses this feature even is available and supported. Also, there are no taskotron checks which check installed RPMs. I see two possible ways how to do it: 1. Implement the same functionality directly in taskotron. It would be quite simple but the problem here is that taskotron is running checks on a minimal system and therefore installing RPM and its dependencies before every check could take a lot of time. Kamil Páral warned me that this check could be disabled if it would consume too many resources. 2. It might be possible to implement the same check without installing RPM package with just checking a package content but I am afraid that it can be really complex and we might end with the same issues mentioned in point 1. A decision has to be made here and I am not sure that it worth to spend a lot of time on it. [0] https://github.com/rpm-software-management/rpmlint/pull/131
Note that Fedora-Review runs check on installed RPMS. On the other hand I think it already has this check.
Lumír, I think we can't co anything here. Should we close it as WONTFIX or NOTABUG?
You are right, closing as WONTFIX.