Bug 2054951
| Summary: | RPM Python package doesn't provide python3dist(opencv-python). | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | panzer8.iasm8 |
| Component: | opencv | Assignee: | Nicolas Chauvet (kwizart) <kwizart> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | low | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 35 | CC: | andrew, hhorak, jkucera, jmlich83, jridky, karlthered, kwizart, lx, mhroncok, rakesh.pandit, sergio, viktor.vix.jancik |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-10-14 13:51:26 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
It's a bit harder than I thought. The wheel is build from https://github.com/opencv/opencv-python it generate 4 different packages: * opencv-python * opencv-contrib-python * opencv-python-headless * opencv-contrib-python-headless ok , but python3-opencv for openv package jjust have one file [1] , so I guess is not opencv package fault [1] /usr/lib64/python3.9/site-packages/cv2.cpython-39-x86_64-linux-gnu.so I compared file names in opencv-python and opencv-contrib-python. They are almost identical if not for a folder name. Moreover most of those files are Qt libs and ttf. Considering they included opencv as submodule in opencv-python to build the wheel, it is unfair to expect the RPM to match Wheel package. In theory we should build https://github.com/opencv/opencv-python; but compiling opencv 4 times to match Pypi packages would be an overkill. That goes without saying that /usr/lib64/python3.9/site-packages/cv2.cpython-39-x86_64-linux-gnu.so works! I'm very new to this so I may doing this wrong. I was following Fedora guidelines to package mediapipe (which depends on opencv). The RPM Requires, Provides are automatically generated from dist-info/ and setup.cfg and it resolves to: python3dist(opencv-python) or python3dist(opencv-contrib-python) Then it never resolves despite manually installing python3-opencv My current solution is to have an empty RPM with: Requires: python3-opencv Provides: python3-opencv-python Provides: python3-opencv-python-headless Provides: python3-opencv-contrib-python Provides: python3-opencv-contrib-python-headless Provides: python3.10-opencv-python Provides: python3.10-opencv-python-headless Provides: python3.10-opencv-contrib-python Provides: python3.10-opencv-contrib-python-headless Provides: python3.10dist(opencv-python) Provides: python3.10dist(opencv-python-headless) Provides: python3.10dist(opencv-contrib-python) Provides: python3.10dist(opencv-contrib-python-headless) Provides: python3dist(opencv-python) Provides: python3dist(opencv-python-headless) Provides: python3dist(opencv-contrib-python) Provides: python3dist(opencv-contrib-python-headless) Hi, I think , you may contact https://github.com/opencv/opencv-python , and ask how we add the "wheel" to our packages , For what I understood , he provides all open pré-compiled opencv which we don't need or even want. I did projects with Fedora python3-opencv "python import cv2 " and we have access to all opencv , just maybe we don't export the "wheel" ... if you understand what I mean Hi Sergio, how do you package your projects to use python3-opencv ? you must write your Requires manually instead of using %pyproject_buildrequires, right? Indeed python3-opencv looks more sane than the wheel. If it is functionally equivalent, then we add those "provides" to the spec. This way dnf resolves python3dist(opencv-python) to python3-opencv pull requests are welcomed (In reply to panzer8.iasm8 from comment #5) > how do you package your projects to use python3-opencv ? > you must write your Requires manually instead of using > %pyproject_buildrequires, right? Almost right. You add the requirement manually and you patch/sed out the requirement on opencv-python/opencv-python-headless/opencv-contrib-python/opencv-contrib-python-headless out of upstream metadata (e.g. in %prep). Then you can use %pyproject_buildrequires. > Indeed python3-opencv looks more sane than the wheel. If it is functionally > equivalent, then we add those "provides" to the spec. > This way dnf resolves python3dist(opencv-python) to python3-opencv That, however, is really dangerous and forbidden. Python tools that read Python metadata and search for "opencv-python" would fail to find it. E.g. the %pyproject_buildrequires implementation actually reads the Python metadata to determine if something is installed or not. In this case, it could lead to this hard to debug bug: 1. %pyproject_buildrequires looks for opencv-python installation and does not find it 2. %pyproject_buildrequires instructs dnf to install python3dist(opencv-python) 3. dnf installs python3-opencv 4. %pyproject_buildrequires looks for opencv-python installation and still does not find it 5. %pyproject_buildrequires instructs dnf to install python3dist(opencv-python) (again) 6. dnf reports that there is nothing new to install 7. %generate_buildrequires is considered finished, despite more dependencies might be missing The only possibly-sane way to add python3dist(opencv-python) provide manually is to craft the Python metadata according to https://packaging.python.org/en/latest/specifications/core-metadata/ -- see for example the python-tomli Fedora package that does that in %build when built int he bootstrap mode. > The only possibly-sane way to add python3dist(opencv-python) provide manually is to craft the Python metadata according to https://packaging.python.org/en/latest/specifications/core-metadata/ -- see for example the python-tomli Fedora package that does that in %build when built int he bootstrap mode.
However, if you decide to do this, you must assure that the python3-opencv package provides all the APIs that the opencv-python package does. Otherwise other packages might blow up.
> Almost right. You add the requirement manually and you patch/sed out the requirement on opencv-python/opencv-python-headless/opencv-contrib-python/opencv-contrib-python-headless out of upstream metadata (e.g. in %prep). Then you can use %pyproject_buildrequires. Thank you Miro, It is the answer I was looking for. I couldn't have been the first bumping into this, but I didn't find anything online. Feel free to close this ticket, as it seems to have a well known solution among experienced packagers. > 4. %pyproject_buildrequires looks for opencv-python installation and still does not find it Alright, I assumed it was using rpm's provides to find it's dependencies/ Hi, can ypu test my copr repo ? dnf copr enable sergiomb/opencv https://copr.fedorainfracloud.org/coprs/sergiomb/opencv/build/3792271/ I added to %build pushd modules/python/package %py3_build popd I added to %install pushd modules/python/package %py3_install popd and in %files -n python3-opencv %{python3_sitelib}/cv2 %{python3_sitelib}/opencv-*.egg-info $ rpm -qlp python3-opencv-4.5.5-7.fc37.x86_64.rpm /usr/lib/.build-id /usr/lib/.build-id/a4 /usr/lib/.build-id/a4/a5723d4e9b2871e23f0e3213bcd9f1c3ff0fe4 /usr/lib/python3.10/site-packages/cv2 /usr/lib/python3.10/site-packages/cv2/__init__.py /usr/lib/python3.10/site-packages/cv2/__pycache__ /usr/lib/python3.10/site-packages/cv2/__pycache__/__init__.cpython-310.opt-1.pyc /usr/lib/python3.10/site-packages/cv2/__pycache__/__init__.cpython-310.pyc /usr/lib/python3.10/site-packages/cv2/__pycache__/load_config_py2.cpython-310.opt-1.pyc /usr/lib/python3.10/site-packages/cv2/__pycache__/load_config_py2.cpython-310.pyc /usr/lib/python3.10/site-packages/cv2/__pycache__/load_config_py3.cpython-310.opt-1.pyc /usr/lib/python3.10/site-packages/cv2/__pycache__/load_config_py3.cpython-310.pyc /usr/lib/python3.10/site-packages/cv2/load_config_py2.py /usr/lib/python3.10/site-packages/cv2/load_config_py3.py /usr/lib/python3.10/site-packages/opencv-4.5.5-py3.10.egg-info /usr/lib/python3.10/site-packages/opencv-4.5.5-py3.10.egg-info/PKG-INFO /usr/lib/python3.10/site-packages/opencv-4.5.5-py3.10.egg-info/SOURCES.txt /usr/lib/python3.10/site-packages/opencv-4.5.5-py3.10.egg-info/dependency_links.txt /usr/lib/python3.10/site-packages/opencv-4.5.5-py3.10.egg-info/requires.txt /usr/lib/python3.10/site-packages/opencv-4.5.5-py3.10.egg-info/top_level.txt /usr/lib64/python3.10/site-packages/cv2.cpython-310-x86_64-linux-gnu.so $ rpm -qPp python3-opencv-4.5.5-7.fc37.x86_64.rpm python-opencv = 4.5.5-7.fc37 python3-opencv = 4.5.5-7.fc37 python3-opencv(x86-64) = 4.5.5-7.fc37 python3.10-opencv = 4.5.5-7.fc37 python3.10dist(opencv) = 4.5.5 python3dist(opencv) = 4.5.5 Considering https://pypi.org/project/opencv/ does not exist, I don't know if this helps at all. Also, the mixture of /lib/ and /lib64/ modules worries me a little bit. `cv2` seems to be both an extension module and importable package. Hi, python3-opencv package, as is , only have cv2.cpython-39-x86_64-linux-gnu.so [1] , this cv2.cpython-39-x86_64-linux-gnu.so is "linked" to all opencv [2] reporter need the package provides python3.10dist(opencv-python) for that we need egg-info . 1 - opencv python bindings can't be easily put in pypi.org 2 - seems upstream misses the python wheels , I think we should report it upstream , but I don't know what to write . after review what I wrote, https://pypi.org/project/opencv-python/ exist and https://github.com/opencv/opencv-python is part of opencv organization , what you suggest ? [1] rpm -q python3-opencv -l /usr/lib/.build-id /usr/lib/.build-id/78 /usr/lib/.build-id/78/702acc5d694370cd8bec6bc7e34142cb0325ed /usr/lib64/python3.9/site-packages/cv2.cpython-39-x86_64-linux-gnu.so [2] ldd /usr/lib64/python3.9/site-packages/cv2.cpython-39-x86_64-linux-gnu.so | grep opencv libopencv_hdf.so.4.5 => /lib64/libopencv_hdf.so.4.5 (0x00007f084a329000) libopencv_intensity_transform.so.4.5 => /lib64/libopencv_intensity_transform.so.4.5 (0x00007f084a303000) libopencv_reg.so.4.5 => /lib64/libopencv_reg.so.4.5 (0x00007f084a2e9000) libopencv_surface_matching.so.4.5 => /lib64/libopencv_surface_matching.so.4.5 (0x00007f084a29c000) libopencv_xphoto.so.4.5 => /lib64/libopencv_xphoto.so.4.5 (0x00007f084a258000) libopencv_alphamat.so.4.5 => /lib64/libopencv_alphamat.so.4.5 (0x00007f084a228000) libopencv_dnn_superres.so.4.5 => /lib64/libopencv_dnn_superres.so.4.5 (0x00007f084a216000) libopencv_freetype.so.4.5 => /lib64/libopencv_freetype.so.4.5 (0x00007f084a20b000) libopencv_fuzzy.so.4.5 => /lib64/libopencv_fuzzy.so.4.5 (0x00007f084a1f9000) libopencv_hfs.so.4.5 => /lib64/libopencv_hfs.so.4.5 (0x00007f084a1e5000) libopencv_img_hash.so.4.5 => /lib64/libopencv_img_hash.so.4.5 (0x00007f084a1cf000) libopencv_line_descriptor.so.4.5 => /lib64/libopencv_line_descriptor.so.4.5 (0x00007f084a1a6000) libopencv_saliency.so.4.5 => /lib64/libopencv_saliency.so.4.5 (0x00007f084a17f000) libopencv_wechat_qrcode.so.4.5 => /lib64/libopencv_wechat_qrcode.so.4.5 (0x00007f084a116000) libopencv_mcc.so.4.5 => /lib64/libopencv_mcc.so.4.5 (0x00007f084a0ad000) libopencv_rapid.so.4.5 => /lib64/libopencv_rapid.so.4.5 (0x00007f084a09b000) libopencv_rgbd.so.4.5 => /lib64/libopencv_rgbd.so.4.5 (0x00007f0849fc8000) libopencv_shape.so.4.5 => /lib64/libopencv_shape.so.4.5 (0x00007f0849f9f000) libopencv_stitching.so.4.5 => /lib64/libopencv_stitching.so.4.5 (0x00007f0849ef9000) libopencv_structured_light.so.4.5 => /lib64/libopencv_structured_light.so.4.5 (0x00007f0849ee4000) libopencv_aruco.so.4.5 => /lib64/libopencv_aruco.so.4.5 (0x00007f0849e82000) libopencv_bgsegm.so.4.5 => /lib64/libopencv_bgsegm.so.4.5 (0x00007f0849e65000) libopencv_bioinspired.so.4.5 => /lib64/libopencv_bioinspired.so.4.5 (0x00007f0849e2b000) libopencv_ccalib.so.4.5 => /lib64/libopencv_ccalib.so.4.5 (0x00007f0849dd0000) libopencv_face.so.4.5 => /lib64/libopencv_face.so.4.5 (0x00007f0849d4b000) libopencv_gapi.so.4.5 => /lib64/libopencv_gapi.so.4.5 (0x00007f0849a0b000) libopencv_stereo.so.4.5 => /lib64/libopencv_stereo.so.4.5 (0x00007f08499e7000) libopencv_quality.so.4.5 => /lib64/libopencv_quality.so.4.5 (0x00007f08499d4000) libopencv_phase_unwrapping.so.4.5 => /lib64/libopencv_phase_unwrapping.so.4.5 (0x00007f08499c6000) libopencv_viz.so.4.5 => /lib64/libopencv_viz.so.4.5 (0x00007f0849963000) libopencv_photo.so.4.5 => /lib64/libopencv_photo.so.4.5 (0x00007f08498c3000) libopencv_objdetect.so.4.5 => /lib64/libopencv_objdetect.so.4.5 (0x00007f084983e000) libopencv_optflow.so.4.5 => /lib64/libopencv_optflow.so.4.5 (0x00007f08497da000) libopencv_ximgproc.so.4.5 => /lib64/libopencv_ximgproc.so.4.5 (0x00007f084959e000) libopencv_tracking.so.4.5 => /lib64/libopencv_tracking.so.4.5 (0x00007f08493a5000) libopencv_plot.so.4.5 => /lib64/libopencv_plot.so.4.5 (0x00007f084939b000) libopencv_text.so.4.5 => /lib64/libopencv_text.so.4.5 (0x00007f084932d000) libopencv_ml.so.4.5 => /lib64/libopencv_ml.so.4.5 (0x00007f08492a1000) libopencv_highgui.so.4.5 => /lib64/libopencv_highgui.so.4.5 (0x00007f0849257000) libopencv_videoio.so.4.5 => /lib64/libopencv_videoio.so.4.5 (0x00007f08491cf000) libopencv_imgcodecs.so.4.5 => /lib64/libopencv_imgcodecs.so.4.5 (0x00007f084916d000) libopencv_video.so.4.5 => /lib64/libopencv_video.so.4.5 (0x00007f08490ef000) libopencv_dnn.so.4.5 => /lib64/libopencv_dnn.so.4.5 (0x00007f0848cfe000) libopencv_calib3d.so.4.5 => /lib64/libopencv_calib3d.so.4.5 (0x00007f0848ad9000) libopencv_features2d.so.4.5 => /lib64/libopencv_features2d.so.4.5 (0x00007f0848a0e000) libopencv_flann.so.4.5 => /lib64/libopencv_flann.so.4.5 (0x00007f08489a4000) libopencv_imgproc.so.4.5 => /lib64/libopencv_imgproc.so.4.5 (0x00007f0848455000) libopencv_core.so.4.5 => /lib64/libopencv_core.so.4.5 (0x00007f08480da000) >> Almost right. You add the requirement manually and you patch/sed out the requirement on opencv-python/opencv-python-headless/opencv-contrib-python/opencv-contrib-python-headless out of upstream metadata (e.g. in %prep). Then you can use %pyproject_buildrequires. >Thank you Miro, It is the answer I was looking for. I couldn't have been the first bumping into this, but I didn't find anything online. >Feel free to close this ticket, as it seems to have a well known solution among experienced packagers. Closing this ticket as it seems there is a mix between python3-opencv from the opencv github tree and opencv-python which is a different code base... Hi, sorry, I thought that I added a comment to this bug report at around 2022-04-14, where I switch the build of opencv to new Python loader [2] and is now available on F37 ( Beta ) the result is [1] [1] dnf --disablerepo='*' --release=37 --enablerepo=fedora repoquery --available -l python3-opencv /usr/lib/.build-id /usr/lib/.build-id/57 /usr/lib/.build-id/57/afe2df0f9171d3fa9c576c7a116416c82770a6 /usr/lib64/python3.11/site-packages/cv2 /usr/lib64/python3.11/site-packages/cv2/__init__.py /usr/lib64/python3.11/site-packages/cv2/__pycache__ /usr/lib64/python3.11/site-packages/cv2/__pycache__/__init__.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/__init__.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/config-3.11.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/config-3.11.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/config.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/config.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/load_config_py2.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/load_config_py2.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/load_config_py3.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/load_config_py3.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/config-3.11.py /usr/lib64/python3.11/site-packages/cv2/config.py /usr/lib64/python3.11/site-packages/cv2/cv2.cpython-311-x86_64-linux-gnu.so /usr/lib64/python3.11/site-packages/cv2/gapi /usr/lib64/python3.11/site-packages/cv2/gapi/__init__.py /usr/lib64/python3.11/site-packages/cv2/gapi/__pycache__ /usr/lib64/python3.11/site-packages/cv2/gapi/__pycache__/__init__.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/gapi/__pycache__/__init__.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/load_config_py2.py /usr/lib64/python3.11/site-packages/cv2/load_config_py3.py /usr/lib64/python3.11/site-packages/cv2/mat_wrapper /usr/lib64/python3.11/site-packages/cv2/mat_wrapper/__init__.py /usr/lib64/python3.11/site-packages/cv2/mat_wrapper/__pycache__ /usr/lib64/python3.11/site-packages/cv2/mat_wrapper/__pycache__/__init__.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/mat_wrapper/__pycache__/__init__.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/misc /usr/lib64/python3.11/site-packages/cv2/misc/__init__.py /usr/lib64/python3.11/site-packages/cv2/misc/__pycache__ /usr/lib64/python3.11/site-packages/cv2/misc/__pycache__/__init__.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/misc/__pycache__/__init__.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/misc/__pycache__/version.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/misc/__pycache__/version.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/misc/version.py /usr/lib64/python3.11/site-packages/cv2/utils /usr/lib64/python3.11/site-packages/cv2/utils/__init__.py /usr/lib64/python3.11/site-packages/cv2/utils/__pycache__ /usr/lib64/python3.11/site-packages/cv2/utils/__pycache__/__init__.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/utils/__pycache__/__init__.cpython-311.pyc [2] https://src.fedoraproject.org/rpms/opencv/c/feee2b1ca875ab0d3c8fb8a83e8f5f310c28d809 |
Description of problem: I'm exploring how to make a RPM package for Python application. This application depends on opencv-python which is provided in Fedora by python3-opencv. however the RPM marcro %pyproject_buildrequires Version-Release number of selected component (if applicable): How reproducible: Any Python project with dependency on opencv-python Steps to Reproduce: 1. Given an empty Python project with setup.py: from setuptools import setup setup( name='mypackage', version='0.0.1', packages=['mypackage'], install_requires=[ 'opencv-python', ], ) 2. Create a spec file according to https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/ 3. Build Actual results: When I try to install - nothing provides python3.10dist(opencv-python) >= 4.4.0.46 needed by Expected results: The macro should match the correct Fedora package Additional info: I compared with python3-numpy package/spec. From my understanding is that python3-opencv doesn't provide python3dist(opencv-python).