Description of problem: I'm exploring how to make a RPM package for Python application. This application depends on opencv-python which is provided in Fedora by python3-opencv. however the RPM marcro %pyproject_buildrequires Version-Release number of selected component (if applicable): How reproducible: Any Python project with dependency on opencv-python Steps to Reproduce: 1. Given an empty Python project with setup.py: from setuptools import setup setup( name='mypackage', version='0.0.1', packages=['mypackage'], install_requires=[ 'opencv-python', ], ) 2. Create a spec file according to https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/ 3. Build Actual results: When I try to install - nothing provides python3.10dist(opencv-python) >= 4.4.0.46 needed by Expected results: The macro should match the correct Fedora package Additional info: I compared with python3-numpy package/spec. From my understanding is that python3-opencv doesn't provide python3dist(opencv-python).
It's a bit harder than I thought. The wheel is build from https://github.com/opencv/opencv-python it generate 4 different packages: * opencv-python * opencv-contrib-python * opencv-python-headless * opencv-contrib-python-headless
ok , but python3-opencv for openv package jjust have one file [1] , so I guess is not opencv package fault [1] /usr/lib64/python3.9/site-packages/cv2.cpython-39-x86_64-linux-gnu.so
I compared file names in opencv-python and opencv-contrib-python. They are almost identical if not for a folder name. Moreover most of those files are Qt libs and ttf. Considering they included opencv as submodule in opencv-python to build the wheel, it is unfair to expect the RPM to match Wheel package. In theory we should build https://github.com/opencv/opencv-python; but compiling opencv 4 times to match Pypi packages would be an overkill. That goes without saying that /usr/lib64/python3.9/site-packages/cv2.cpython-39-x86_64-linux-gnu.so works! I'm very new to this so I may doing this wrong. I was following Fedora guidelines to package mediapipe (which depends on opencv). The RPM Requires, Provides are automatically generated from dist-info/ and setup.cfg and it resolves to: python3dist(opencv-python) or python3dist(opencv-contrib-python) Then it never resolves despite manually installing python3-opencv My current solution is to have an empty RPM with: Requires: python3-opencv Provides: python3-opencv-python Provides: python3-opencv-python-headless Provides: python3-opencv-contrib-python Provides: python3-opencv-contrib-python-headless Provides: python3.10-opencv-python Provides: python3.10-opencv-python-headless Provides: python3.10-opencv-contrib-python Provides: python3.10-opencv-contrib-python-headless Provides: python3.10dist(opencv-python) Provides: python3.10dist(opencv-python-headless) Provides: python3.10dist(opencv-contrib-python) Provides: python3.10dist(opencv-contrib-python-headless) Provides: python3dist(opencv-python) Provides: python3dist(opencv-python-headless) Provides: python3dist(opencv-contrib-python) Provides: python3dist(opencv-contrib-python-headless)
Hi, I think , you may contact https://github.com/opencv/opencv-python , and ask how we add the "wheel" to our packages , For what I understood , he provides all open pré-compiled opencv which we don't need or even want. I did projects with Fedora python3-opencv "python import cv2 " and we have access to all opencv , just maybe we don't export the "wheel" ... if you understand what I mean
Hi Sergio, how do you package your projects to use python3-opencv ? you must write your Requires manually instead of using %pyproject_buildrequires, right? Indeed python3-opencv looks more sane than the wheel. If it is functionally equivalent, then we add those "provides" to the spec. This way dnf resolves python3dist(opencv-python) to python3-opencv
https://src.fedoraproject.org/rpms/opencv/blob/rawhide/f/opencv.spec#_215
pull requests are welcomed
See also https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/message/EQ4IJRXWN2VNRNADL5UZDS3WHTAUTZJY/
(In reply to panzer8.iasm8 from comment #5) > how do you package your projects to use python3-opencv ? > you must write your Requires manually instead of using > %pyproject_buildrequires, right? Almost right. You add the requirement manually and you patch/sed out the requirement on opencv-python/opencv-python-headless/opencv-contrib-python/opencv-contrib-python-headless out of upstream metadata (e.g. in %prep). Then you can use %pyproject_buildrequires. > Indeed python3-opencv looks more sane than the wheel. If it is functionally > equivalent, then we add those "provides" to the spec. > This way dnf resolves python3dist(opencv-python) to python3-opencv That, however, is really dangerous and forbidden. Python tools that read Python metadata and search for "opencv-python" would fail to find it. E.g. the %pyproject_buildrequires implementation actually reads the Python metadata to determine if something is installed or not. In this case, it could lead to this hard to debug bug: 1. %pyproject_buildrequires looks for opencv-python installation and does not find it 2. %pyproject_buildrequires instructs dnf to install python3dist(opencv-python) 3. dnf installs python3-opencv 4. %pyproject_buildrequires looks for opencv-python installation and still does not find it 5. %pyproject_buildrequires instructs dnf to install python3dist(opencv-python) (again) 6. dnf reports that there is nothing new to install 7. %generate_buildrequires is considered finished, despite more dependencies might be missing The only possibly-sane way to add python3dist(opencv-python) provide manually is to craft the Python metadata according to https://packaging.python.org/en/latest/specifications/core-metadata/ -- see for example the python-tomli Fedora package that does that in %build when built int he bootstrap mode.
> The only possibly-sane way to add python3dist(opencv-python) provide manually is to craft the Python metadata according to https://packaging.python.org/en/latest/specifications/core-metadata/ -- see for example the python-tomli Fedora package that does that in %build when built int he bootstrap mode. However, if you decide to do this, you must assure that the python3-opencv package provides all the APIs that the opencv-python package does. Otherwise other packages might blow up.
> Almost right. You add the requirement manually and you patch/sed out the requirement on opencv-python/opencv-python-headless/opencv-contrib-python/opencv-contrib-python-headless out of upstream metadata (e.g. in %prep). Then you can use %pyproject_buildrequires. Thank you Miro, It is the answer I was looking for. I couldn't have been the first bumping into this, but I didn't find anything online. Feel free to close this ticket, as it seems to have a well known solution among experienced packagers. > 4. %pyproject_buildrequires looks for opencv-python installation and still does not find it Alright, I assumed it was using rpm's provides to find it's dependencies/
Hi, can ypu test my copr repo ? dnf copr enable sergiomb/opencv https://copr.fedorainfracloud.org/coprs/sergiomb/opencv/build/3792271/ I added to %build pushd modules/python/package %py3_build popd I added to %install pushd modules/python/package %py3_install popd and in %files -n python3-opencv %{python3_sitelib}/cv2 %{python3_sitelib}/opencv-*.egg-info
$ rpm -qlp python3-opencv-4.5.5-7.fc37.x86_64.rpm /usr/lib/.build-id /usr/lib/.build-id/a4 /usr/lib/.build-id/a4/a5723d4e9b2871e23f0e3213bcd9f1c3ff0fe4 /usr/lib/python3.10/site-packages/cv2 /usr/lib/python3.10/site-packages/cv2/__init__.py /usr/lib/python3.10/site-packages/cv2/__pycache__ /usr/lib/python3.10/site-packages/cv2/__pycache__/__init__.cpython-310.opt-1.pyc /usr/lib/python3.10/site-packages/cv2/__pycache__/__init__.cpython-310.pyc /usr/lib/python3.10/site-packages/cv2/__pycache__/load_config_py2.cpython-310.opt-1.pyc /usr/lib/python3.10/site-packages/cv2/__pycache__/load_config_py2.cpython-310.pyc /usr/lib/python3.10/site-packages/cv2/__pycache__/load_config_py3.cpython-310.opt-1.pyc /usr/lib/python3.10/site-packages/cv2/__pycache__/load_config_py3.cpython-310.pyc /usr/lib/python3.10/site-packages/cv2/load_config_py2.py /usr/lib/python3.10/site-packages/cv2/load_config_py3.py /usr/lib/python3.10/site-packages/opencv-4.5.5-py3.10.egg-info /usr/lib/python3.10/site-packages/opencv-4.5.5-py3.10.egg-info/PKG-INFO /usr/lib/python3.10/site-packages/opencv-4.5.5-py3.10.egg-info/SOURCES.txt /usr/lib/python3.10/site-packages/opencv-4.5.5-py3.10.egg-info/dependency_links.txt /usr/lib/python3.10/site-packages/opencv-4.5.5-py3.10.egg-info/requires.txt /usr/lib/python3.10/site-packages/opencv-4.5.5-py3.10.egg-info/top_level.txt /usr/lib64/python3.10/site-packages/cv2.cpython-310-x86_64-linux-gnu.so $ rpm -qPp python3-opencv-4.5.5-7.fc37.x86_64.rpm python-opencv = 4.5.5-7.fc37 python3-opencv = 4.5.5-7.fc37 python3-opencv(x86-64) = 4.5.5-7.fc37 python3.10-opencv = 4.5.5-7.fc37 python3.10dist(opencv) = 4.5.5 python3dist(opencv) = 4.5.5 Considering https://pypi.org/project/opencv/ does not exist, I don't know if this helps at all. Also, the mixture of /lib/ and /lib64/ modules worries me a little bit. `cv2` seems to be both an extension module and importable package.
Hi, python3-opencv package, as is , only have cv2.cpython-39-x86_64-linux-gnu.so [1] , this cv2.cpython-39-x86_64-linux-gnu.so is "linked" to all opencv [2] reporter need the package provides python3.10dist(opencv-python) for that we need egg-info . 1 - opencv python bindings can't be easily put in pypi.org 2 - seems upstream misses the python wheels , I think we should report it upstream , but I don't know what to write . after review what I wrote, https://pypi.org/project/opencv-python/ exist and https://github.com/opencv/opencv-python is part of opencv organization , what you suggest ? [1] rpm -q python3-opencv -l /usr/lib/.build-id /usr/lib/.build-id/78 /usr/lib/.build-id/78/702acc5d694370cd8bec6bc7e34142cb0325ed /usr/lib64/python3.9/site-packages/cv2.cpython-39-x86_64-linux-gnu.so [2] ldd /usr/lib64/python3.9/site-packages/cv2.cpython-39-x86_64-linux-gnu.so | grep opencv libopencv_hdf.so.4.5 => /lib64/libopencv_hdf.so.4.5 (0x00007f084a329000) libopencv_intensity_transform.so.4.5 => /lib64/libopencv_intensity_transform.so.4.5 (0x00007f084a303000) libopencv_reg.so.4.5 => /lib64/libopencv_reg.so.4.5 (0x00007f084a2e9000) libopencv_surface_matching.so.4.5 => /lib64/libopencv_surface_matching.so.4.5 (0x00007f084a29c000) libopencv_xphoto.so.4.5 => /lib64/libopencv_xphoto.so.4.5 (0x00007f084a258000) libopencv_alphamat.so.4.5 => /lib64/libopencv_alphamat.so.4.5 (0x00007f084a228000) libopencv_dnn_superres.so.4.5 => /lib64/libopencv_dnn_superres.so.4.5 (0x00007f084a216000) libopencv_freetype.so.4.5 => /lib64/libopencv_freetype.so.4.5 (0x00007f084a20b000) libopencv_fuzzy.so.4.5 => /lib64/libopencv_fuzzy.so.4.5 (0x00007f084a1f9000) libopencv_hfs.so.4.5 => /lib64/libopencv_hfs.so.4.5 (0x00007f084a1e5000) libopencv_img_hash.so.4.5 => /lib64/libopencv_img_hash.so.4.5 (0x00007f084a1cf000) libopencv_line_descriptor.so.4.5 => /lib64/libopencv_line_descriptor.so.4.5 (0x00007f084a1a6000) libopencv_saliency.so.4.5 => /lib64/libopencv_saliency.so.4.5 (0x00007f084a17f000) libopencv_wechat_qrcode.so.4.5 => /lib64/libopencv_wechat_qrcode.so.4.5 (0x00007f084a116000) libopencv_mcc.so.4.5 => /lib64/libopencv_mcc.so.4.5 (0x00007f084a0ad000) libopencv_rapid.so.4.5 => /lib64/libopencv_rapid.so.4.5 (0x00007f084a09b000) libopencv_rgbd.so.4.5 => /lib64/libopencv_rgbd.so.4.5 (0x00007f0849fc8000) libopencv_shape.so.4.5 => /lib64/libopencv_shape.so.4.5 (0x00007f0849f9f000) libopencv_stitching.so.4.5 => /lib64/libopencv_stitching.so.4.5 (0x00007f0849ef9000) libopencv_structured_light.so.4.5 => /lib64/libopencv_structured_light.so.4.5 (0x00007f0849ee4000) libopencv_aruco.so.4.5 => /lib64/libopencv_aruco.so.4.5 (0x00007f0849e82000) libopencv_bgsegm.so.4.5 => /lib64/libopencv_bgsegm.so.4.5 (0x00007f0849e65000) libopencv_bioinspired.so.4.5 => /lib64/libopencv_bioinspired.so.4.5 (0x00007f0849e2b000) libopencv_ccalib.so.4.5 => /lib64/libopencv_ccalib.so.4.5 (0x00007f0849dd0000) libopencv_face.so.4.5 => /lib64/libopencv_face.so.4.5 (0x00007f0849d4b000) libopencv_gapi.so.4.5 => /lib64/libopencv_gapi.so.4.5 (0x00007f0849a0b000) libopencv_stereo.so.4.5 => /lib64/libopencv_stereo.so.4.5 (0x00007f08499e7000) libopencv_quality.so.4.5 => /lib64/libopencv_quality.so.4.5 (0x00007f08499d4000) libopencv_phase_unwrapping.so.4.5 => /lib64/libopencv_phase_unwrapping.so.4.5 (0x00007f08499c6000) libopencv_viz.so.4.5 => /lib64/libopencv_viz.so.4.5 (0x00007f0849963000) libopencv_photo.so.4.5 => /lib64/libopencv_photo.so.4.5 (0x00007f08498c3000) libopencv_objdetect.so.4.5 => /lib64/libopencv_objdetect.so.4.5 (0x00007f084983e000) libopencv_optflow.so.4.5 => /lib64/libopencv_optflow.so.4.5 (0x00007f08497da000) libopencv_ximgproc.so.4.5 => /lib64/libopencv_ximgproc.so.4.5 (0x00007f084959e000) libopencv_tracking.so.4.5 => /lib64/libopencv_tracking.so.4.5 (0x00007f08493a5000) libopencv_plot.so.4.5 => /lib64/libopencv_plot.so.4.5 (0x00007f084939b000) libopencv_text.so.4.5 => /lib64/libopencv_text.so.4.5 (0x00007f084932d000) libopencv_ml.so.4.5 => /lib64/libopencv_ml.so.4.5 (0x00007f08492a1000) libopencv_highgui.so.4.5 => /lib64/libopencv_highgui.so.4.5 (0x00007f0849257000) libopencv_videoio.so.4.5 => /lib64/libopencv_videoio.so.4.5 (0x00007f08491cf000) libopencv_imgcodecs.so.4.5 => /lib64/libopencv_imgcodecs.so.4.5 (0x00007f084916d000) libopencv_video.so.4.5 => /lib64/libopencv_video.so.4.5 (0x00007f08490ef000) libopencv_dnn.so.4.5 => /lib64/libopencv_dnn.so.4.5 (0x00007f0848cfe000) libopencv_calib3d.so.4.5 => /lib64/libopencv_calib3d.so.4.5 (0x00007f0848ad9000) libopencv_features2d.so.4.5 => /lib64/libopencv_features2d.so.4.5 (0x00007f0848a0e000) libopencv_flann.so.4.5 => /lib64/libopencv_flann.so.4.5 (0x00007f08489a4000) libopencv_imgproc.so.4.5 => /lib64/libopencv_imgproc.so.4.5 (0x00007f0848455000) libopencv_core.so.4.5 => /lib64/libopencv_core.so.4.5 (0x00007f08480da000)
>> Almost right. You add the requirement manually and you patch/sed out the requirement on opencv-python/opencv-python-headless/opencv-contrib-python/opencv-contrib-python-headless out of upstream metadata (e.g. in %prep). Then you can use %pyproject_buildrequires. >Thank you Miro, It is the answer I was looking for. I couldn't have been the first bumping into this, but I didn't find anything online. >Feel free to close this ticket, as it seems to have a well known solution among experienced packagers. Closing this ticket as it seems there is a mix between python3-opencv from the opencv github tree and opencv-python which is a different code base...
Hi, sorry, I thought that I added a comment to this bug report at around 2022-04-14, where I switch the build of opencv to new Python loader [2] and is now available on F37 ( Beta ) the result is [1] [1] dnf --disablerepo='*' --release=37 --enablerepo=fedora repoquery --available -l python3-opencv /usr/lib/.build-id /usr/lib/.build-id/57 /usr/lib/.build-id/57/afe2df0f9171d3fa9c576c7a116416c82770a6 /usr/lib64/python3.11/site-packages/cv2 /usr/lib64/python3.11/site-packages/cv2/__init__.py /usr/lib64/python3.11/site-packages/cv2/__pycache__ /usr/lib64/python3.11/site-packages/cv2/__pycache__/__init__.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/__init__.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/config-3.11.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/config-3.11.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/config.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/config.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/load_config_py2.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/load_config_py2.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/load_config_py3.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/__pycache__/load_config_py3.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/config-3.11.py /usr/lib64/python3.11/site-packages/cv2/config.py /usr/lib64/python3.11/site-packages/cv2/cv2.cpython-311-x86_64-linux-gnu.so /usr/lib64/python3.11/site-packages/cv2/gapi /usr/lib64/python3.11/site-packages/cv2/gapi/__init__.py /usr/lib64/python3.11/site-packages/cv2/gapi/__pycache__ /usr/lib64/python3.11/site-packages/cv2/gapi/__pycache__/__init__.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/gapi/__pycache__/__init__.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/load_config_py2.py /usr/lib64/python3.11/site-packages/cv2/load_config_py3.py /usr/lib64/python3.11/site-packages/cv2/mat_wrapper /usr/lib64/python3.11/site-packages/cv2/mat_wrapper/__init__.py /usr/lib64/python3.11/site-packages/cv2/mat_wrapper/__pycache__ /usr/lib64/python3.11/site-packages/cv2/mat_wrapper/__pycache__/__init__.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/mat_wrapper/__pycache__/__init__.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/misc /usr/lib64/python3.11/site-packages/cv2/misc/__init__.py /usr/lib64/python3.11/site-packages/cv2/misc/__pycache__ /usr/lib64/python3.11/site-packages/cv2/misc/__pycache__/__init__.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/misc/__pycache__/__init__.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/misc/__pycache__/version.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/misc/__pycache__/version.cpython-311.pyc /usr/lib64/python3.11/site-packages/cv2/misc/version.py /usr/lib64/python3.11/site-packages/cv2/utils /usr/lib64/python3.11/site-packages/cv2/utils/__init__.py /usr/lib64/python3.11/site-packages/cv2/utils/__pycache__ /usr/lib64/python3.11/site-packages/cv2/utils/__pycache__/__init__.cpython-311.opt-1.pyc /usr/lib64/python3.11/site-packages/cv2/utils/__pycache__/__init__.cpython-311.pyc [2] https://src.fedoraproject.org/rpms/opencv/c/feee2b1ca875ab0d3c8fb8a83e8f5f310c28d809