Bug 1693759 - mock: Switch to a string for RPM calls (oncoming rpm API fix change)
Summary: mock: Switch to a string for RPM calls (oncoming rpm API fix change)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: mock
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Miroslav Suchý
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1693751 1779194
TreeView+ depends on / blocked
 
Reported: 2019-03-28 15:06 UTC by Panu Matilainen
Modified: 2019-12-03 13:47 UTC (History)
6 users (show)

Fixed In Version: mock-1.4.15-1.fc30 mock-1.4.15-1.fc28 mock-1.4.15-1.fc29 mock-1.4.15-1.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-04-27 21:27:41 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Panu Matilainen 2019-03-28 15:06:10 UTC
Description of problem:

Rpm's python3 API has been totally braindamaged all this time but people are only noticing now that it's starting to get used. 

We're changing rpm to return all string data as surrogate-escaped utf-8 python strings everywhere (instead of bytes with unknown encoding that the API doesn't otherwise even accept, see bug 1631292). This makes most rpm-scripts written for python2 just work with python3 too (from the rpm pov).

Most software that has kept python2 compatibility are automatically compatible with the fixed API, but unfortunately python3-only users like mock need fixing for the new behavior.

There's at least one affected place in mock, resulting in a traceback similar to this with the new behavior, unless fixed:

Traceback (most recent call last):
  File "/usr/libexec/mock/mock", line 977, in <module>
    main()
  File "/usr/lib/python3.7/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 766, in main
    run_command(options, args, config_opts, commands, buildroot, state)
  File "/usr/lib/python3.7/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 861, in run_command
    do_rebuild(config_opts, commands, buildroot, options, args)
  File "/usr/lib/python3.7/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 541, in do_rebuild
    post=post_build, clean=clean)
  File "/usr/lib/python3.7/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/libexec/mock/mock", line 482, in rebuild_generic
    ret = cmd(item)
  File "/usr/libexec/mock/mock", line 522, in build
    check=config_opts['check'], spec=options.spec)
  File "/usr/lib/python3.7/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.7/site-packages/mockbuild/backend.py", line 275, in build
    self.installSrpmDeps(rebuilt_srpm)
  File "/usr/lib/python3.7/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.7/site-packages/mockbuild/backend.py", line 198, in installSrpmDeps
    deps.extend(util.getAddtlReqs(hdr, self.more_buildreqs))
  File "/usr/lib/python3.7/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.7/site-packages/mockbuild/util.py", line 400, in getAddtlReqs
    (name, epoch, ver, rel, arch) = getNEVRA(hdr)
  File "/usr/lib/python3.7/site-packages/mockbuild/trace_decorator.py", line 96, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.7/site-packages/mockbuild/util.py", line 386, in getNEVRA
    return tuple(x.decode() if i != 1 else x for i, x in enumerate(ret))
  File "/usr/lib/python3.7/site-packages/mockbuild/util.py", line 386, in <genexpr>
    return tuple(x.decode() if i != 1 else x for i, x in enumerate(ret))
AttributeError: 'str' object has no attribute 'decode'

As broken versions of the rpm bindings are widely in use, it's recommended that compatibility with both versions is kept for the time being. A possible simple fix is just:
--- util.py.orig	2019-03-28 16:32:22.378591934 +0200
+++ util.py	2019-03-28 16:32:58.009530429 +0200
@@ -342,6 +342,10 @@
     for dummy in yieldSrpmHeaders(srpms, plainRpmOk):
         pass
 
+def b2s(data):
+    if isinstance(data, bytes):
+        return data.decode('utf-8')
+    return data
 
 @traceLog()
 def getNEVRA(hdr):
@@ -354,7 +358,7 @@
     if epoch is None:
         epoch = 0
     ret = (name, epoch, ver, rel, arch)
-    return tuple(x.decode() if i != 1 else x for i, x in enumerate(ret))
+    return tuple(b2s(x) if i != 1 else x for i, x in enumerate(ret))
 

Alternatively, upstream mock version already has a suitable helper added in commit 08b6e4a0eaf7e8317bbef4877302528cb60e6ea5, so the upstream fix would be simply:

--- a/mock/py/mockbuild/util.py
+++ b/mock/py/mockbuild/util.py
@@ -383,7 +383,7 @@ def getNEVRA(hdr):
     if epoch is None:
         epoch = 0
     ret = (name, epoch, ver, rel, arch)
-    return tuple(x.decode() if i != 1 else x for i, x in enumerate(ret))
+    return tuple(_to_text(x) if i != 1 else x for i, x in enumerate(ret))

Comment 1 Panu Matilainen 2019-03-29 07:10:04 UTC
I was pointed out that the bug referred to in the message is a private RHEL bug, sorry about that. 
The public, Fedora side counterpart with the background story is the one blocked by this, ie 
https://bugzilla.redhat.com/show_bug.cgi?id=1693751

Comment 2 Miroslav Suchý 2019-04-19 18:12:17 UTC
Committed as:
  * c7ce1fc (HEAD -> devel) switch to string rpm's API [RHBZ#1693759]

Comment 3 Fedora Update System 2019-04-22 19:13:28 UTC
mock-1.4.15-1.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-7a7a12aff5

Comment 4 Fedora Update System 2019-04-22 19:13:34 UTC
mock-1.4.15-1.fc30 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-98a73907d3

Comment 5 Fedora Update System 2019-04-22 19:13:40 UTC
mock-1.4.15-1.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-9f8e99528d

Comment 6 Fedora Update System 2019-04-22 19:13:46 UTC
mock-1.4.15-1.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2019-0649a04cab

Comment 7 Fedora Update System 2019-04-23 14:55:50 UTC
mock-1.4.15-1.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-98a73907d3

Comment 8 Fedora Update System 2019-04-23 19:33:43 UTC
mock-1.4.15-1.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-0649a04cab

Comment 9 Fedora Update System 2019-04-23 21:15:57 UTC
mock-1.4.15-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-7a7a12aff5

Comment 10 Fedora Update System 2019-04-23 23:40:47 UTC
mock-1.4.15-1.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2019-9f8e99528d

Comment 11 Fedora Update System 2019-04-27 21:27:41 UTC
mock-1.4.15-1.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.

Comment 12 Fedora Update System 2019-04-30 01:40:32 UTC
mock-1.4.15-1.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.

Comment 13 Fedora Update System 2019-04-30 02:27:40 UTC
mock-1.4.15-1.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.

Comment 14 Fedora Update System 2019-05-03 03:02:48 UTC
mock-1.4.15-1.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.