Bug 2078744

Summary: %generate_buildrequires fail mockbuilds
Product: [Fedora] Fedora Reporter: Miro Hrončok <mhroncok>
Component: rpmAssignee: Packaging Maintenance Team <packaging-team-maint>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: rawhideCC: carl, decathorpe, igor.raits, mhayden, mjw, msuchy, packaging-team-maint, pmatilai, pmoravco, praiskup, vmukhame
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: rpm-4.18.0-0.alpha1.4 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-04-26 16:49:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2078414    

Description Miro Hrončok 2022-04-26 06:50:09 UTC
Since RPM rpm-4.18.0-0.alpha1.1.fc37 it seem that many (if not all) packages with Python %generate_buildrequires fail mockbuilds.

To reproduce, submit a Koji scratchbuild or do the following:

$ mock -r fedora-rawhide-x86_64 clean 
$ mock -r fedora-rawhide-x86_64 --scrub=all
$ fedpkg --release rawhide mockbuild -N --enablerepo=local


For example with python-parso, python-setuptools or the example from [1] (use mock -r fedora-rawhide-x86_64 --enablerepo=local python-pello-1.0.3-1.fc35.src.rpm).


[1] https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#_example_spec_file 




We get, more or less:

Executing(%generate_buildrequires): 
Dependencies resolved... dnf installs.

Executing(%generate_buildrequires): 
Dependencies resolved... dnf installs.

Executing(%generate_buildrequires): 
Dependencies resolved... Nothing to do.

Executing(%generate_buildrequires): 
Wrote: /builddir/build/SRPMS/python-pello-1.0.3-1.fc37.buildreqs.nosrc.rpm

Finish: rpmbuild python-pello-1.0.3-1.fc35.src.rpm
Finish: build phase for python-pello-1.0.3-1.fc35.src.rpm
ERROR: Exception(/home/churchyard/rpmbuild/SRPMS/python-pello-1.0.3-1.fc35.src.rpm) Config(fedora-rawhide-x86_64) 0 minutes 53 seconds


Previously, we got:


Executing(%generate_buildrequires): 
Dependencies resolved... dnf installs.

Executing(%generate_buildrequires): 
Dependencies resolved... dnf installs.

Executing(%generate_buildrequires): 
Dependencies resolved... Nothing to do.

Executing(%build): ...

Comment 1 Miro Hrončok 2022-04-26 06:51:32 UTC
This happens in local mock, in Koji, and in Copr.

Comment 2 Miro Hrončok 2022-04-26 06:55:16 UTC
This also happens with Rust packages, not just Python.

Comment 3 Panu Matilainen 2022-04-26 07:30:16 UTC
Ack, I'll look into it. Thanks for the report.

Comment 4 Panu Matilainen 2022-04-26 08:09:14 UTC
Should be fixed in rpm-4.18.0-0.alpha1.2.fc37, building now.

Comment 5 Fabio Valentini 2022-04-26 09:45:24 UTC
It's not fixed. All packages using %generate_buildrequires still fail to build in rawhide:

(from the build.log of rust-anyhow from koschei):

Executing(%generate_buildrequires): /bin/sh -e /var/tmp/rpm-tmp.KfRR7b
+ umask 022
+ cd /builddir/build/BUILD
+ cd anyhow-1.0.56
+ /usr/bin/cargo-inspector -BR Cargo.toml
+ /usr/bin/cargo-inspector -TR Cargo.toml
+ RPM_EC=0
++ jobs -p
+ exit 0
Wrote: /builddir/build/SRPMS/rust-anyhow-1.0.56-1.fc37.buildreqs.nosrc.rpm
Child return code was: 11
EXCEPTION: [Error()]
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/mockbuild/trace_decorator.py", line 93, in trace
    result = func(*args, **kw)
  File "/usr/lib/python3.10/site-packages/mockbuild/util.py", line 599, in do_with_status
    raise exception.Error("Command failed: \n # %s\n%s" % (command, output), child.returncode)
mockbuild.exception.Error: Command failed: 
 # bash --login -c /usr/bin/rpmbuild -ba --noprep --target x86_64 --nodeps /builddir/build/SPECS/rust-anyhow.spec

Comment 6 Miro Hrončok 2022-04-26 10:41:13 UTC
Indeed rpm-4.18.0-0.alpha1.2.fc37 does not fix this.

Comment 7 Panu Matilainen 2022-04-26 11:56:27 UTC
Ok, there's more than one bug then... back to investigating then.

Comment 8 Pavel Raiskup 2022-04-26 12:15:08 UTC
I'm curious if we could coordinate on some /bin/mock integration test
for %generate_buildrequires for the future?

Comment 9 Panu Matilainen 2022-04-26 12:18:29 UTC
I'm adding tests for anything found here to rpm's test-suite. It covers a bunch of %generate_depends cases already but apparently some important ones are missing... so yes.

Comment 10 Pavel Raiskup 2022-04-26 12:50:19 UTC
I meant something like "Fedora CI" check, something which would trigger
a mock build for the PR before it gets merged (with fixed RPM on host/in
chroot).  Then you would have early feedback about a potential breakage
like this.  Example: https://src.fedoraproject.org/rpms/tar/pull-request/7

Comment 11 Panu Matilainen 2022-04-26 13:39:17 UTC
And yeah there was another regression, occuring specifically with -ba --nodeps when dynamic buildrequires are present. Building now, should be fixed in rpm-4.18.0-0.alpha1.3. Knock wood.

Covering all the myriad of switches in all possible scenarios is simply impossible but since this is something mock relies on for normal operation we'll better add a test. I wont stop anybody from writing a Fedora CI level check, but I want my behind covered at rpm upstream level.

Comment 12 Panu Matilainen 2022-04-26 14:05:31 UTC
Meh, nope, it gets further but runs into some yet another untested problem :(

Comment 13 Panu Matilainen 2022-04-26 15:48:51 UTC
Okay, seems third time's the charm, at least python-parso and python-setuptools are in mock again with rpm-4.18.0-0.alpha1.4. Pooh.

Apologies for the inconvenience.

Comment 14 Miro Hrončok 2022-04-26 16:49:10 UTC
> I wont stop anybody from writing a Fedora CI level check...

Feel free to reopen this BZ for that purpose, but the problem itself is fixed, so closing. Thanks for the fix.


For inspiration, see https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/rawhide/f/tests/mocktest.sh

Comment 15 Miro Hrončok 2022-04-26 17:08:02 UTC
And I guess the easiest way is to just run our own tests: https://src.fedoraproject.org/rpms/rpm/pull-request/24

Note that this runs when you open a PR and it seems that is sadly not your workflow in the rpm package.