Bug 2262430

Summary: rpm packaging, with multiple forge sources, fails to unpack correct source/target dirs
Product: [Fedora] Fedora Reporter: pgnd <pgnd>
Component: forge-srpm-macrosAssignee: Maxwell G <maxwell>
Status: CLOSED NOTABUG QA Contact:
Severity: medium Docs Contact:
Priority: unspecified    
Version: rawhideCC: ajax, carl, ffesti, fweimer, igor.raits, j, maxwell, mhroncok, ngompa13, nickc, pmatilai, sipoyare, torsava
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
URL: https://copr.fedorainfracloud.org/coprs/pgfed/test/builds/
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2024-02-02 20:06:56 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description pgnd 2024-02-02 18:28:48 UTC
given 2 rpm specs,

	https://pagure.io/pgnd/test/blob/main/f/test1.spec
	https://pagure.io/pgnd/test/blob/main/f/test2.spec

where the diff is only,

	diff -ur test1.spec test2.spec
		--- test1.spec  2024-02-02 12:52:00.771258038 -0500
		+++ test2.spec  2024-02-02 12:52:55.415194669 -0500
		@@ -63,8 +63,14 @@
		 %description                   %{_test0_descrip}

		 %prep
		-%forgesetup -z 0
		-%forgesetup -z 1
		+cd /builddir/build/BUILD
		+rm -rf %{extractdir0}
		+rm -rf %{extractdir1}
		+/usr/lib/rpm/rpmuncompress -x -v %{SOURCE0}
		+pigz -dcq "%{SOURCE0}" | tar -xvvof -
		+/usr/lib/rpm/rpmuncompress -x -v %{SOURCE1}
		+pigz -dcq "%{SOURCE1}" | tar -xvvof -
		+

		 %build

building

	rm -rf /home/mock/resultdir/*/*
	mock -r fedora-39-x86_64 --no-bootstrap-chroot --buildsrpm --resultdir /home/mock/resultdir/1 --spec ./test1.spec --sources ./ 
	mock -r fedora-39-x86_64 --no-bootstrap-chroot --resultdir /home/mock/resultdir/2 --no-cleanup-after rebuild /home/mock/resultdir/1/*src.rpm

fails in %prep

	...
	Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.3W1j0F
	+ umask 022
	+ cd /builddir/build/BUILD
	+ cd /builddir/build/BUILD
	+ rm -rf pgnd-testrepo0-e93e20d
(1)	+ /usr/lib/rpm/rpmuncompress -x /builddir/build/SOURCES/e93e20d
	+ STATUS=0
	+ '[' 0 -ne 0 ']'
(2)	+ cd pgnd-testrepo0-e93e20d
	+ rm -rf /builddir/build/BUILD/pgnd-testrepo0-e93e20d-SPECPARTS
	+ /usr/bin/mkdir -p /builddir/build/BUILD/pgnd-testrepo0-e93e20d-SPECPARTS
	+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
	+ cd /builddir/build/BUILD
	+ rm -rf pgnd-testrepo1-51d7e48
(3)	+ /usr/lib/rpm/rpmuncompress -x /builddir/build/SOURCES/e93e20d
	+ STATUS=0
	+ '[' 0 -ne 0 ']'
(4)	+ cd pgnd-testrepo1-51d7e48
	/var/tmp/rpm-tmp.3W1j0F: line 53: cd: pgnd-testrepo1-51d7e48: No such file or directory
	error: Bad exit status from /var/tmp/rpm-tmp.3W1j0F (%prep)
	...

NOTE!! 

whereas at 1st 'forgesetup',

	-%forgesetup -z 0

dir IS found and unpacked, @ (4), dir is NOT FOUND, because in (3), the 2nd 'forgesetup'

	-%forgesetup -z 1

incorrectly unpacks the 1st forgesource dir a 2nd time,

	/usr/lib/rpm/rpmuncompress -x /builddir/build/SOURCES/e93e20d

same as (1), rather than what'd be expected/needed,

	/usr/lib/rpm/rpmuncompress -x /builddir/build/SOURCES/51d7e48


OTOH, replacing the forgesetup stanzas with MANUAL prep, as in test2.spec (diff above), then building,

	rm -rf /home/mock/resultdir/*/*
	mock -r fedora-39-x86_64 --no-bootstrap-chroot --buildsrpm --resultdir /home/mock/resultdir/1 --spec ./test2.spec --sources ./ 
	mock -r fedora-39-x86_64 --no-bootstrap-chroot --resultdir /home/mock/resultdir/2 --no-cleanup-after rebuild /home/mock/resultdir/1/*src.rpm

succeeds

	ls -al /home/mock/resultdir/2/*rpm

		-rw-r--r--+ 1 pgnd mock  11K Feb  2 13:15 /home/mock/resultdir/2/test-git_main-0.20240202_181552.pgnd.fc39.src.rpm
		-rw-r--r--+ 1 pgnd mock 6.3K Feb  2 13:15 /home/mock/resultdir/2/test-git_main-0.20240202_181554.pgnd.fc39.x86_64.rpm

without error


both forgesetup cmds should work correctly on specified targets

Reproducible: Always

Comment 1 Miro HronĨok 2024-02-02 19:21:27 UTC
I was going to assign this to forge-srpm-macros, but apparently in Fedora 39, this still lives in redhat-rpm-config :(

Anyway, assigning Maxwell.

Comment 2 pgnd 2024-02-02 19:24:46 UTC
hopefully a bit clearer, e.g., with .spec

	...
	%prep
	%forgesetup -z 0
	%forgesetup -z 1
	...

where

	Packaging variables read or set by %forgemeta
	  ...
	  forgesource0:      https://api.github.com/repos/pgnd/testrepo0/tarball/e93e20d
	  forgesetupargs0:   -n pgnd-testrepo0-e93e20d
	  ...
	  extractdir0:       pgnd-testrepo0-e93e20d
	...
	Packaging variables read or set by %forgemeta
	  ...
	  forgesource1:      https://api.github.com/repos/pgnd/testrepo1/tarball/51d7e48
	  forgesetupargs1:   -n pgnd-testrepo1-51d7e48
	  extractdir1:       pgnd-testrepo1-51d7e48
	  ...

then, enterint %prep,

	Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.TmIXbW

@ `%forgesetup -z 0`

	...
	+ cd /builddir/build/BUILD
	+ rm -rf pgnd-testrepo0-e93e20d
	+ /usr/lib/rpm/rpmuncompress -x /builddir/build/SOURCES/e93e20d
	...
	+ cd pgnd-testrepo0-e93e20d

& @ `%forgesetup -z 1`

	...
	+ cd /builddir/build/BUILD
	+ rm -rf pgnd-testrepo1-51d7e48                                   <---- *correct*   target
	+ /usr/lib/rpm/rpmuncompress -x /builddir/build/SOURCES/e93e20d   <---- *incorrect* target
	...
	+ cd pgnd-testrepo1-51d7e48

resulting in

	RPM build errors:
	/var/tmp/rpm-tmp.TmIXbW: line 53: cd: pgnd-testrepo1-51d7e48: No such file or directory

Comment 3 Maxwell G 2024-02-02 19:53:57 UTC
> I was going to assign this to forge-srpm-macros, but apparently in Fedora 39, this still lives in redhat-rpm-config :(

I wanted to apply the same change to Fedora 39, but nobody responded to my PR. I do not plan to make any further changes to the forge macros contained within redhat-rpm-config. I just rebased https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/264 if someone wants to take a look at it. In the meantime, I will retarget this against forge-srpm-macros, as I could reproduce this with the current version of that.

Comment 4 Maxwell G 2024-02-02 20:06:56 UTC
Thanks for the report! It seems the issue here is that you're overriding %forgesetupargs (and many of the other macros that %forgemeta set). I'm not sure why you're doing that. The whole point of the forge macros is that they determine those values for you. For the %forgesetupargs case, %forgemeta sets that macro so that the second %forgesetup invocation unpacks the second source archive on top of the first one. You're overriding %forgesetupargs1 to `-n %{archivename1}` which incorrectly tries to unpack the first archive using the top directory name of the second. Usually, the forge macros would set "%forgesetupargs1" to "-T -D -b 1 -n %{extractdir1}" which works as expected.

If you have any further questions, feel free to stop by the forge-srpm-macros mailing list: https://lists.sr.ht/~gotmax23/forge-srpm-macros/ Thanks again!

Comment 5 pgnd 2024-02-02 20:16:57 UTC
Can you clarify what's occurring @

	+ rm -rf pgnd-testrepo1-51d7e48                                   <---- *correct*   target
	+ /usr/lib/rpm/rpmuncompress -x /builddir/build/SOURCES/e93e20d   <---- *incorrect* target

the `rm` clearly has the correct target
the immediately subsequent `rpmuncompress` has the incorrect target

since both steps are (?) expanded from the same `-%forgesetup -z 1`, shouldn't both be targeting the same target?

Comment 6 Maxwell G 2024-02-02 21:42:32 UTC
I don't fully understand your question. It's unpacking Source0 because that's what you overrode %forgesetupargs1 to do. `%forgesetup -z1` just calls `%setup %{forgesetupargs1}`. You set `%forgesetupargs1` to ` -n %{archivename1}`. Passing `-n` tells `%setup` what the name of the top-level directory in the source archive is, but it still unpacks the first archive (i.e., Source0) unless you pass the appropriate -a / -b flags to change that.

Comment 7 pgnd 2024-02-02 21:50:10 UTC
`%forgesetup -z 1`,

this line

	+ rm -rf pgnd-testrepo1-51d7e48

'uses' the 2nd target.

what 'tells' the `rm` to use the 2nd target, if not `forgesetup` ?

the very next line,

	+ /usr/lib/rpm/rpmuncompress -x /builddir/build/SOURCES/e93e20d

"switches back to", and uses the 1st target.

Comment 8 Maxwell G 2024-02-02 22:00:59 UTC
http://ftp.rpm.org/max-rpm/s1-rpm-inside-macros.html#S2-RPM-INSIDE-SETUP-MACRO provides more information about how the %setup macro works and can explain it better than I can.

> 	+ rm -rf pgnd-testrepo1-51d7e48

This has nothing to do with "targets." You explicitly "told" the %setup macro (by passing `-n pgnd-testrepo1-51d7e48`) that `pgnd-testrepo1-51d7e48` is the top-level directory of the archive. %setup tries to clean up that top-level directory name before decompressing the archive. That is separate from the decompression of the new archive which defaults to Source0.