Bug 1551653

Summary:	Composes taking nearly a day and a half to complete
Product:	[Fedora] Fedora	Reporter:	Adam Williamson <awilliam>
Component:	pungi	Assignee:	Lubomír Sedlář <lsedlar>
Status:	CLOSED ERRATA	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	28	CC:	admiller, dennis, dmach, dustymabe, fzatlouk, herrold, jkeating, lsedlar, robatino, satellitgo, sergio, tdawson, wwoods
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:	AcceptedBlocker
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-03-12 23:16:28 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1469204

Description Adam Williamson 2018-03-05 16:02:39 UTC

Recent Rawhide and Fedora 28 composes seem to be taking an unreasonably long amount of time to run. Fedora-Rawhide-20180303.n.0 took just over 32 hours (from 2018-03-03 15:55:19 to 2018-03-05 00:05:46: https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-Rawhide-20180303.n.0/logs/global/pungi.global.log ).

'Normal' compose time is closer to 8-12 hours, at present, I believe.

Proposing as a Beta blocker per this wording in the criteria:

"A bug is considered a Beta blocker bug if any of the following criteria are met: 
...
Bug hinders execution of required Beta test plans or dramatically reduces test coverage"

Composes taking 3x as long as they should will *clearly* do that; it severely reduces the amount of times we can feasibly respin, and the amount of time QA will have to test any given candidate compose.

Comment 1 František Zatloukal 2018-03-05 17:38:14 UTC

Discussed at blocker bug meeting [1]:
    
AcceptedBlocker (Beta) - this is considered to block the release on the basis that it "hinders execution of required Beta test plans or dramatically reduces test coverage", as per https://fedoraproject.org/wiki/Fedora_28_Beta_Release_Criteria#Beta_Blocker_Bugs

[1] https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2018-03-05/

Comment 2 Sergio Basto 2018-03-06 01:13:15 UTC

I found one notification [1] in Fedora-28-20180303.n.1 that may explain
why [2]. Is waiting for a commit sign for 22 hours. 

Cheers,

[1]
https://kojipkgs.fedoraproject.org/compose/branched/Fedora-28-20180303.n.1/logs/global/notifications/notification-2018-03-05_02-36-29.log

[2]
2018-03-05 02:36:30.394652: Commit not signed yet, waiting
(...)
2018-03-05 23:11:20.314618: Commit not signed yet, waiting

Comment 3 Lubomír Sedlář 2018-03-09 18:01:05 UTC

Part of the slowdown was caused by a bug in Pungi that only appeared when traditional and modular content was included in the compose. That should be fixed in the latest build. Unfortunately that build also introduced another bug that caused a lot of packages to be missing from the compose. Hopefully that would be fixed in pungi-4.1.22-8.fc27 that I'm building now.

Comment 4 Adam Williamson 2018-03-12 23:16:28 UTC

per nirik, this is considered sufficiently fixed now. closing.