Fedora 25 composes are currently failing. I believe this is because pungi-fedora has been updated for the backwards-incompatible change to how 'failable' deliverables are defined: https://pagure.io/pungi-fedora/c/2ac4a62920900ae8198e2f0fadd2196ed0c57ce3?branch=master but pungi 4.1.8 - which introduces the change - is not 'stable' for Fedora 25 yet. It has been built: http://koji.fedoraproject.org/koji/buildinfo?buildID=790649 but it was after the Bodhi activation point, and there is no update submitted. So we need, I believe, to have an update for pungi 4.1.8 submitted (and marked as fixing this bug), and then we need to push it stable so the compose process uses it. Either that, or we need to revert the pungi-fedora change (which would in turn break Rawhide, since 4.1.8 has gone stable there...so in fact we'd need an f25 branch for pungi-fedora so we could diverge configuration between f25 and Rawhide). The symptom is that the compose process fails as soon as any image fails to compose, as no images are considered 'failable' by the old Pungi, after the configuration changes. 20160815.n.1 compose failed due to the KDE live image compose failing. All the other composes between 20160810.n.0 and 20160815.n.0 failed when an OStree compose attempt failed. If you look here: https://kojipkgs.fedoraproject.org/compose//branched/Fedora-25-20160810.n.0/logs/x86_64/ostree/ostree-2/runroot.log you can see the OStree failure actually occurred in 20160810.n.0 too, but did not fail the whole compose. And we know of course that live images often fail to compose due to https://bugzilla.redhat.com/show_bug.cgi?id=1315541 . This is an automatic Alpha blocker: "Bugs which entirely prevent the composition of one or more of the release-blocking images required to be built for a currently-pending (pre-)release" - this bug prevents the compose of *all* images.
Ah, so in fact I think the issue was the opposite: the new pungi is being used (apparently it doesn't matter if it was stable), but the f25 branch of pungi-fedora was not updated with the new 'failable' configuration stuff until 7 hours ago: https://pagure.io/pungi-fedora/c/716f612efd6a8e5e3a6d980f382343169bd8047e?branch=f25 the currently running compose - https://kojipkgs.fedoraproject.org/compose//branched/Fedora-25-20160815.n.2/ - should I *think* be the first run with the updated pungi-fedora config, so we'll see how that goes.
You can see what configuration was used for a compose in logs/global/config-dump-$date.global.log.
yes, but we can't (at least, nowhere I could spot) see what version of pungi is used. I think the situation is as described in #c1, though. 20160815.n.2 completed, so I think this is fixed, but I'll leave it open in case dgilmore wants to chime in.
You're right, which is exactly why I opened https://pagure.io/pungi/pull-request/364 to add pungi version information to the log.
bugzilla is not an appropriate place for filing issues like this. the composes run on f24 using the f24 build. we have put pungi in the infra builder repo to ensure we use the version we want.
We need to file release blocking issues on Bugzilla in order to track them. Complete failure to compose for multiple days in a row is clearly a release blocking issue.