Bug 1026834

Created attachment 819775 [details]
anaconda failure screenshot

Created attachment 819776 [details]
ks.cfg

Created attachment 819777 [details]
anaconda.log (with method=)

Created attachment 819778 [details]
anaconda-yum.conf (with method=)

Created attachment 819779 [details]
packaging.log (with method=)

Created attachment 819780 [details]
program.log (with method=)

Created attachment 819781 [details]
storage.log (with method=)

Created attachment 819782 [details]
syslog (with method=)

Full anaconda logs available here:
http://kparal.fedorapeople.org/bugs/rhbz_1026834.7z

Created attachment 830241 [details]
screenshot with error in TC3

The reason why I've found this is because of bug 1026841. virt-install uses method= and therefore fails to install Fedora.

I have tried again with F20 TC3, the same problem occurs. This time it fails because of sqlite-3.8.1-2.fc20.x86_64 (which is in updates-testing, but not in stable repos). I used netinst and a kickstart. However, it's very interesting:

* BROKEN: method=http://dl.fedoraproject.org/pub/alt/stage/20-TC3/Fedora/x86_64/os/ inst.ks=http://192.168.11.1:8000/ks.cfg

* WORKS: repo=http://dl.fedoraproject.org/pub/alt/stage/20-TC3/Fedora/x86_64/os/ inst.ks=http://192.168.11.1:8000/ks.cfg

* WORKS: inst.repo=http://dl.fedoraproject.org/pub/alt/stage/20-TC3/Fedora/x86_64/os/ inst.ks=http://192.168.11.1:8000/ks.cfg

* WORKS: method=http://dl.fedoraproject.org/pub/fedora/linux/development/20/x86_64/os/ inst.ks=http://192.168.11.1:8000/ks.cfg

* WORKS: repo=http://dl.fedoraproject.org/pub/fedora/linux/development/20/x86_64/os/ inst.ks=http://192.168.11.1:8000/ks.cfg

* WORKS: inst.repo=http://dl.fedoraproject.org/pub/fedora/linux/development/20/x86_64/os/ inst.ks=http://192.168.11.1:8000/ks.cfg

The only broken combination is method= _and_ a TC repository (i.e. not full development repository). The TC repository is just a small selection of packages (equivalent to loop-mounted DVD.iso), so I tried to replace it with a mounted DVD.iso image and it behaves the same - also spits out an error with method= (and not with repo= or inst.repo=).

I can't reproduce this with interactive installation, just with a kickstart.

The full TC3 logs are available at:
http://kparal.fedorapeople.org/bugs/rhbz_1026834_TC3.7z

This could be a violation of
"The installer must be able to complete a scripted installation which duplicates the default interactive installation as closely as possible. " or
"Any installation method or process designed to run unattended must do so. There should be no prompts requiring user intervention. "
https://fedoraproject.org/wiki/Fedora_20_Beta_Release_Criteria

Only method= + ks= installations seem to be affected, and only if you use DVD.iso-like repository. But it might be a common use case, if people are used to mount DVD.iso and then run "virt-install --location" (and a kickstart by any approach possible). That's exactly a combination of DVD repository, method= and ks=.

I reproduced failure from comment 10 with the same steps (kickstart and parameters).

In case I can't attend today's blocker bug meeting, I vote +1 blocker. We should either
a) make virt-install to issue repo= instead of method= (that's bug 1026841). If people use method= manually, it's their fault, it has been obsolete for ages.
or
b) make anaconda behave the same for method= and repo= (that could be a fairly simple fix, the code probably contains some switch that activates different code path for method= than for repo=). This would fix virt-install as well and no changes to it would be necessary.

I withdraw my previous blocker vote. I was convinced I can reproduce it any time, just by using an old DVD.iso or TC directory. But no, today neither me nor Martin Kolman can reproduce it, even when using a over old TC repository. It is clearly very much affected by current yum repos contents. So, probably -1 blocker right now until we can better demonstrate the affected use cases.

My latest suspicion is that this happens when the TC folder/DVD compose contains _newer_ packages than current stable repos. It was the case for ​sqlite-3.8.1-2.fc20 [1]. We will see when we have TC4 and it again includes some packages that are available in updates-testing and not in stable updates.

[1] https://fedorahosted.org/rel-eng/ticket/5808#comment:8

Discussed at 2013-12-02 blocker review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2013-12-02/f20-blocker-review-%234.2013-12-02-17.02.log.txt . Agreed to delay the decision until after this can be tested with TC4, as outlined in c#14.

Created attachment 832201 [details]
screenshot with error in TC4

Here we go with TC4 :-) 

error populating transaction: failure: Packages/f/fedora-release-20-1.noarch.rpm from anaconda: [Errno 256] No more mirrors to try.

These boot options were used:
method=http://dl.fedoraproject.org/pub/alt/stage/20-TC4/Fedora/x86_64/os/ inst.ks=http://192.168.11.1:8000/ks.cfg

fedora-release-20-1.noarch.rpm is available at:
http://dl.fedoraproject.org/pub/alt/stage/20-TC4/Fedora/x86_64/os/Packages/f/
and also in updates-testing. But it's not available in stable updates (only older version).

If you perform a GNOME installation instead of a minimal one, it crashes on webkitgtk3-2.2.2-2.x86_64.fc20, available in TC repo, but unavailable in neither updates-testing nor stable updates.

We debugged this with mkolman and we found out that 'url' kickstart command clashes with 'method=' boot argument in these circumstances. If we remove 'url' from kickstart, everything works as expected (or if we use repo= instead of method=).

So, to recapitulate current findings:
1. This affects virt-install users that use --location with a DVD-sized repo (either mounted DVD.iso or a TC/RC compose repo) and a kickstart containing 'url' command. It affects them only in certain conditions, probably when their --location contains newer packages than what 'url' and default Fedora repos point at.
2. It can be worked around by
a) starting installation by hand with repo= (i.e. avoiding virt-install)
b) removing 'url' from kickstart

This bug lowers the reliability of our installation process, but no longer seems serious enough to warrant a release blocker, in my opinion.

IIUC, it's going to be just about impossible to hit post-release, right? As the remote repos will be at least as new as the installation media at release time, and only get 'newer' as time goes on.

(In reply to Adam Williamson from comment #17)
> IIUC, it's going to be just about impossible to hit post-release, right? As
> the remote repos will be at least as new as the installation media at
> release time, and only get 'newer' as time goes on.

Yes, currently it seems that way. I seem to hit it only when the provided repository in method= is newer than the default Fedora repositories, which should never happen after Fedora 20 is released. But I can't really vouch for that. It's a black box for me.

Hmm, let's complicate the things even more, shall we?

Today I tried to boot Final TC4 netinst with a very old TC repo:

method=http://dl.fedoraproject.org/pub/alt/stage/20-Alpha-TC3/Fedora/x86_64/os/ inst.ks=http://192.168.11.1:8000/ks.cfg

The kickstart contained:
> url --mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-$releasever&arch=$basearch
> repo --name=fedora
> repo --name=updates
> repo --name=updates-testing
> 
> %packages --default
> %end

It failed to find liberation-fonts-common-1.07.3-2.fc20.noarch.rpm, which is available in both Alpha-TC3 repo:
http://dl.fedoraproject.org/pub/alt/stage/20-Alpha-TC3/Fedora/x86_64/os/Packages/l/
and current main repo:
http://dl.fedoraproject.org/pub/fedora/linux/development/20/x86_64/os/Packages/l/
(it's not available in updates-testing repo)

Full logs available at: http://kparal.fedorapeople.org/bugs/rhbz_1026834_liberation.7z


When I do the same with 20-Alpha-TC1 repo, it crashes because of ql2200-firmware-2.02.08-8.fc20.noarch.rpm. But when I repeat that with 20-Beta-TC3 repo, everything works OK.

Can somebody explain me _that_? This indicates that this might affect even a post-release installations, in certain (unknown conditions).

Additional information: If I replace 'url --mirrorlist=' in the kickstart with 'url --url=' command, it no longer crashes.

James, can you please give us some yum-related feedback? Thanks.

Discussed in 2013-12-04 Blocker Review Meeting [1]. It was voted as a RejectedBlocker but an AcceptedFreezeException. Non-functional method affects only virt users and post-release users won't be probably affected. Rejected as blocker but it's consider as FE. 

[1] http://meetbot.fedoraproject.org/fedora-blocker-review/2013-12-04/

Fixing tags.

I'm experiencing the same issue when trying to install F20 TC4 in virt-manager.

In the GUI I select "Create a new VM" -> Network Install (HTTP, FTP, or NFS)...

Note that:
- I don't use any kickstart, but manually doing all of the installation
steps.
- I'm installing the minimal installation.

Fix has been posted for review.

anaconda-20.25.15-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/anaconda-20.25.15-1.fc20

Package anaconda-20.25.15-1.fc20, python-blivet-0.23.9-1.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing anaconda-20.25.15-1.fc20 python-blivet-0.23.9-1.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-23257/python-blivet-0.23.9-1.fc20,anaconda-20.25.15-1.fc20
then log in and leave karma (feedback).

anaconda-20.25.15-1.fc20, python-blivet-0.23.9-1.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days