Bug 790348 - If specified repo= doesn't contain package repository, fall back to default online repos
If specified repo= doesn't contain package repository, fall back to default o...
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: anaconda (Show other bugs)
17
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Anaconda Maintenance Team
Fedora Extras Quality Assurance
AcceptedBlocker
:
Depends On:
Blocks: F17Blocker/F17FinalBlocker
  Show dependency treegraph
 
Reported: 2012-02-14 05:18 EST by Kamil Páral
Modified: 2012-05-08 00:18 EDT (History)
15 users (show)

See Also:
Fixed In Version: anaconda-17.25-1.fc17
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-05-08 00:18:16 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
shot1.png (35.89 KB, image/png)
2012-02-14 05:19 EST, Kamil Páral
no flags Details
shot2.png (46.93 KB, image/png)
2012-02-14 05:19 EST, Kamil Páral
no flags Details
shot3.png (29.83 KB, image/png)
2012-02-14 05:19 EST, Kamil Páral
no flags Details
anaconda.log (14.78 KB, text/plain)
2012-02-14 05:20 EST, Kamil Páral
no flags Details
anaconda-yum.conf (367 bytes, text/plain)
2012-02-14 05:20 EST, Kamil Páral
no flags Details
ifcfg.log (574 bytes, text/plain)
2012-02-14 05:20 EST, Kamil Páral
no flags Details
program.log (65.03 KB, text/plain)
2012-02-14 05:20 EST, Kamil Páral
no flags Details
storage.log (186.75 KB, text/plain)
2012-02-14 05:20 EST, Kamil Páral
no flags Details
syslog (61.73 KB, text/plain)
2012-02-14 05:20 EST, Kamil Páral
no flags Details
X.log (24.34 KB, text/plain)
2012-02-14 05:20 EST, Kamil Páral
no flags Details
yum.log (1.14 KB, text/plain)
2012-02-14 05:20 EST, Kamil Páral
no flags Details

  None (edit)
Description Kamil Páral 2012-02-14 05:18:49 EST
Description of problem:
I booted with this command:

# virt-install --name Foo --ram 1024 --disk /dev/vg/Foo,size=10 --location http://download.englab.brq.redhat.com/pub/fedora/fedora-alt/stage/17-Alpha.TC2/Fedora/i386/os/ --arch=i386 --extra-args "root=live:http://download.englab.brq.redhat.com/pub/fedora/fedora-alt/stage/17-Alpha.TC2/Fedora/i386/os/LiveOS/squashfs.img"

In GUI in the step where I should select yum repositories, I am given an error message "no more mirrors to try". The pre-configured "Installation Repo" points to the directory where I booted the vmlinuz+initrd from. According to the documentation, this "Installation Repo" shouldn't be present at all and the default online repositories should be used (I checked they are defined in /etc/anaconda.repos.d).

http://fedoraproject.org/wiki/Anaconda/Kickstart#repo
"By default, anaconda has a configured set of repos taken from /etc/anaconda.repos.d plus a special Installation Repo in the case of a media install."

Version-Release number of selected component (if applicable):
anaconda 17.7

How reproducible:
always
Comment 1 Kamil Páral 2012-02-14 05:19:50 EST
Created attachment 561852 [details]
shot1.png
Comment 2 Kamil Páral 2012-02-14 05:19:54 EST
Created attachment 561853 [details]
shot2.png
Comment 3 Kamil Páral 2012-02-14 05:19:58 EST
Created attachment 561854 [details]
shot3.png
Comment 4 Kamil Páral 2012-02-14 05:20:28 EST
Created attachment 561855 [details]
anaconda.log
Comment 5 Kamil Páral 2012-02-14 05:20:31 EST
Created attachment 561856 [details]
anaconda-yum.conf
Comment 6 Kamil Páral 2012-02-14 05:20:35 EST
Created attachment 561857 [details]
ifcfg.log
Comment 7 Kamil Páral 2012-02-14 05:20:38 EST
Created attachment 561858 [details]
program.log
Comment 8 Kamil Páral 2012-02-14 05:20:41 EST
Created attachment 561859 [details]
storage.log
Comment 9 Kamil Páral 2012-02-14 05:20:45 EST
Created attachment 561860 [details]
syslog
Comment 10 Kamil Páral 2012-02-14 05:20:49 EST
Created attachment 561861 [details]
X.log
Comment 11 Kamil Páral 2012-02-14 05:20:52 EST
Created attachment 561862 [details]
yum.log
Comment 12 Kamil Páral 2012-02-14 05:24:34 EST
This kinda breaks pxeboot and kickstart (unless some repos are manually defined). Hard to say which milestone this should block, proposing as Beta blocker.
Comment 13 Vratislav Podzimek 2012-02-14 05:40:34 EST
from anaconda.log:
10:03:40,019 INFO loader: kernel command line:
10:03:40,019 INFO loader:     method=http://download.englab.brq.redhat.com/pub/fedora/fedora-alt/stage/17-Alpha.TC2/Fedora/i386/os/


10:04:12,745 ERR loader: Error downloading http://download.englab.brq.redhat.com/pub/fedora/fedora-alt/stage/17-Alpha.TC2/Fedora/i386/os//images/product.img: HTTP response code said error
10:04:13,137 INFO loader: Running anaconda script /sbin/anaconda
10:04:13,338 INFO anaconda: /sbin/anaconda 17.7
10:04:13,339 WARN anaconda.stdout: Boot argument 'method' is deprecated. In the future, use 'inst.method'.
10:04:14,767 INFO loader: 1048576 kB (1024 MB) are available
10:04:14,782 INFO anaconda: anaconda called with cmdline = ['/sbin/anaconda', '--graphical', '--selinux', '--lang', 'en_US.UTF-8', '--keymap', 'us', '--repo', 'http://download.englab.brq.redhat.com/pub/fedora/fedora-alt/stage/17-Alpha.TC2/Fedora/i386/os/']

Why do you need "--location=..."? You specify the repo this way.

Moreover this should definitely result in 'repo=' not 'method=' adding crobinso@redhat.com to CC list.
Comment 14 Kamil Páral 2012-02-14 05:59:05 EST
I didn't know that virt-install also adds 'method=' kernel option when used with --location. That's the reason why anaconda is confused.

Reassigning to virt-install. The problem is that --location supposes that the same directory contains both kernel+initrd and yum package repository. But for development composes this is not true, there's just the kernel+initrd pair. And we have to use default online yum repositories for installation.

As far as I understand this, either a change in virt-install behavior is needed (e.g. a new option that will suppress the additional 'method=' argument), or anaconda must be able to intelligently deal with nonexistent repositories (having some boot/kickstart option to suppress these error messages).
Comment 15 Vratislav Podzimek 2012-02-14 06:06:58 EST
(In reply to comment #14)
> I didn't know that virt-install also adds 'method=' kernel option when used
> with --location. That's the reason why anaconda is confused.
> 
> Reassigning to virt-install. The problem is that --location supposes that the
> same directory contains both kernel+initrd and yum package repository. But for
> development composes this is not true, there's just the kernel+initrd pair. And
> we have to use default online yum repositories for installation.
> 
> As far as I understand this, either a change in virt-install behavior is needed
> (e.g. a new option that will suppress the additional 'method=' argument), or
> anaconda must be able to intelligently deal with nonexistent repositories
> (having some boot/kickstart option to suppress these error messages).
Anaconda has no way to determine whether a nonexistent repo was provided by virt-install or by user (in that case the "screenshotted" behaviour is adequate).
Comment 16 Cole Robinson 2012-02-14 07:37:57 EST
Would be nice if we could depend on .treeinfo packagedir= here, but that seems to be empty on at least f16 GA .treeinfo as well so it isn't reliable. Anyone know where to file a bug for that?

And if method= is deprecated, can anyone tell me the earliest that repo= was introduced? RHEL5? RHEL6? Just so I know where to do the cut off.
Comment 17 Kamil Páral 2012-02-14 07:43:25 EST
I am removing Beta blocker, because this most probably influences only development composes. virt-install will work fine with official public composes.

However, we would like to have this solved soon, because our automated installation test is blocked on this.

There appears to be one workaround and that is to re-define the method= option using --extra-args.
Comment 18 Vratislav Podzimek 2012-02-15 06:33:08 EST
(In reply to comment #16)
> Would be nice if we could depend on .treeinfo packagedir= here, but that seems
> to be empty on at least f16 GA .treeinfo as well so it isn't reliable. Anyone
> know where to file a bug for that?
Lorax creates .treeinfo file, but at the time when it has no idea where the packagedir would be located. This info could be provided by tools creating trees like http://download.englab.brq.redhat.com/pub/fedora/fedora-alt/stage/17-Alpha.TC2/Fedora/i386/os/ but as it can be seen from this particular example there could be a tree without any packagedir.
So I think .treeinfo would not be a good source of info. Rather do not automatically provide any 'repo=...' and let the user to set it with an option.

> 
> And if method= is deprecated, can anyone tell me the earliest that repo= was
> introduced? RHEL5? RHEL6? Just so I know where to do the cut off.
RHEL5 uses 'method=' RHEL6 use 'repo='.
Comment 19 Kamil Páral 2012-02-15 06:45:57 EST
1. Can virt-install check whether the package repo is present and add 'repo=' option only when it is?

2. The default behavior (adding 'repo=' option) is fine and desired for most users using final composes, it creates problems just for people trying out development composes. If solution 1) is not possible, why not simply add a new option, that says "just boot from the provided kernel+initrd, but don't do any further magic, like adding further boot options"?
Comment 20 Kamil Páral 2012-02-15 07:05:44 EST
I have found a simple workaround, just use --extra-args "method=". But that's highly implementation specific and may change if anaconda code changes.
Comment 21 Cole Robinson 2012-03-01 18:06:16 EST
Okay, thinking about this some, it's not an easy situation.


To clarify, the reason we specify method= is a convenience for the user: virt-install already knows the URL, pass it to anaconda so the user doesn't need to type it again into anaconda text UI. This is doubly convenient since copy+paste between host guest isn't going to work here. Though this isn't essential functionality so if it regressed it's not the complete end of the world.

Options here are:

- Check .treeinfo, if no packagedir=, don't specify method= to anaconda. Sounds great in theory but as previously mentioned that packagedir field is historically useless, so we would have to conditionalize on fedora versions which sucks. Otherwise this would be the ideal way.

- Poke into the tree, only specify a method= argument if we see the Packages directory. This sucks because if the tree layout ever changes for say a new version of fedora, we have to patch virtinst to handle it, otherwise method= isn't specified when it could be. Those types of things are supposed to be what treeinfo saves us from.

How does anaconda error in this case? If it just throws an error message and then gives an option to continue, specifying actual install media, then it's probably not worth patching around in virt-install.
Comment 22 Kamil Páral 2012-03-02 03:31:35 EST
Cole, you're right. I have to repeat that AFAIK this only concerns Fedora development composes, which are not fully created (package tree is missing). For Final (and official Alpha, Beta) composes this should not be a problem. I don't propose any drastic changes.

After reading the previous comment, I think this use case can be easily solved by anaconda. Currently it's completely adamant that "Installation Repo" (specified by method=) must work. Look at the screenshots. But why? Let's pop up an error message if the URL is not available, why not. But allow user to cancel that dialog. That would move user into package+repo selection screen. The Installation Repo would get disabled. The user would be free to enable other pre-defined repositories (the online ones) or define a custom one.

Current anaconda behavior is too harsh.

If you agree, let's assign back to anaconda.
Comment 23 Cole Robinson 2012-03-02 09:42:25 EST
Agreed, reassigning to anaconda.
Comment 24 Kamil Páral 2012-03-28 05:21:28 EDT
In the latest QA meeting Will Woods claimed that anaconda should gracefully continue if the yum repository is not present in the repo= location. That is not the case as this bug documents. We have found another use case for this (apart from using Branched composes), it is described in bug 805166 comment 13 and comment 10.
Comment 25 Adam Williamson 2012-03-28 16:32:05 EDT
Note also this bug I filed yesterday:

http://bugzilla.redhat.com/show_bug.cgi?id=807562

anaconda seems to be entirely too insistent that the repository you pass with repo= be a *complete* repository, at present.



-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers
Comment 26 Kamil Páral 2012-04-19 17:44:37 EDT
Proposing as F17 Final Blocker due to our criteria:

"It must be possible to install by booting the installation kernel directly, including via PXE, and correctly specifying a remote source for the installer itself, using whichever protocols are required to work for package retrieval at the current phase (Alpha, Beta, Final). This must work if the remote source is not a complete repository but contains only the files necessary for the installer itself to run."
https://fedoraproject.org/wiki/Fedora_17_Beta_Release_Criteria

Currently people can't run just the installer from PXE and download the packages from online repos. Anaconda insists that a full package repository is mirrored. In F16 and earlier it worked.
Comment 27 Adam Williamson 2012-04-20 14:14:22 EDT
Discussed at 2012-04-20 blocker review meeting - http://meetbot.fedoraproject.org/fedora-bugzappers/2012-04-20/fedora-bugzappers.2012-04-20-17.01.log.txt . Agreed this is accepted as a blocker per the criterion cited in comment #26.
Comment 28 Tim Flink 2012-05-01 18:09:16 EDT
Has there been any movement on a fix or a workaround for this? Have we been able to confirm that full release builds won't be affected by this bug?
Comment 29 Will Woods 2012-05-02 14:28:18 EDT
I think anaconda's current behavior for repo=XXX is reasonable: if you pass repo=XXX, we expect XXX to actually be a repo.

If the user (or virt-install) is sure that XXX is, in fact, a repo, then repo=XXX will make anaconda use that repo as its sole install source. This is the expected behavior - and it works fine for regular, non-mangled release trees. So no fix is necessary there.

If you're only using the URL to get the stage2 image, stage2=XXX is what you want. That does the install using the default repos (or whatever other repos you might set up otherwise). This satisfies the blocker criterion above. So no fix necessary here either.

So: I think the current anaconda behavior is fine as-is, and this can be dropped from the blocker list once someone verifies that using stage2=XXX works as described above.

In the future we might provide a way to specify that the default repos should be used *in addition* to the commandline/kickstart specified ones, but I don't think that exists right now.

The remaining question is what virt-install should do. I'd suggest using .treeinfo to figure out whether the target is a repo or not (as described in comment 21), but if virt-install doesn't want to confirm (or require) that the URL given is actually a complete repo, it should probably be using stage2=XXX instead.
Comment 30 Cole Robinson 2012-05-02 15:54:47 EDT
Given that this bug has F17Blocker, and that we are talking about more than just virt-install issues here (PXE is explicitly mentioned several times), just reassigning it at this stage is not helpful. Moving this back to anaconda.

If this shouldn't be a blocker, let's engage QA and get it removed from the tracker. If this is NOTABUG, let's convince QA and then close it.

If this is entirely issues in other components like virtinst, cobbler, etc, let's convince QA, and then file new bugs or reassign this bug.

It certainly sounds like this bug is violating the release criteria mentioned in comment #26... but then again I don't understand the motivation for the release criteria explicitly mentioning incomplete repos, or why test composes need to be incomplete repos, or anaconda's historical behavior here. But I assume the folks that gave blocker+ have better understanding (though the meeting log link is basically empty)
Comment 31 Adam Williamson 2012-05-02 16:36:50 EDT
cole: if stage2= works as described, then I agree the criterion is not violated: it _is_ possible to do an install using a 'repo' that contains only a stage2 image. But we should test that. Kamil or Tim, could you do so prior to the blocker meeting?



-- 
Fedora Bugzappers volunteer triage team
https://fedoraproject.org/wiki/BugZappers
Comment 32 Brian Lane 2012-05-02 20:16:47 EDT
stage2 isn't currently working right if there is no .treeinfo, patch sent to list.
Comment 33 Kamil Páral 2012-05-03 03:30:54 EDT
(In reply to comment #32)
> stage2 isn't currently working right if there is no .treeinfo, patch sent to
> list.

This will satisfy the criterion when it works, thanks Brian.

But we would still prefer virt-install to work even with incomplete repos (example use case: you want to use newer/custom anaconda build with default online repos). I see three ways:

1. virt-install can distinguish complete and incomplete repositories (e.g. checking for Packages/ directory) and pass either repo= (for complete repository) or stage2= (for incomplete repository) to anaconda.

Will, can you tell us how to distinguish complete and incomplete repositories using .treeinfo? 'packagedir=' seems to be empty in both, I wonder why. Shouldn't there be a relative path if the repository contains Packages/?

2. virt-install can provide an option that will disable passing repo= to anaconda. The user can then specify 'stage2=' manually using --extra-args.

3. virt-install can provide an option to boot from a specific kernel+initrd pair. When looking into its manpage, I see only --location, which tries to be clever and passes repo= afterwards. Then I see --boot, which also modifies post-install boot configuration. But I don't see --kernel FILE and --initrd FILE options. Using that and specifying --extra-args 'stage2=' would also work.

These virt-install changes are not a blocker though (IMHO). We can create a new bug report about it because this one will be probably closed with that stage2= fix.
Comment 34 Kamil Páral 2012-05-03 05:23:22 EDT
(In reply to comment #33)
> (In reply to comment #32)
> > stage2 isn't currently working right if there is no .treeinfo, patch sent to
> > list.
> 
> This will satisfy the criterion when it works, thanks Brian.
> 

Thinking more about this - our incomplete repos usually do contain .treeinfo, so this is a nice bonus, but it's probably not fully mandatory. However, wouldn't it be easier to use a direct link to squashfs.img rather then a link to the top directory (i.e. stage2=http://.../squashfs.img)? Do you need some other files from that directory? If not, this behavior would be much easier for those guys who want to set up a PXE with just anaconda and use online repos. They wouldn't have to create a 'fake' directories mirroring current layout (/LiveOS/ etc). They would just download three files (kernel, initrd, squashfs.img) and put that into pxelinux.cfg/default.
Comment 35 Fedora Update System 2012-05-04 18:59:55 EDT
anaconda-17.25-1.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/anaconda-17.25-1.fc17
Comment 36 Fedora Update System 2012-05-04 22:09:30 EDT
Package anaconda-17.25-1.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing anaconda-17.25-1.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-7353/anaconda-17.25-1.fc17
then log in and leave karma (feedback).
Comment 37 Kamil Páral 2012-05-07 06:35:51 EDT
I have created a separate bug 819465 for tracking changes inside virt-install wrt stage2= support. Let's leave this bug solely for stage2= changes inside anaconda.
Comment 38 Fedora Update System 2012-05-08 00:18:16 EDT
anaconda-17.25-1.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.