Bug 989133

Created attachment 779178 [details]
screenshot of virt-manager guest booting F19 x86_64 DVD using "inst.ks=http://robatino.fedorapeople.org/ks.cfg"

Created attachment 779221 [details]
sosreport-1.txt

Confirmed with the F19 DVD and ks=http://robatino.fedorapeople.org/ks.cfg appended to the kernel command line.

Tested with:
$ qemu-kvm -m 4096 -hda f19-test-2.img -cdrom ~/xfr/fedora/F19/Fedora-19-x86_64-DVD.iso -vga std -boot menu=on

Testing note: The dracut shell doesn't have scp, so you have to mount a file system and then copy sosreport.txt to that. I did that by first pre-installing a minimal system on f19-test-2.img and then mounting /dev/sda1 from the dracut shell. Finally, I rebooted into the pre-installed system to scp sosreport.txt to my VM host.

See also: man dracut re /run/initramfs/sosreport.txt

With a 0-byte kickstart file served over NFS, the installer starts as expected:

1. $ touch ks-empty.cfg
2. $ ls -l ks-empty.cfg # verify that the file has read permissions
3. Initialize a disc image:
   $ qemu-img create f19-test-2.img 12G
4. Start the F19 installer DVD:
   $ qemu-kvm -m 4096 -hda f19-test-2.img -cdrom ~/xfr/fedora/F19/Fedora-19-x86_64-DVD.iso -vga std -boot menu=on
5. Append to kernel command line:
   ks=nfs:host.name:/var/tmp/kickstart/ks-empty.cfg

The installer displays the Welcome title and the language menu with English selected. I can tell that it is a kickstart install, because the menu item is at the top of the window, as expected, but it is partially obscured.

NB: The NFS server is the VM host.

Testing note: To test with the NFS server on the VM host:

1. $ cat /etc/exports
/var/tmp/kickstart 192.168.0.0/24(ro,insecure) 10.0.2.0/24(ro,insecure)
   The "insecure" option is required: man exports.
2. $ sudo systemctl start nfs

While we're on the subject of trivial kickstart bugs, thought I'd mention that you can normally use fpaste to create an http downloadable ks file. (This is useful for people who don't have another server to upload to.) To download, you use the corresponding raw URL, *without* the trailing slash. It fails with the trailing slash. The fpaste upload fails with a 0-byte file (which is why I had to use my fedorapeople account), but that's an fpaste issue. Is it expected that ks downloads fail when the URL has a trailing slash? If so, I'll file a separate bug for that.

P.S. When downloading the raw URL using wget, it generates a file called "raw" without the trailing slash, and "index.html" with the slash. Maybe that's related.

Thanks for the fpaste.org tip. That sounds very useful.

The difference in the two URLs (trailing slash vs no trailing slash) is in the HTTP return code:

slash       HTTP/1.1 200 OK
            file is returned
no slash:   HTTP/1.1 302 Found
            no file is returned

I believe the latter should return a redirect with an appended slash
(see below for an example):
HTTP/1.1 301 Moved Permanently

$ curl -I http://fpaste.org/28327/13749621/raw/
HTTP/1.1 200 OK
Date: Sat, 27 Jul 2013 21:57:28 GMT
Server: Apache/2.2.15 (Red Hat)
X-Powered-By: PHP/5.3.3
Content-Disposition: inline; filename="pastedata"
Content-Type: text/plain; charset=UTF-8
Content-Length: 422
X-Varnish: 2115628899 2115628696
Age: 44
Via: 1.1 varnish
ProxyTime: D=773
ProxyServer: proxy01.phx2.fedoraproject.org

$ curl -I http://fpaste.org/28327/13749621/raw
HTTP/1.1 302 Found
Date: Sat, 27 Jul 2013 21:57:22 GMT
Server: Apache/2.2.15 (Red Hat)
X-Powered-By: PHP/5.3.3
Location: /28327/13749621/raw/
Content-Type: text/html
X-Varnish: 2115628887 2115628705
Age: 33
Via: 1.1 varnish
ProxyTime: D=1192
ProxyServer: proxy01.phx2.fedoraproject.org

$ curl -I http://mirrors.kernel.org/fedora/releases/18/Fedora/x86_64/os
HTTP/1.1 301 Moved Permanently
Date: Sat, 27 Jul 2013 22:04:34 GMT
Server: Apache/2.2.15 (Red Hat)
Location: http://mirrors.kernel.org/fedora/releases/18/Fedora/x86_64/os/
Content-Type: text/html; charset=iso-8859-1

(In reply to Andre Robatino from comment #5)
> While we're on the subject of trivial kickstart bugs,
...

Dropping into the dracut shell for no apparent reason is not trivial for the end user. Leaving off the trailing slash is usually acceptable, because the redirect is handled automatically, so your fpaste.org example would baffle anyone ...

Thanks for opening this bug.

This message in the attached sosreport-1.txt comes from fetch-kickstart-net.sh, which is an anaconda shell script:[1]
... anaconda fetching kickstart from http://robatino.fedorapeople.org/ks.cfg

The function fetch_url is defined in a dracut shell script:[2]
/usr/lib/dracut/modules.d/45url-lib/url-lib.sh

That function ultimately calls "curl" for http URLs.

[1] $ less -N anaconda-19.30.13-1/dracut/fetch-kickstart-net.sh
...
     47 info "anaconda fetching kickstart from $kickstart"
     48 if fetch_url "$kickstart" /tmp/ks.cfg; then
     49     parse_kickstart /tmp/ks.cfg
     50     run_kickstart
     51 else
     52     warn "failed to fetch kickstart from $kickstart"
     53 fi

[2] dracut-029-2.fc19.x86_64

The fix here appears to be to redirect the curl output to stdout:
curl --output - $url > x.txt
That will create an empty file if curl returns no output. Details follow:

Using the curl arguments from url-lib.sh, curl succeeds and returns an exit code of zero:

$ curl --globoff --location --retry 3 --fail --show-error --output x.txt http://robatino.fedorapeople.org/ks.cfg
$ echo $?
0

However, no file x.txt is created.

If the curl output is directed to stdout, x.txt is created, and it is empty:

$ curl --globoff --location --retry 3 --fail --show-error --output - http://robatino.fedorapeople.org/ks.cfg > x.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
$ echo $?
0
$ ls x.txt 
x.txt
$ hexdump -C x.txt
$ 

If the file exists, this test will fail, and no error will be reported:
Snippet from url-lib.sh:
...
     73     if ! [ -f "$outloc" ]; then
     74             warn "Downloading '$url' failed!"
     75             return 253
     76     fi
...

See the "--output" option in the curl man page:

"Specifying the output as '-' (a single dash) will force the output to be done to stdout."

(In reply to Steve Tyler from comment #6)
...
> The difference in the two URLs (trailing slash vs no trailing slash) is in
> the HTTP return code:
> 
> slash       HTTP/1.1 200 OK
>             file is returned
> no slash:   HTTP/1.1 302 Found
>             no file is returned
...

With the "--location" option, curl will follow redirects:

With slash:
$ curl http://fpaste.org/28357/98958213/raw/
hello, world!

Without slash:
$ curl http://fpaste.org/28357/98958213/raw
$ curl --location http://fpaste.org/28357/98958213/raw
hello, world!

Note that dracut invokes curl with the "--location" option.

(In reply to Andre Robatino from comment #5)
> ... To
> download, you use the corresponding raw URL, *without* the trailing slash.
> It fails with the trailing slash.
...

Before invoking fetch_url, fetch-kickstart-net.sh appends a string to the URL, if the URL ends with a slash. That could explain the difference you are seeing between URLs ending with and without a slash. The dracut error message shows the URL is no longer valid. Do you get something like this?

http://fpaste.org/28358/99156213/raw/10.0.2.15-kickstart
                                     ^^^^^^^^^^^^^^^^^^^

$ less -N anaconda-19.30.13-1/dracut/fetch-kickstart-net.sh
...
     42 # kickstart is appended with '$IP_ADDR-kickstart' if ending in '/'
     43 if [ "${kickstart%/}" != "$kickstart" ]; then
     44     kickstart="${kickstart}${new_ip_address}-kickstart"
     45 fi
...

Yes, that's exactly what happens (in my case the IP is 192.168.122.202). BTW, people would have to be told to use the long form of the fpaste URL and not try to append "/raw" to the short URL (which is tempting since it's easier to write down). That doesn't work and in fact redirected me to a random twitter page when I tried it in the browser.

Thanks for confirming that. I thought you were getting a 0-byte file when a slash was appended to the URL. Those random short URLs could be "interesting" ... :-)

(In reply to Andre Robatino from comment #5)
> ... The fpaste upload fails with a 0-byte file
> (which is why I had to use my fedorapeople account), but that's an fpaste
> issue. ...

How are you doing that?

If I press Enter once in the fpaste.org window, I get:

$ curl -s http://fpaste.org/28419/50371391/raw/ | hexdump -C
00000000  0d 0a                                             |..|
00000002

With that as a kickstart file, the installer starts as expected and displays the Welcome dialog with English selected.

I've been using command-line fpaste, not the website.

(In reply to Steve Tyler from comment #7)
...
> Dropping into the dracut shell for no apparent reason is not trivial for the
> end user.
...

Bug 989255 - RFE: display graphical error alert instead of dracut shell when invalid kickstart URL is detected

(In reply to Andre Robatino from comment #15)
> I've been using command-line fpaste, not the website.

OK, thanks:
Bug 989261 - [abrt] fpaste-0.3.7.1-8.fc19: fpaste:414:main:TypeError: 'bool' object is not iterable

so, this is not a dracut bug?

Harald: Why are you closing this instead of changing the component? Fedora is not just about your little dracut world. Please reopen and reassign to anaconda.

BTW, did you read the last sentence of Comment 0?

Thanks for reopening, Andre.

Harald: If you going to maintain an infrastructure component, you are going to have to understand your clients. anaconda has a number of dracut modules -- please see Comment 8.[1]

Closing this bug without even asking first is incredibly rude.
You should apologize.

[1] $ sudo repoquery 'anaconda-dracut'
anaconda-dracut-0:19.30.13-1.fc19.x86_64

Well, I'm wrong. Comment 9 proposes a fix, and it is a fix in dracut, not anaconda:

$ rpm -qf /usr/lib/dracut/modules.d/45url-lib/url-lib.sh
dracut-029-2.fc19.x86_64

Apending <ip>-kickstart if the kickstart URL ends in a slash is the expected behavior. See http://wwoods.fedorapeople.org/doc/boot-options.html#_inst_ks, bug 910550.

As noted above, curl does not write any output when it fetches an empty file, which causes dracut's fetch_url to fail. So, if you want anaconda to be able to fetch a 0-byte kickstart and keep plugging along as if that's a normal thing to do, this is a dracut bug.

commit 5c274467d6a4959b7f451f9a211ffbb38d69b641

Thanks, Harald:

url-lib.sh: handle 0-size files with curl
https://bugzilla.redhat.com/show_bug.cgi?id=989133#c9

http://git.kernel.org/cgit/boot/dracut/dracut.git/commit/?id=5c274467d6a4959b7f451f9a211ffbb38d69b641

dracut-032-19.el7

This message is a notice that Fedora 19 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 19. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained. Approximately 4 (four) weeks from now this bug will
be closed as EOL if it remains open with a Fedora 'version' of '19'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 19 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Fedora 19 changed to end-of-life (EOL) status on 2015-01-06. Fedora 19 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.