Created attachment 631226 [details]
File: anaconda-tb

Created attachment 631227 [details]
File: environ

Created attachment 631228 [details]
File: type

Created attachment 631229 [details]
File: ifcfg.log

Created attachment 631230 [details]
File: storage.log

Created attachment 631231 [details]
File: version

Created attachment 631232 [details]
File: program.log

Created attachment 631233 [details]
File: product

Created attachment 631234 [details]
File: anaconda.log

Created attachment 631235 [details]
File: hashmarkername

Created attachment 631236 [details]
File: cmdline_file

Created attachment 631237 [details]
File: release

Created attachment 631238 [details]
File: messages

Created attachment 631239 [details]
File: other involved packages

The following lines out of your syslog look suspicious, and I expect we ought to be getting a lot more bug reports along these lines if there were a real bug here.

Oct 22 00:39:51 localhost kernel: [  467.034080] Buffer I/O error on device dm-1, logical block 491520
Oct 22 00:39:51 localhost kernel: [  467.034084] lost page write due to I/O error on dm-1
Oct 22 00:39:51 localhost kernel: [  467.034086] JBD2: Error -5 detected when updating journal superblock for dm-1-8.

(In reply to comment #15)
> The following lines out of your syslog look suspicious, and I expect we
> ought to be getting a lot more bug reports along these lines if there were a
> real bug here.
...

Define "real bug" ... :-)

Some of these aren't even mine:

Bug 868760 - CreateException: Can't have a partition outside the disk!
Bug 866028 - BootLoaderError: failed to set new efi boot target 
Bug 865849 - ValueError: ('invalid size specification', '0 b')
Bug 864257 - AttributeError: 'NoneType' object has no attribute 'type'
Bug 860393 - SystemError: (32, 'mount: /dev/mapper/live-osimg-min is already mounted or /mnt/install/source busy\n /dev/mapper/live-osimg-min is already mounted on /mnt/install/source') 

https://bugzilla.redhat.com/buglist.cgi?f1=attach_data.thedata&list_id=725161&o1=substring&classification=Fedora&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=MODIFIED&bug_status=ON_DEV&bug_status=ON_QA&bug_status=VERIFIED&bug_status=RELEASE_PENDING&bug_status=POST&v1=Buffer%20I%2FO%20error%20on%20device%20dm-1&component=anaconda&product=Fedora

None of that has anything to do with this bug report.  Please stop confusing things.

The link in Comment 16 is a search of anaconda bug reports with attachments matching:
     Attachment data: Buffer I/O error on device dm-1
So you have indeed received more bug reports with this error.

From Comment 0, this bug report was generated while I was running an edge-test:

Install to 2.65G disc image from Live CD.
Configure '/' only, no swap.

The right thing to do would be to reopen with a needinfo re how to reproduce.

I just tried reporting a bug from Anaconda in Fedora 18 TC7 and was told that my problem was a duplicate and already filed here.

With TC7 I can trivially reproduce this by creating a KVM guest with a 5G disk and booting the Live iso. Then configure the network connection, select the disk and start the installation with the default settings. When the installation starts set a root password and then wait. After a short while Anaconda will crash with the error mentioned above.

(In reply to comment #19)
> I just tried reporting a bug from Anaconda in Fedora 18 TC7 and was told
> that my problem was a duplicate and already filed here.
> 
> With TC7 I can trivially reproduce this by creating a KVM guest with a 5G
> disk and booting the Live iso. Then configure the network connection, select
> the disk and start the installation with the default settings. When the
> installation starts set a root password and then wait. After a short while
> Anaconda will crash with the error mentioned above.

Thanks for your report. Could you attach a log file? The largest one with a name like this would be best:
/tmp/anaconda-tb-*

Created attachment 637576 [details]
anaconda log of failed installation

done

This appears to be related to a lack of available disk space. When using a 10G disk or reducing the swap space on the 5G disk so that enough space becomes available on the root filesystem this crash no longer happens.

Thanks for your follow-up report.

I can't reproduce the traceback starting with a 5120M disc image created with:

$ qemu-img create f18-test-4.img 5120M

That creates a disc image with all zeroes:

$ hexdump -C f18-test-4.img
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
140000000

With that disc image, the installer reports this error:

('new lv is too large to fit in free space', 'fedora')

Have you tried installing to an empty 5120M disc image?

Tested with:
$ qemu-kvm -m 4096 -hda f18-test-4.img -cdrom ~/xfr/fedora/F18/F18-Beta/TC7/Fedora-18-Beta-TC7-x86_64-Live-Desktop.iso -usb -vga qxl -boot menu=on -usbdevice mouse

I'm using "truncate -s 5G <img-name>" to create the image which shouldn't make too much of a difference.

What does make a difference though is the amount of ram assigned to the guest. When I set 4G of ram as you did I also get the 'new lv is too large...' error. But when only set 1G of memory I get a 500MB boot partition, 2G swap and 2.5G root. After a successfull installation on a larger disk I can see that 3.2G of space are used so the 2.5G for root are not sufficient for an install.

Install with 1024M memory to 5120M disc image.

Package: anaconda-18.24-1.fc18.x86_64
OS Release: Fedora release 18

Thanks! The traceback is easily reproduced with 1024M of memory and a 5120M disc image.

I forgot that the installer computes the amount of swap space to allocate based on the amount of memory. From test installs, the installer seemed to use the formula:

    size-of-swap-space == 2 * amount-of-memory

After searching the anaconda source, I found a function, swapSuggestion(), that uses a more complicated formula:

http://git.fedorahosted.org/cgit/anaconda.git/tree/pyanaconda/storage/devicelibs/swap.py

Tested with:
$ qemu-kvm -m 1024 -hda f18-test-4.img -cdrom ~/xfr/fedora/F18/F18-Beta/TC7/Fedora-18-Beta-TC7-x86_64-Live-Desktop.iso -usb -vga qxl -boot menu=on -usbdevice mouse

Both attached log files have the same rsync error messages.[1]

An rsync FAQ explains them:[2]
"Q: Why does my transfer die with something like the following error?"
...
"There are several common causes for a remote rsync process going away:

    The destination disk is full (remember that you need at least the size of the largest file that needs to be updated available in free disk space for the transfer to succeed)."

However, the function LiveImagePayload.install()[3] does not raise an exception. Indeed, it sets "err = None", logs the exit value, and returns as if there were no error. Instead, the installer should immediately terminate the install on any non-zero return value from rsync and display an informative message for the user.

[1] From attached log anaconda-tb-sdYtLk:
Nov  3 13:48:22 localhost program: rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken pipe (32)
Nov  3 13:48:22 localhost program: rsync: connection unexpectedly closed (1658019 bytes received so far) [sender]
Nov  3 13:48:22 localhost program: rsync error: error in rsync protocol data stream (code 12) at io.c(605) [sender=3.0.9]
Nov  3 13:48:22 localhost anaconda: rsync exited with code 12

[2] rsync current issues and debugging
http://rsync.samba.org/issues.html

[3] http://git.fedorahosted.org/cgit/anaconda.git/tree/pyanaconda/packaging/livepayload.py?id=anaconda-18.24-1#n71

The rsync exit values are listed near the end of the rsync man page:
http://rsync.samba.org/ftp/rsync/rsync.html

Created attachment 637953 [details]
screenshot showing PayloadInstallError: "rsync exited with code 12"

This could use some refinement, but it is better than an obscure error message in a log file followed by a traceback in subsequent code:

$ cat livepayload-rsync-fatal-error-1.patch 
--- livepayload.py.ORIG	2012-11-02 20:59:04.000000000 -0400
+++ livepayload.py.EXP1	2012-11-04 05:57:01.876354161 -0500
@@ -84,7 +84,8 @@
         else:
             err = None
             if rc != 0:
-                log.info("%s exited with code %d" % (cmd, rc))
+                err = "%s exited with code %d" % (cmd, rc)
+                log.info(err)
 
         if err:
             exn = PayloadInstallError(err)

*** Bug 872926 has been marked as a duplicate of this bug. ***

(In reply to comment #30)
> *** Bug 872926 has been marked as a duplicate of this bug. ***

After I clicked "Exit Installer" in the PayloadInstallError dialog, I was asked to file a bug report. The user shouldn't need to file a bug report when there is not enough disk space ...

We can't exit on all rsync errors. It normally returns with a non-zero error even though the copy completes.

It *does* look like we need to take a look at the size estimate and maybe add some buffer to it. That's going to be somewhat fuzzy since the source and destination filesystems may be different.

(In reply to comment #32)
> We can't exit on all rsync errors. It normally returns with a non-zero error
> even though the copy completes.

Do you know which non-zero exit values are normal? Those could be white-listed, and the rest could be fatal errors:

             if rc not in rsync_exit_value_whitelist: # [0, rc1, rc2]
                 this_is_a_fatal_error()

> It *does* look like we need to take a look at the size estimate and maybe
> add some buffer to it. That's going to be somewhat fuzzy since the source
> and destination filesystems may be different.

Right. That's why non-zero exit values need to be fatal in most cases. Exit value 12 is definitely bad ...

The rsync exit values are listed near the end of the rsync man page:
http://rsync.samba.org/ftp/rsync/rsync.html

I have to take that back :) By excluding the bind mounted dirs (/dev/ /proc/ and /sys/) from the rsync it exits cleanly, so now I can catch the errors.

Also, it looks like we don't actually check to see if the partition is big enough before we start the install. There are a couple of bugs related to that assigned to dlehman.

Thanks for looking into that. Now we don't have to write up rsync ... :-)

anaconda-18.27-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/anaconda-18.27-1.fc18

Package anaconda-18.27-1.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing anaconda-18.27-1.fc18'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-17823/anaconda-18.27-1.fc18
then log in and leave karma (feedback).

Testing exception handling when rsync returns code 12 after running out of disk space.
See Bug 868755.


Package: anaconda-18.28-1.fc18.x86_64
OS Release: Fedora release 18

Test of exception handling after rsync fails.

Package: anaconda-18.28-1.fc18.x86_64
OS Release: Fedora release 18

Package anaconda-18.28-1.fc18:
* should fix your issue,
* was pushed to the Fedora 18 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing anaconda-18.28-1.fc18'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-17823/anaconda-18.28-1.fc18
then log in and leave karma (feedback).

At the end of installing Fed18 TC8 from Live CD the installation stopped with an rsync error.

Package: anaconda-18.28-1.fc18.x86_64
OS Release: Fedora release 18

(In reply to comment #41)
> At the end of installing Fed18 TC8 from Live CD the installation stopped
> with an rsync error.
> 
> Package: anaconda-18.28-1.fc18.x86_64
> OS Release: Fedora release 18

Please attach the logs from /var/log/*log as individual plain/text files.

*** Bug 875329 has been marked as a duplicate of this bug. ***

These errors are all related to there not being enough space to do the install. This *should* be caught before we leave hub #1.

(In reply to comment #42)
> (In reply to comment #41)
> > At the end of installing Fed18 TC8 from Live CD the installation stopped
> > with an rsync error.
> > 
> > Package: anaconda-18.28-1.fc18.x86_64
> > OS Release: Fedora release 18
> 
> Please attach the logs from /var/log/*log as individual plain/text files.

1) I can't post the logs because I wanted to see how Fed18 worked on this
   notebook and when the TC 8 Live install aborted I used a TC 7 DVD to
   install Fed18. (I had no double sided DVD's in house to put TC 8 on).
2) I doubt if the reason for the abort was "not enough space" as this 
   was on an empty 380 GB disk with default standard partitioning and
   the abort happened well into the 'post installation' activities.
3) the anaconda bug report should have contained some indication

Your bug report (Comment 41) was classified as a duplicate by libreport, and duplicate reports are added as brief comments without attachments or exception strings:
Bug 875312 - exception string not included in report of duplicate
Bug 875233 - comments reporting duplicates not clearly identified as libreport comments

The best we can do is ask if you can recall the rsync error code, and if your bug report was forwarded from:
Bug 872926 - PayloadInstallError: rsync exited with code 12
(Bug 872926 is closed as a duplicate of this bug report, so libreport forwards any reports classified as a duplicate of it to this bug report.)

Complaint to Anaconda Developers:

When you change the bug summary from:
"error: rpmdb open failed"
to:
"livecd doesn't check for enough space to install"

you throw away information that I need to find bug reports.

If you want to add your interpretation of what the bug is really about, please do so by _appending_ your interpretation in parentheses. You are not the only ones who have to read and interpret bug reports. And the first thing I look at is the bug summary. I was looking for this bug in the list of anaconda bug reports and could not figure out what happened to it until I went back and found an email concerning this bug.

(In reply to comment #46)

> Your bug report (Comment 41) was classified as a duplicate by libreport, and
> duplicate reports are added as brief comments without attachments or
> exception strings.

> The best we can do is ask if you can recall the rsync error code, and if
> your bug report was forwarded from:
> Bug 872926 - PayloadInstallError: rsync exited with code 12

Yes it was 872926.

If it is important I could do a fresh install of TC8 Live.
(But that would have to wait till Sunday).

With TC8 I deleted all the Fed 17 partitions to get an 'empty' disk
and then used the default partitions offered by TC8. No problems till
the 'end' of 'post installation'.

Popped up during install of F18 Beta TC8

Package: anaconda-18.28-1.fc18.x86_64
OS Release: Fedora release 18

In my case the error code was:

err: rsync exited with code 23

In this case I used F18-TC8 live cd to install over my F17 install which used raw ext4 partitions for /, /home, /boot and /boot/efi - all of which I told anaconda to reuse and reformat (except for /home which I did not specify reformat) - so I don't think in my case the problem was a lack of disk space...

The following errors were in /var/log/messsages:

Nov 13 07:41:13 localhost program: rsync: rsync_xal_set: lsetxattr(""/mnt/sysimage/boot/efi"","security.selinux") failed: Operation not supported (95)
Nov 13 07:41:13 localhost program: rsync: rsync_xal_set: lsetxattr(""/mnt/sysimage/boot/efi/.mach_kernel.pfjcF9"","security.selinux") failed: Operation not supported (95)
Nov 13 07:41:13 localhost program: rsync: rsync_xal_set: lremovexattr(""/mnt/sysimage/boot/efi/.mach_kernel.pfjcF9"","hfs.type") failed: Operation not supported (95)
Nov 13 07:41:13 localhost program: rsync: rsync_xal_set: lremovexattr(""/mnt/sysimage/boot/efi/.mach_kernel.pfjcF9"","hfs.creator") failed: Operation not supported (95)
.... removed unrelated messages ...
Nov 13 07:43:20 localhost program: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1052) [sender=3.0.9]
Nov 13 07:43:20 localhost anaconda: rsync exited with code 23

In this case it looks like an selinux problem stopping some of the efi related files from being rsynced...

(In reply to comment #47)
> Complaint to Anaconda Developers:
> 
> When you change the bug summary from:
> "error: rpmdb open failed"
> to:
> "livecd doesn't check for enough space to install"
> 
> you throw away information that I need to find bug reports.
> 
> If you want to add your interpretation of what the bug is really about,
> please do so by _appending_ your interpretation in parentheses. You are not
> the only ones who have to read and interpret bug reports. And the first
> thing I look at is the bug summary. I was looking for this bug in the list
> of anaconda bug reports and could not figure out what happened to it until I
> went back and found an email concerning this bug.

Please don't tell us how to handle our bug reports.  bugzilla has pretty thorough searching facilities, and you can very easily search through for bug reports containing that string.  This is the bug queue we need to work from, and it needs to match out work flow - not anyone else's.

(In reply to comment #51)
...
> Please don't tell us how to handle our bug reports.  bugzilla has pretty
> thorough searching facilities, and you can very easily search through for
> bug reports containing that string.  This is the bug queue we need to work
> from, and it needs to match out work flow - not anyone else's.

They aren't _your_ bug reports, Chris. This is FOSS, remember? And you aren't the only ones with a "work flow" ...

The primary consumer and fixer of anaconda bug reports is the anaconda team.  If you want to be involved with working in anaconda bugs, you will need to adapt to how we work with bugs.  That involves us renaming things to be more meaningful *for us*, reassigning things, etc.  Lecturing us on how we should work with our bugs doesn't help get your reports fixed faster.

rsync fails to set xattrs on files lying on /boot/efi.
This is correct and not an error because fat16/32 does not support xattrs.


Package: anaconda-18.28-1.fc18.x86_64
OS Release: Fedora release 18

Proposing as blocker for Beta - this will break any live UEFI install.

The latest case of this is not fixed in 18.29.1, so setting back to ASSIGNED, bcl is working on an anaconda-side fix for this which will involve excluding some rsync exit codes from being fatal.

+1 blocker, UEFI installs need to work.

slight +1 blocker, UEFI live installs should work but there are workarounds (use the installer)

[f18-beta2-branch] only raise rsync error on error 12 (#868755)
https://lists.fedorahosted.org/pipermail/anaconda-patches/2012-November/002170.html

If I'm reading this patch right, we are going to get an empty error message[1] when rsync returns 12 ...

ISTM, an rsync wrapper that raised exceptions would make this more Python-like. As it is, the code reads like a conflation of C-style return-code checking and Python-style exception handling.

[1] http://git.fedorahosted.org/cgit/anaconda.git/tree/pyanaconda/packaging/livepayload.py?id=2032f1ee0d43df6e0cddf4004fd71a2ce7b63c67#n114

summary: rsync error codes don't always mean it failed to copy. On UEFI you can't rsync the xattrs so an error 23 is normal. But on the other hand if the install runs out of space it will return an error 12.

As it affects UEFI live installs, I'm +1 blocker but as Tim pointed out - we could point out the workaround for UEFI live installs in case of emergency.

That makes it +4 blocker (with Adam's implied +1) - moving to accepted.

$ man rsync
       23     Partial transfer due to error

That could mean other problems occurred too. 

It is becoming apparent the rsync is the wrong tool for the job. It doesn't return error codes that help callers figure out what actually occurred. It doesn't reliably return meaningful error messages. And it doesn't integrate well with Python programs.

Isn't there a Python module that could be used instead?

install with dd TC8 Live Desktop x86_64 to 8GB USB as / only (no swap) 2.650GB 2648 got to 90% installed and failed rsync exited with error code 12

Package: anaconda-18.28-1.fc18.x86_64
OS Release: Fedora release 18

2nd bug from install

Package: anaconda-18.28-1.fc18.x86_64
OS Release: Fedora release 18

Reproduced "rsync exited with code 23".
Create a 500 MB /boot/efi mount point with the installer.


Package: anaconda-18.29.1-1.fc18.x86_64
OS Release: Fedora release 18

(In reply to comment #66)
> Reproduced "rsync exited with code 23".
> Create a 500 MB /boot/efi mount point with the installer.
> 
> 
> Package: anaconda-18.29.1-1.fc18.x86_64
> OS Release: Fedora release 18

When I created /boot/efi, the installer configured it this way:
Device Type: Standard Partition
File System: EFI System Partition

If the installer understands "EFI System Partition"s, it should understand how to install to them. ISTM, that could be accomplished by calling rsync with the appropriate arguments for an "EFI System Partition" -- without "-X", say.

BTW, the installer would not let me create vfat "/" or "/boot" mount points, so the installer already understands that some file systems are special.

Created attachment 648839 [details]
[18.29.1 log anaconda-tb-flLbPu] PayloadInstallError: rsync exited with code 23

This is the log file for the reproducer in Comment 66, Comment 67.

anaconda-18.29.2-1.fc18 has been submitted as an update for Fedora 18.
https://admin.fedoraproject.org/updates/FEDORA-2012-18468/anaconda-18.29.2-1.fc18

(In reply to comment #68)
> Created attachment 648839 [details]
> [18.29.1 log anaconda-tb-flLbPu] PayloadInstallError: rsync exited with code
> 23
> 
> This is the log file for the reproducer in Comment 66, Comment 67.

The installer shouldn't be trying to do this on /boot/efi either:

15:30:13,030 INFO anaconda: failed to set SELinux context for /mnt/sysimage/boot/efi: [Errno 95] Operation not supported

I don't hit this issue on F18 Beta RC1 anymore.

BTW, as it was mentioned here earlier, the title of the bug is incorrect, the rsync error is not limited to "not enough space" according to my testing.

Attempting to install to a dual boot system with Windows XP.
The installer appears to be putting "/" on the unpartitioned, "hidden" 8 MB disk space after the extended partition.
Reproduced twice.

F18-Beta-RC1 Live CD on a USB stick


Package: anaconda-18.29.2-1.fc18.i686
Architecture: i686
OS Release: Fedora release 18

That doesn't have anything to do with rsync. Open a new bug with full logs.

anaconda-18.29.2-1.fc18 has been pushed to the Fedora 18 stable repository.  If problems still persist, please make note of it in this bug report.

there seems to be a problem wuth identifying whch free space is a usable target. i have 120G/30G u12.04/800M and I assume it chocked trying to use the final space.