Bug 565223 - getOutputFrom() failure terminates builds
Summary: getOutputFrom() failure terminates builds
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: rpm
Version: 12
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
Assignee: Panu Matilainen
QA Contact: Fedora Extras Quality Assurance
URL: http://linux.dell.com/files/fedora/Fi...
Whiteboard:
Depends On:
Blocks: F13FTBFS 564757 565045
TreeView+ depends on / blocked
 
Reported: 2010-02-13 20:33 UTC by FTBFS
Modified: 2013-01-10 05:42 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of: 564757
Environment:
Last Closed: 2010-12-03 22:50:38 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description FTBFS 2010-02-13 20:33:25 UTC
+++ This bug was initially created as a clone of Bug #564757 +++

tex-musixtex-0.114-8.fc13.src.rpm Failed To Build From Source against the rawhide tree.  See http://fedoraproject.org/wiki/FTBFS for more information.
If you believe this is actually a bug in another package, do NOT change the component in this bug or close this bug.  Instead, add the appropriate bug number from the other package to the "Depends on" line in this bug.  If the other package does not yet have a bug created that you think matches, please create one.  Doing so helps us properly track bugs and their dependencies, just as we track package dependencies.  (If you close this bug, and the other package is not fixed before the next FTBFS run, a new bug will get created.  Please follow the above advice to avoid such duplication.)

--- Additional comment from ftbfs on 2010-02-13 11:49:36 EST ---

Created an attachment (id=391559)
root.log

root.log for x86_64

--- Additional comment from ftbfs on 2010-02-13 11:49:37 EST ---

Created an attachment (id=391560)
build.log

build.log for x86_64

--- Additional comment from ftbfs on 2010-02-13 11:49:38 EST ---

Created an attachment (id=391561)
mock.log

mock.log for x86_64

--- Additional comment from oget.fedora on 2010-02-13 13:10:16 EST ---

This one built fine for me 
   http://koji.fedoraproject.org/koji/taskinfo?taskID=1983690


I think there was a problem with the builder when you folks built it because the build.log tails:
   + rm -rf /builddir/build/BUILDROOT/tex-musixtex-0.114-8.fc13.x86_64/usr/share/doc/tex-musixtex-0.114
   + /bin/mkdir -p /builddir/build/BUILDROOT/tex-musixtex-0.114-8.fc13.x86_64/usr/share/doc/tex-musixtex-0.114
   + cp -pr MusiXTeXDistribution/copying changelog.txt /builddir/build/BUILDROOT/tex-musixtex-0.114-8.fc13.x86_64/usr/share/doc/tex-musixtex-0.114
   + exit 0
   getOutputFrom(): Broken pipe
   Child returncode was: 1
   EXCEPTION: Command failed. See logs for output.

Shall we just close the bug?

--- Additional comment from ftbfs on 2010-02-13 15:32:55 EST ---

I built everything, and then re-built all the ones that failed.  This one failed to build twice for me.  I don't then know if it's a mock bug, or something else, but will need to investigate.

Do not close this here; I'll dupe it into a mock bug for now.

Comment 1 FTBFS 2010-02-14 04:20:46 UTC
google points to getOutputFrom() failures dating back to 2002, as a problem in rpmbuild.  Moving this there.

Comment 2 Dan Horák 2010-02-15 07:37:19 UTC
I have a package (zabbix) where I see this error quite often.

Comment 3 Jeff Johnson 2010-02-27 05:10:23 UTC
(from ancient memories)

If you are seeing getOutputFrom() EPIPE errors,
you are likely overriding RPM filters that are
exiting before rpm has a chance to write files.

Try reading from stdin in the filter, which will
give rpm to finish writing.

Comment 4 Orcan Ogetbil 2010-03-14 03:32:16 UTC
Any workarounds for when we are not messing with filters at all?

Comment 5 Jeff Johnson 2010-03-14 11:44:24 UTC
Sure, but the "workaround" is largely equivalent to a "fix".

Since getOutputFrom() is being called, an external script is being invoked.

You need to find which script is being invoked.

The annoyance is that this is a race condition" a script is exiting before
rpmbuild has a chance to get ready to read. Otherwise one could just
add

Comment 6 Jeff Johnson 2010-03-14 11:58:26 UTC
(continued)
...
add
    set -x
or equiv and figger which script was exiting. But that
will change all the timing and so will cause rpmbuild to "work".

(aside)
Adding
   sleep 2
at beginning of all helper scripts might be a workaround.
Otherwise make sure all child helpers read _ALL_ of stdin
before exiting.

The next level of "fix" is to adjust the timers to fdReadable/fdWritable
or increase the size of the output write buffer in getOutputFrom().

IIRC, shortening the fdWritable timer is part of what is needed so that rpmbuild
can write the entire file list before the helper exits.

THe best fix of all is not write/read from parent <-> child in the same
loop in rpmbuild.

Comment 7 Jeff Johnson 2010-03-14 12:17:10 UTC
BTW, not only filters, but also any package that
disables the internal dependency generator, starts
to use getOutputFrom(). So the problem likely
tracks with a disabled internal dependency generator.

The othe usage case is with %doc copying, but there's
no data transferred on the helper pipe.

DIsabling the internal dependency generator and setting
one of the %_find_provides/%_find_requires extractors
to /bin/true in some package with a modestly
large (more than 8Kb perhaps much more) file list should
give you an easier reproducer than random failures in a remote
build system.

And replacing /bin/true with a short script that reads
everything from stdin before exiting will indicate the
nature of the fix.

Comment 8 Jeff Johnson 2010-03-14 12:35:54 UTC
One last note ...

Re-reading the build log in comment #1 ... the last commands displayed are
copying %doc files. Which means that all of what rpmbuild traditionally
foes with helper scripts has already been accomplished.

Note "traditionally".

Hint:
    grep rpmfcExec build/files.c
The checkfiles script is the only helper invocation after %doc is copied.

Try draining stdin always, not just when the $RPM_BUILD_ROOT dir exists:

if [ ! -d "$RPM_BUILD_ROOT" ] ; then
        cat > /dev/null
        exit 1
fi

Comment 9 Panu Matilainen 2010-06-30 10:00:24 UTC
The problem in case of tex-musixtex is almost certainly due to font provides extraction not eating stdin when /usr/bin/fq-query isn't there (fontconfig is not installed). That case should be fixed now in rawhide, and can be worked around by adding "BuildRequires: fontconfig".

As of zabbix often triggering this, no "obvious" candidates, unless it contains some data that libmagic thinks of as fonts. But by looks of things, there are cases of exiting without consuming all of stdin in corner cases all over the scripts...

Rawhide rpm now actually tells you what the failing script was so finding the guilty bits is easier. If/when you get more of these from rawhide, please drop a note here.

Comment 10 Jeff Johnson 2010-06-30 11:31:58 UTC
Displaying the name of a the script with the failing behavior
-- while helpful -- is hardly a fix for getOutputFrom() failures.

THe best fix of all is not write/read from parent <-> child in the same
loop in rpmbuild.

Have fun!

Comment 11 Bug Zapper 2010-11-03 22:10:17 UTC
This message is a reminder that Fedora 12 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 12.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '12'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 12's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 12 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 12 Bug Zapper 2010-12-03 22:50:38 UTC
Fedora 12 changed to end-of-life (EOL) status on 2010-12-02. Fedora 12 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.