Bug 1368352 - /tmp/packaging.log is not uploaded if Anaconda hits an error before mounting /mnt/sysimage
Summary: /tmp/packaging.log is not uploaded if Anaconda hits an error before mounting ...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Beaker
Classification: Retired
Component: reports
Version: 23
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: 23.2
Assignee: Dan Callaghan
QA Contact: tools-bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-08-19 07:25 UTC by Jan Stodola
Modified: 2016-09-02 07:01 UTC (History)
4 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2016-09-02 07:01:20 UTC
Embargoed:


Attachments (Terms of Use)

Description Jan Stodola 2016-08-19 07:25:01 UTC
Description of problem:
/tmp/packaging.log is not uploaded if the installation fails, for example because of missing dependencies.
This makes debugging of such installations more time consuming, since the reason of the failure is not provided in any other log file. Please, fix it soon.

Version-Release number of selected component (if applicable):
Beaker 23.1 

Actual results:
/tmp/packaging.log not uploaded

Expected results:
/tmp/packaging.log uploaded and visible in web UI

Comment 2 Dan Callaghan 2016-08-25 03:54:34 UTC
/tmp/packaging.log is already on the list of files monitored by anamon, so it should be being uploaded, hmm...

Comment 3 Dan Callaghan 2016-08-25 04:03:41 UTC
Oh I think I got it.

In sys.log I noticed:

10:53:13,533 INFO chronyd: Selected source 10.5.27.10
10:53:13,533 WARNING chronyd: System clock wrong by 14400.007768 seconds, adjustment started
14:53:13,541 WARNING chronyd: System clock was stepped by 14400.007768 seconds

I wondered, in anamon are we sensitive to clock warps? Yes, we are. MountWatcher checks time.time() as part of its logic for looking in /proc/mounts -- it waits until the line for /mnt/sysimage has been present in the file for more than 60 seconds after it was first seen. But if the clock warps backwards in between the time the line was first seen, and before the 60 second wait has succeeded, it will end up waiting for hours instead.

Surely that makes no difference to /tmp/packaging.log, which doesn't need to wait for /mnt/sysimage to appear... but it does, because /tmp/packaging.log was added to the wrong list.

Comment 4 Dan Callaghan 2016-08-25 04:13:05 UTC
http://gerrit.beaker-project.org/5156 anamon: /tmp/packaging.log doesn't need to wait for /mnt/sysimage

Dunno about my chances of successfully reproducing this or proving that it's fixed though, hmm...

Comment 5 Dan Callaghan 2016-08-25 04:15:06 UTC
Also not sure how to fix the problem of anamon being sensitive to clock warping... Python added a monotonic clock in 3.3 but that doesn't help us on any RHELs. Even some craziness like this:

http://stackoverflow.com/a/1205762/120202

won't work on RHEL <= 5 due to missing ctypes. And I wouldn't really be keen to have some mess like that in anamon anyway.

Comment 6 Dan Callaghan 2016-08-25 04:18:21 UTC
(In reply to Dan Callaghan from comment #3)

Actually not sure if this theory holds water anyway, since in that case the clock was being warped *forwards* not backwards so it wouldn't have prevented anamon from considering the mount as stable.

Comment 7 Dan Callaghan 2016-08-25 04:20:21 UTC
The explanation could be simpler... Anaconda might have never mounted /mnt/sysimage because it bailed out with an error before it reached that point.

Comment 8 Dan Callaghan 2016-08-25 04:53:26 UTC
One way to get Anaconda to bail out with an error before mounting /mnt/sysimage is to just add a deliberately broken %packages section -- although it doesn't reproduce the exact same Anaconda error condition as in the recipe in comment 1.

<ks_appends>
<ks_append>
%packages
doesnotexist
%end
</ks_append>
</ks_appends>

Expected result:
Anaconda errors out of the installation. packaging.log should appear in the recipe logs.

Actual result:
Anaconda errors out of the installation, but packaging.log is not present in Beaker.

Confirmed that the patch in comment 4 fixes this.

Comment 13 Dan Callaghan 2016-09-02 07:01:20 UTC
Beaker 23.2 has been released.


Note You need to log in before you can comment on or make changes to this bug.