Bug 1413827

Summary: [RFE] install failure detection should catch "This is a fatal error and installation will be aborted" message, even when broken across lines
Product: [Retired] Beaker Reporter: Dan Callaghan <dcallagh>
Component: generalAssignee: Roman Joost <rjoost>
Status: CLOSED DUPLICATE QA Contact: Roman Joost <rjoost>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 24CC: achatter, dcallagh, mjia, mtyson, rjoost
Target Milestone: ---Keywords: FutureFeature, Patch
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-03-15 23:42:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Dan Callaghan 2017-01-17 04:57:22 UTC
On our KVM guests, Fedora installation failures can manifest with console output like the following (include the strange wrapping and extraneous blank lines):


The following error occurred while installing.  This is a fatal error and instal      

lation will be aborted. 

Failed to download the following packages: Cannot download Packages/l/libxshmfen      

ce-1.2-3.fc24.x86_64.rpm: All mirrors were tried 



It would be good if Beaker's installation failure detection would catch this.

Comment 2 Anwesha Chatterjee 2018-02-01 06:34:29 UTC
https://gerrit.beaker-project.org/#/c/5990/

Comment 5 Dan Callaghan 2018-03-07 07:12:09 UTC
(In reply to Roman Joost from comment #4)
> Error      
>    The following error occurred while installing.  This is a fatal error and 
>    installation will be aborted. 
>     
>    Failed to download the following packages: Cannot download 
>    thttpd-2.27-1.fc27.x86_64.rpm: All mirrors were tried      

This message appears to be neatly wrapped and also indented -- I would guess this is done by some (new?) code in recent Anaconda.

This bug was actually about messages which are longer than 80 columns but then get forcibly wrapped at 80 columns by the serial console.

I would suggest you try reproducing this with Fedora 24. I would also suggest investigating whether Anaconda has introduced some recent changes to how it formats and displays its error messages, as mentioned in the related bug 1552401.

Comment 6 Roman Joost 2018-03-08 00:55:06 UTC
Reran a job with Fedora 24 and the installation was killed by EWD again with:

Error 

The following error occurred while installing.  This is a fatal error and instal      

lation will be aborted. 

Failed to download the following packages: Cannot download thttpd-2.27-1.fc27.x8      

6_64.rpm: All mirrors were tried



The trailing white space is the reason as to why the detector hasn't fired. The log shows (trailing whitespace indicated by dots):

The following error occurred while installing.  This is a fatal error and instal······                                                        
                                                                                                                                              
lation will be aborted.·

but going back to the sample console in the patch it seems all trailing white space as been stripped (could it be the editor doing this by default?). Anyway, I think starting off with an unfortunate sample, the implementation did not consider the trailing white space and therefore never tested against this scenario.

The installation failure detector tracks previous lines. When it tries to detect the problem above, the value becomes:

  'The following error occurred while installing.  This is a fatal error and instal      lation will be aborted. '

and we're trying to 'search' with the regular expression:

  'This is a fatal error and installation will be aborted'

which would have been true if it were not for the trailing white space.

Comment 7 Dan Callaghan 2018-03-08 01:00:50 UTC
Indeed, if you go back to the original recipe in comment 1 you can see the console log has the same trailing whitespace in the error message. That must have got trimmed off somewhere along the line when it was turned into a sample.

Comment 8 Roman Joost 2018-03-08 01:02:57 UTC
Going forward I want to propose the following:

Short term tweak the existing pattern matching against:

Press Enter to exit.

because at least we detect the installation failure. I do understand that this is not very helpful for the user.

Next steps I think we should re-factor the code and split it up into:

a) code which cleans up the log files in order to get to a line we can match on
b) code which tries to detect the failure and/or panic (PanicDetector and InstallFailureDetector are pretty much doing the same thing atm)
c) code which produces a useful error message for the user
d) code which stores the block

Perhaps the pieces will need separate backlog items. Something we can discuss in the next backlog meeting.

Comment 9 Roman Joost 2018-03-13 06:24:03 UTC
Filed Bug 1554656 for a possible refactoring. I'll add the short term fix for this ...

Comment 10 Dan Callaghan 2018-03-14 01:15:45 UTC
You probably want to adjust the bug title since this bug has now morphed into something quite different. Or just dupe it onto bug 1552401.

Comment 11 Roman Joost 2018-03-15 23:42:44 UTC
I'll dupe it on to Bug 1552401.

*** This bug has been marked as a duplicate of bug 1552401 ***