Bug 1311886

Summary: gcc6 - thunderbird - miscompiled for cycle when this == NULL
Product: [Fedora] Fedora Reporter: Martin Stransky <stransky>
Component: gccAssignee: Jakub Jelinek <jakub>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: davejohansen, jakub, jwakely, law, mpolacek
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-02-25 09:49:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Martin Stransky 2016-02-25 09:43:31 UTC
Description of problem:

This code is miscompiled when optimization is enabled:

6090	uint8_t
6091	nsBlockFrame::FindTrailingClear()
6092	{
6093	  // find the break type of the last line
6094	  for (nsIFrame* b = this; b; b = b->GetPrevInFlow()) {
6095	    nsBlockFrame* block = static_cast<nsBlockFrame*>(b);
6096	    line_iterator endLine = block->end_lines();
6097	    if (endLine != block->begin_lines()) {
6098	      --endLine;
6099	      return endLine->GetBreakTypeAfter();
6100	    }
6101	  }
6102	  return NS_STYLE_CLEAR_NONE;
6103	}

this == NULL so the code may quit immediately in

6094  for (nsIFrame* b = this; b; b = b->GetPrevInFlow())

but this is ignored and it executes line 6095 instead. Does gcc6 expect this can't be null?

Comment 1 Martin Stransky 2016-02-25 09:46:26 UTC
I see this may be related:

Optimizations remove null pointer checks for this

When optimizing, GCC now assumes the this pointer can never be null, which is guaranteed by the language rules. Invalid programs which assume it is OK to invoke a member function through a null pointer (possibly relying on checks like this != NULL) may crash or otherwise fail at run time if null pointer checks are optimized away. With the -Wnull-dereference option the compiler tries to warn when it detects such invalid code.

If the program cannot be fixed to remove the undefined behavior then the option -fno-delete-null-pointer-checks can be used to disable this optimization. That option also disables other optimizations involving pointers, not only those involving this.

Comment 2 Jonathan Wakely 2016-02-25 09:46:59 UTC
(In reply to Martin Stransky from comment #0)
> Does gcc6 expect this can't be null?

Yes, see https://gcc.gnu.org/gcc-6/porting_to.html

It is undefined behaviour, and nonsensical, to call a member function on an object that doesn't exist.

Comment 3 Marek Polacek 2016-02-25 09:49:15 UTC
Right, so closing.