Bug 1311886 - gcc6 - thunderbird - miscompiled for cycle when this == NULL
gcc6 - thunderbird - miscompiled for cycle when this == NULL
Product: Fedora
Classification: Fedora
Component: gcc (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Jakub Jelinek
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2016-02-25 04:43 EST by Martin Stransky
Modified: 2016-02-25 05:20 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2016-02-25 04:49:15 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

External Trackers
Tracker ID Priority Status Summary Last Updated
Mozilla Foundation 1167145 None None None 2016-02-25 05:20 EST

  None (edit)
Description Martin Stransky 2016-02-25 04:43:31 EST
Description of problem:

This code is miscompiled when optimization is enabled:

6090	uint8_t
6091	nsBlockFrame::FindTrailingClear()
6092	{
6093	  // find the break type of the last line
6094	  for (nsIFrame* b = this; b; b = b->GetPrevInFlow()) {
6095	    nsBlockFrame* block = static_cast<nsBlockFrame*>(b);
6096	    line_iterator endLine = block->end_lines();
6097	    if (endLine != block->begin_lines()) {
6098	      --endLine;
6099	      return endLine->GetBreakTypeAfter();
6100	    }
6101	  }
6102	  return NS_STYLE_CLEAR_NONE;
6103	}

this == NULL so the code may quit immediately in

6094  for (nsIFrame* b = this; b; b = b->GetPrevInFlow())

but this is ignored and it executes line 6095 instead. Does gcc6 expect this can't be null?
Comment 1 Martin Stransky 2016-02-25 04:46:26 EST
I see this may be related:

Optimizations remove null pointer checks for this

When optimizing, GCC now assumes the this pointer can never be null, which is guaranteed by the language rules. Invalid programs which assume it is OK to invoke a member function through a null pointer (possibly relying on checks like this != NULL) may crash or otherwise fail at run time if null pointer checks are optimized away. With the -Wnull-dereference option the compiler tries to warn when it detects such invalid code.

If the program cannot be fixed to remove the undefined behavior then the option -fno-delete-null-pointer-checks can be used to disable this optimization. That option also disables other optimizations involving pointers, not only those involving this.
Comment 2 Jonathan Wakely 2016-02-25 04:46:59 EST
(In reply to Martin Stransky from comment #0)
> Does gcc6 expect this can't be null?

Yes, see https://gcc.gnu.org/gcc-6/porting_to.html

It is undefined behaviour, and nonsensical, to call a member function on an object that doesn't exist.
Comment 3 Marek Polacek 2016-02-25 04:49:15 EST
Right, so closing.

Note You need to log in before you can comment on or make changes to this bug.