Bug 1463706

Summary: Inconsistency when switching from read to write in std::fstream
Product: Red Hat Enterprise Linux 7 Reporter: Paulo Andrade <pandrade>
Component: gccAssignee: Jonathan Wakely <jwakely>
Status: CLOSED ERRATA QA Contact: Michael Petlan <mpetlan>
Severity: medium Docs Contact: Vladimír Slávik <vslavik>
Priority: high    
Version: 7.2CC: amike, chorn, cww, jakub, jwakely, law, mcermak, mnewsome, mpolacek, mpoole, ohudlick, tschelle, vslavik
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: gcc-4.8.5-31.el7 Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-30 07:27:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1420851, 1471969, 1477664, 1505884, 1546815, 1609081    

Description Paulo Andrade 2017-06-21 14:20:37 UTC
A test program crashes with a stack overflow, for example on
Fedora 25:
#0  0x00007ffff7b3ec70 in std::basic_filebuf<char, std::char_traits<char> >::_M_terminate_output (this=this@entry=0x7fffffffdc38)
    at /usr/src/debug/gcc-6.3.1-20161221/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/fstream.tcc:925
#1  0x00007ffff7b3eccb in std::basic_filebuf<char, std::char_traits<char> >::_M_seek (this=this@entry=0x7fffffffdc38, __off=0, __way=__way@entry=std::_S_cur, __state=...)
    at /usr/src/debug/gcc-6.3.1-20161221/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/fstream.tcc:878
#2  0x00007ffff7b3ee11 in std::basic_filebuf<char, std::char_traits<char> >::overflow (this=0x7fffffffdc38, __c=-1)
    at /usr/src/debug/gcc-6.3.1-20161221/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/fstream.tcc:519
#3  0x00007ffff7b3ec73 in std::basic_filebuf<char, std::char_traits<char> >::_M_terminate_output (this=this@entry=0x7fffffffdc38)
    at /usr/src/debug/gcc-6.3.1-20161221/obj-x86_64-redhat-linux/x86_64-redhat-linux/libstdc++-v3/include/bits/fstream.tcc:925

  The C++ version is :
"""
#include <iostream>
#include <cstring>

int main(int, char**) {
	std::fstream s;
	s.open("test.txt", std::ios_base::in | std::ios_base::out | std::ios_base::binary);

	char data[8 * 1024];
	memset(data, 'A', sizeof(data));
	s.write(data, sizeof(data));

	s.seekg(0);

	char buf[8 * 1024];
	memset(buf, 0, sizeof(buf));
	s.read(buf, sizeof(buf));

	char data2[8 * 1024];
	memset(data2, 'B', sizeof(data2));
	s.write(data2, sizeof(data2));

	return 0;
}
"""
(might need to first create an empty test.txt file)

  But the equivalent C version works as expected:
"""
#include <stdio.h>
#include <string.h>

int main(int argc, char *argv[]) {
	FILE	*s;
	s = fopen("ctest.txt", "w+");
	char data[8 * 1024];
	memset(data, 'A', sizeof(data));
	fwrite(data, sizeof(data), 1, s);

	rewind(s);

	char buf[8 * 1024];
	memset(buf, 0, sizeof(buf));
	fread(buf, sizeof(buf), 1, s);

	char data2[8 * 1024];
	memset(data2, 'B', sizeof(data2));
	fwrite(data2, sizeof(data2), 1, s);
	fclose(s);

	return 0;
}
"""

  The C++ version requires keeping track of the "implicit" offset,
and doing an absolute seek (and calculating it for reads) when
switching from read to write, to avoid the crash.

Comment 2 Christian Horn 2017-06-22 00:55:00 UTC
For me, the C++ code in the description does not compile.
I used this:
"""
#include <fstream>
#include <cstring>
  
int main () {
  std::fstream s;
  s.open ("test.txt", std::ios_base::in | std::ios_base::out | std::ios_base::binary);
  
  char data[8 * 1024];
  memset(data, 'A', sizeof(data));
  s.write(data, sizeof(data));
  
  s.seekg(0);
  
  char buf[8 * 1024];
  memset(buf, 0, sizeof(buf));
  s.read(buf, sizeof(buf));

  // s.seekg(8 * 1024);
  char data2[8 * 1024];
  memset(data2, 'B', sizeof(data2));
  s.write(data2, sizeof(data2));
  
  s.close();
  
  return 0;
}
"""

When removing the comment from the second s.seekg(), then the code does no longer segfault.

Comment 3 Jos Collin 2017-06-22 09:07:05 UTC
An additional information:
Read a character less and it won't segfault.
s.read(buf, sizeof(buf)-1);

As far as I have researched about this, Bidirection file streams require setting the output position indicator for the write after a read to get the desired results.

Comment 13 Jonathan Wakely 2017-07-10 20:52:49 UTC
(In reply to Jos Collin from comment #3)
> As far as I have researched about this, Bidirection file streams require
> setting the output position indicator for the write after a read to get the
> desired results.

That's correct. Mixing reads and writes without intervening seeks or flushes is undefined, which is why the fstream gets into a corrupt state.

I have a prototype patch that prevents the recursion, and should only affect behaviour in undefined cases like this. It won't make the program correct, but it will prevent the infinite recursion that overflows the stack.

Comment 14 Christian Horn 2017-07-11 01:08:14 UTC
(In reply to Jonathan Wakely from comment #13)
> I have a prototype patch that prevents the recursion, and should only affect
> behaviour in undefined cases like this. It won't make the program correct,
> but it will prevent the infinite recursion that overflows the stack.

From a user perspective, I think the most desirable behaviour would be to detect the situation already at compile time and bail out.  Not sure if that detection is possible (or allowed by conventions/standards).

Comment 17 Jonathan Wakely 2017-07-11 08:47:42 UTC
(In reply to Christian Horn from comment #14)
> From a user perspective, I think the most desirable behaviour would be to
> detect the situation already at compile time and bail out.  Not sure if that
> detection is possible (or allowed by conventions/standards).

That's impossible.

Comment 18 Jonathan Wakely 2017-07-19 00:33:10 UTC
Although the C++ standard says that the seeks are required for correct behaviour, changes were to GCC's fstream code several years ago to insert the seeks automatically as needed. Although relying on this isn't necessarily portable to other implementations, GCC tries to make it work. However, the customer's testcase hits an edge case where the automatic seeking isn't done correctly and leads to a crash.

This is now fixed in upstream GCC, in a way that makes the original testcase work as expected: no intervening seek is needed between the read and write.

Comment 29 Jonathan Wakely 2018-06-12 13:22:17 UTC
While we believe we have the right fix for this problem, the affected code is complex and we believe the fix should receive further real world testing by way of upstream GCC, DTS releases and Y stream RHEL releases before we support it as a Z stream update.

Comment 32 Michael Petlan 2018-07-16 16:36:49 UTC
Reproduced with gcc-4.8.5-28.el7_5.1, works OK with gcc-4.8.5-36.el7.
VERIFIED

Comment 34 errata-xmlrpc 2018-10-30 07:27:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3016