For comparison, on FreeBSD 12.1:

$ make glibcbug
cc -O2 -pipe  glibcbug.c  -o glibcbug
$ ./glibcbug 
Running test: fopen()
position after fwrite: 24
position after SEEK_END: 24

Running test: fmemopen()
position after fwrite: 24
position after SEEK_END: 24

Running test: open_memstream()
position after fwrite: 24
position after SEEK_END: 24

open_memstream() final buffer size: 24

Created attachment 1713683 [details]
test-open_memstream.c

Minor edit to http://web.mit.edu/freebsd/head/tools/regression/lib/libc/stdio/test-open_memstream.c to compile cleanly with glibc.

It appears it would fail on open_group_test() for this specific
report, but fails before for yet another test...

This appears to be the way that open_memstream is designed.  The internal structure only keeps track of a read pointer and a write pointer, so the call to seek to the beginning and write four bytes resets the "end of file" to be at the four byte mark.

Note that fmemopen uses a completely different internal implementation, and has separate variables for current and maximum position.

(In reply to DJ Delorie from comment #5)

My understanding is that the behavior in RHEL-8 matches the documentation
but the results are not consistent between rhel-7, rhel-8 and FreeBSD:

* RHEL-7

$ cat /etc/redhat-release 
Red Hat Enterprise Linux Server release 7.8 (Maipo)
$ gcc -o memstream-test-rhel-7 memstream-test.c
$ ./memstream-test-rhel-7
Running test: fopen()
position after fwrite: 24
position after SEEK_END: 24

Running test: fmemopen()
position after fwrite: 24
position after SEEK_END: 64

Running test: open_memstream()
position after fwrite: 24
position after SEEK_END: 48

open_memstream() final buffer size: 48

* RHEL-8

$ cat /etc/redhat-release 
Red Hat Enterprise Linux release 8.2 (Ootpa)
$ gcc -o memstream-test-rhel-8 memstream-test.c
$ ./memstream-test-rhel-8
Running test: fopen()
position after fwrite: 24
position after SEEK_END: 24

Running test: fmemopen()
position after fwrite: 24
position after SEEK_END: 24

Running test: open_memstream()
position after fwrite: 24
position after SEEK_END: 4

open_memstream() final buffer size: 4

* FreeBSD 12.1

$ uname -a
FreeBSD freebsd-12-1.example.com 12.1-RELEASE FreeBSD 12.1-RELEASE r354233 GENERIC  amd64
$ cc -o memstream-test-fbsd-12 memstream-test.c
$ ./memstream-test-fbsd-12
Running test: fopen()
position after fwrite: 24
position after SEEK_END: 24

Running test: fmemopen()
position after fwrite: 24
position after SEEK_END: 24

Running test: open_memstream()
position after fwrite: 24
position after SEEK_END: 24

open_memstream() final buffer size: 24

DJ and I reviewed this bug.

It's a problem with the standard. The specification of SEEK_END is underspecified.

We aren't going to make any changes in Red Hat Enterprise Linux 7.

In Red Hat Enterprise Linux 8 we can make some adjustments based on the outcome of an upstream conversation to harmonize the behaviour.

I reached out to Rich Felker (musl) and discussed the specification. He agrees it is underspecified.

I passed the recent updates along to our developers and they aren't too pleased about RHEL 7 not being targeted for a fix as we still use RHEL7 internally a lot in addition to RHEL8.  They also don't understand why folks believe that the specification of SEEK_END is underspecified for fmemopen(). The problem from an ISV and developer is that the interpretation of SEEK_END is already defined for file-based streams, and the behavior for memory-based streams is not consistent with it. Consistency is our issue. If there is something in the spec of fmemopen() or open_memstream() that implies the resulting stream should behave differently from fopen(), then that could be an issue. But as far as I'm aware, the resulting memory-based stream should have the same behavior as a file-based stream, only it's in memory.

(In reply to bob.huemmer from comment #9)
> I passed the recent updates along to our developers and they aren't too
> pleased about RHEL 7 not being targeted for a fix as we still use RHEL7
> internally a lot in addition to RHEL8.  They also don't understand why folks
> believe that the specification of SEEK_END is underspecified for fmemopen().
> The problem from an ISV and developer is that the interpretation of SEEK_END
> is already defined for file-based streams, and the behavior for memory-based
> streams is not consistent with it. Consistency is our issue. If there is
> something in the spec of fmemopen() or open_memstream() that implies the
> resulting stream should behave differently from fopen(), then that could be
> an issue. But as far as I'm aware, the resulting memory-based stream should
> have the same behavior as a file-based stream, only it's in memory.

Red Hat Enterprise Linux 7 is in Maintenance Support 2 and as such only urgent priority bug fixes will be considered. Existing customers on RHEL 7 are likely to have applications that may be depenent on the *existing* open_memstream() behaviour in terms of reported buffer sizes. If we change the behaviour of the API to make it consistent with RHEL 8 we risk breaking those existing applications. Conservatively we aim to keep RHEL 7 applications working as this is one of the important requirements from an enterprise operating system point of view.

Customers looking to release and test a single binary application across RHEL 7 and RHEL 8 deployments will see these differences and will need to account for them. This is always going to be the case as older products enter later life-cycle phases.

Please feel free to point your developers here: https://www.openwall.com/lists/libc-coord/2020/09/24/1, if they wish to engage in the libc coordination list discussion with various authors of C libraries to weigh in on the specification of SEEK_END. We would be more than happy to have input from application developers regarding the direction the libraries should take.

Thank you for your input.

Feed back from our developers regarding the standard:

"If a writable FILE* stream returned by open_memstream() behaves any differently from one returned by fopen(), then open_memstream() is useless to me, because my basic expectation of being able to pass either one to code that has been written/tested against the latter is broken. The application that uncovered this issue in the first place was a WAV audio file writer. In cases where the audio data size is not initially known, you first have to write out the data, then seek to the beginning, and write the data size into the file header. I see no reason why the standard should be elaborated in a way that this simple use case works with one type of stream, but not the other."

> Feed back from our developers

Thanks for the feedback, and normally I'd agree with you, but the spec specifically states that a seek/write changes the value of "buffer size" returned after fflush/fclose (value pointed to by sizep), and that an fseek wrt SEEK_SET is shown resetting the buffer size to the "end".  That confuses the issue enough to warrant asking them to clarify.  Once they clarify, all the implementations can decide what (if any) fixes are required to meet the spec.  And since three projects came to three different conclusions about "what makes sense", I'd say it's not obvious, and perhaps there's some unsaid requirement or expectation that we don't know about.

Also, the problem case is open_memstream, not fmemopen.  They are specified differently upstream, which also adds to the confusion.

Reported to the Austin Group...

https://www.austingroupbugs.net/view.php?id=1406

Additional feedback from SAS Development regarding this issue and the latest response from Red Hat:

"I think this issue needs to be approached as an oversight/error in the specification, then. I have tested open_memstream() on multiple systems aside from GNU/Linux, and in every instance where it is available, the expected file-like behavior is observed: FreeBSD 11.1, MacOS X 10.13 (i.e. mainstream FreeBSD), NetBSD 8.1, even AIX 7.1. It is clear that in-the-wild implementations have taken a pragmatic approach, rather than follow what appears to be the (ill-advised) letter of the spec. If the GNU/Linux interpretation is upheld, I foresee that new standards work would be needed in this area to provide for the behavior that everyone assumed was the correct one."

This bug needs to get fixed upstream so that the OSs and distributions implement a fix that is consistent across all of the software ecosystems. This will likely require the Austin Group to comment on the expected behaviour.

For now we are marking this CLOSED/UPSTREAM and will use the following bug to track upstream progress:
https://sourceware.org/bugzilla/show_bug.cgi?id=26557

For now RHEL8 will continue to be have is it does until we have further standards resolution on the issue.

Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.