Created attachment 1398709 [details] reproducer Description of problem: When postgresql is compiled with the -O3 option on ppc64 using gcc version 8 it quickly runs into out-of-memory issues due to postgresql trying to allocate more memory for its cache even though the cache is not yet fully used. It seems that when using -O3 on ppc64 postgresql does not even look at the important part of the cache to see if it is indeed full and just reallocates the cache outright. I have tried reducing the issue to a simple reproducer (attached). Version-Release number of selected component (if applicable): gcc-8.0.1-0.14.fc28.ppc64 How reproducible: Always Steps to Reproduce: 1. gcc -O3 oom-reproducer.c -o oom-reproducer 2. run oom-reproducer Actual results: Doubles the cache size with each call, runs out of memory quickly. Expected results: Doubles the cache size only when all of the cache is used. Additional info: If postgresql is compiled with gcc version 7 it works correctly. Also works correctly when compiling with -O2 instead. The reproducer is able to reproduce the same issue even on different architectures (x86_64) and gcc versions (7.3) when compiled using -O3. This might be because the reproducer is just smallest set of necessary code pulled from postgresql and run inside a simple never-ending loop. It does not reproduce the issue when compiled with -O2, however.
The MemSet macro is clearly invalid, violates aliasing; for GCC changing - long *_start = (long *) _vstart; \ + long __attribute__((__may_alias__)) *_start = (long *) _vstart; \ should cure that though. Wonder why you just don't use memset instead though, gcc does inline memset with constant and reasonably small sizes. That said, that doesn't help in this case, so looking into it.
(In reply to Jakub Jelinek from comment #1) > Wonder why you just don't use memset instead The MemSet macro is circa 20 years old, far predating compilers that would optimize memset() calls by themselves (or that would have aliasing concerns). There have been a few discussions about changing it to plain memset(), but inertia has won out so far. PG uses -fno-strict-aliasing because there's been no vetting of the code base for aliasing problems, so it's unsurprising that we've seen no concrete problems from the existing coding of MemSet.
Started with http://gcc.gnu.org/r241789, work-around would be -O3 -fno-store-merging or just using memset unconditionally. I'll try to reduce it further and debug.
Should be fixed in gcc-8.0.1-0.16.fc28.