Bug 1984808 - autofs: FTBFS due to dynamic PTHREAD_STACK_MIN (glibc 2.34 related)
Summary: autofs: FTBFS due to dynamic PTHREAD_STACK_MIN (glibc 2.34 related)
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: autofs
Version: 35
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Ian Kent
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks: 1984813
TreeView+ depends on / blocked
 
Reported: 2021-07-22 09:40 UTC by Florian Weimer
Modified: 2021-10-20 01:40 UTC (History)
1 user (show)

Fixed In Version: autofs-5.1.7-19.el9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1984813 (view as bug list)
Environment:
Last Closed: 2021-10-20 01:40:24 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Florian Weimer 2021-07-22 09:40:12 UTC
Building autofs fails with:

gcc -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_REENTRANT -D_FILE_OFFSET_BITS=64 -I/usr/include/tirpc  -D_REENTRANT -D_FILE_OFFSET_BITS=64 -I/usr/include/tirpc  -rdynamic -fPIE -D_GNU_SOURCE -I../include -DAUTOFS_LIB_DIR=\"/usr/lib64/autofs\"  -DAUTOFS_MAP_DIR=\"/etc\" -DAUTOFS_CONF_DIR=\"/etc/sysconfig\" -DAUTOFS_FIFO_DIR=\"/run\" -DAUTOFS_FLAG_DIR=\"/run\" -DVERSION_STRING=\"5.1.7-18.fc35\" -DWITH_SYSTEMD -I/usr/include/libxml2  -c automount.c
In file included from /usr/include/bits/local_lim.h:81,
                 from /usr/include/bits/posix1_lim.h:161,
                 from /usr/include/dirent.h:233,
                 from automount.c:22:
automount.c:87:37: error: initializer element is not constant
   87 | size_t detached_thread_stack_size = PTHREAD_STACK_MIN * 144;
      |                                     ^~~~~~~~~~~~~~~~~
make[1]: Leaving directory '/builddir/build/BUILD/autofs-5.1.7/daemon'

This use of PTHREAD_STACK_MIN is a bit strange because the computed stack size varies wildly between architectures. And it was done deliberately to *increase* stack size (most uses of PTHREAD_STACK_MIN intend to do the opposite):

commit ef4886702177f1bbeba66858181a7db44ca8ebd6
Author: Ian Kent <raven>
Date:   Fri Apr 21 08:17:52 2017 +0800

    autofs-5.1.2 - increase worker thread per-thread stack size
    
    automount(8) uses a worker thread per-thread stack size of 1M which is
    sufficient for its needs.
    
    But some glibc functions (such as nscd_getgr_r()) use alloca() to allocate
    working storage that can have an arbitary size. Since alloca() never fails
    there is no way to check for stack overflow. This has been fixed in more
    recent versions of glibc.
    
    But for older version of glibc all that can be done by automount to avoid
    this is to increase the stack size.
    
    Signed-off-by: Ian Kent <raven>

Maybe this can be disabled for glibc 2.34 and later. Due to the symbol version changes in glibc 2.34, the compiled binary cannot run on earlier glibc versions, so a compile-time check would be safe in this case.

Comment 1 Ian Kent 2021-07-22 11:19:58 UTC
Quite a bit of that alloca() usage has gone now.
I'll need to have a look at that, there might not bee too much left.

But, for the time being, you recommend simply using the default stack
size, correct?

I'm pretty sure the original change was motivated because I had
problems with stack overflow.

Another option would be to set a actual size there but I don't have
enough info. about what that setting should be just yet. So maybe
eliminating the alloca() usage and just using the default thread
stack size will be the most sensible thing to do. Sadly I suspect
I'll find there's more of it than I think.

Suggestions, thoughts?

Comment 2 Florian Weimer 2021-07-22 11:32:07 UTC
(In reply to Ian Kent from comment #1)
> Quite a bit of that alloca() usage has gone now.
> I'll need to have a look at that, there might not bee too much left.
> 
> But, for the time being, you recommend simply using the default stack
> size, correct?

Yes, I think so. If there are glibc bugs, we should fix them in glibc. (And we have fixed many such bugs over the years, but there could be more.)

> I'm pretty sure the original change was motivated because I had
> problems with stack overflow.

Yes, the commit message suggests that.

> Another option would be to set a actual size there but I don't have
> enough info. about what that setting should be just yet. So maybe
> eliminating the alloca() usage and just using the default thread
> stack size will be the most sensible thing to do. Sadly I suspect
> I'll find there's more of it than I think.

What stack size did you try to set? A typical glibc default is 8 MiB (inherited fro ulimit), but historically, 144 * PTHREAD_STACK_MIN varies from 2.25 MiB to 18 MiB, depending on architecture (even 27 MiB for ia64, but that doesn't really matter).

Comment 3 Ian Kent 2021-07-22 11:44:18 UTC
(In reply to Florian Weimer from comment #2)
> (In reply to Ian Kent from comment #1)
> > Quite a bit of that alloca() usage has gone now.
> > I'll need to have a look at that, there might not bee too much left.
> > 
> > But, for the time being, you recommend simply using the default stack
> > size, correct?
> 
> Yes, I think so. If there are glibc bugs, we should fix them in glibc. (And
> we have fixed many such bugs over the years, but there could be more.)
> 
> > I'm pretty sure the original change was motivated because I had
> > problems with stack overflow.
> 
> Yes, the commit message suggests that.
> 
> > Another option would be to set a actual size there but I don't have
> > enough info. about what that setting should be just yet. So maybe
> > eliminating the alloca() usage and just using the default thread
> > stack size will be the most sensible thing to do. Sadly I suspect
> > I'll find there's more of it than I think.
> 
> What stack size did you try to set? A typical glibc default is 8 MiB
> (inherited fro ulimit), but historically, 144 * PTHREAD_STACK_MIN varies
> from 2.25 MiB to 18 MiB, depending on architecture (even 27 MiB for ia64,
> but that doesn't really matter).

It was a very long time ago so I can't remember the process but I did
think it would be more sensible to use a multiple of a pthreads constant
rather than an arbitrary value.

But, at that time, I also had the impression the resulting stack size
of this calculation was much smaller than the values you mentioned above
so maybe the default is actually enough. I think 8 MiB should be enough,
maybe things have changed since I did that change.

I'll have to change to use the default and run some tests.

Comment 4 Ian Kent 2021-07-28 06:35:57 UTC
So it looks like the glibc scratch buffer changes done around 2015 (probably
available in glibc version 2.24 and later) eliminated the alloca() usage I
had seen in getting groups info.

That's quite a while ago now.

Given this was so long ago and that this was originally a rather pathological
case of groups with a very large number of members I'd rather not add checks
on glibc version in the next autofs release.

The fact is the pthreads default stack size should be fine with distributions
that are even remotely up to date with glibc in new releases and if there are
any problems reported by people using really old glibc versions I'll deal with
those on the autofs mailing list.

Comment 8 Ian Kent 2021-07-28 06:41:57 UTC
Let's have this sit is Rawhide for a little and see how it goes.
It should be fine.

Comment 9 Ben Cotton 2021-08-10 13:14:26 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 35 development cycle.
Changing version to 35.


Note You need to log in before you can comment on or make changes to this bug.