A flaw was found in the way Samba handles the file descriptor sets (fd_set)
datastructure.

The Samba codebase uses file descriptor sets in various places. The fd_set
structure is a fixed size defined by the FD_SETSIZE variable. If a file
descriptor with a value greater than or equal to FD_SETSIZE is added to a
set, it can set a single bit on the stack to a '1'.

In Red Hat Enterprise Linux, all samba processes except for smbd have a
limit set which prevents a process from allocating more than 1024 file
descriptors by default. 1024 is the value of FD_SETSIZE on Red Hat
Enterprise Linux.

smbd does not cap the maximum allowed file descriptors below 1024. This
means that if a remote attacker has the ability to open files on a Samba
server, they may be able to flip arbitrary stack bits to a '1'. It is not
currently believed that this flaw can be used for arbitrary code execution,
but the possibility should not be ruled out.

Acknowledgements:

Red Hat would like to thank the Samba team for reporting this issue. Upstream acknowledges Volker Lendecke of SerNet as the original reporter.