The pam_userlist module has numerous problems. There is no documentation. The module determine's the user's identity by using the getuid() function, which will cause it to not work properly when called from an application like login or ftpd. When parsing arguments, it uses strtok(), which will destroy any context a calling application might have. While this is allowed, it is highly undesirable. When determining the current user's name, it uses getpwuid(), which will destroy any preexisting queries an application might have made. This problem in pam_limits led to strange behavior with OpenSSH's sshd. Like strtok(), this is allowed but undesirable. When checking if the current user is in a group, only the primary group ID is checked. Supplemental group memberships are ignored.
As discussed in email, getuid() and friends have to be used to get the functionality it provides. I've added documentation explaining what it is and why it can't be used for services like login or ftpd. The strtok usage is ok since it operates on a strdup()'ed string, the calling application isn't affected. Support for supplementary groups has been added in 1.0.1-1.
You seem to be confusing strtok() and strtok_r() here. This bug is still present in 1.0.1, and my other misgivings about including it in a release still stand.
pam_userlist checks the current group, and then the user's default group list, which may have changed if the user ran sg(1) or newgrp(1), or if the login session used pam_groups to modify the user's group list. It should use getgroups() to retrieve the current supplemental groups list instead.
Really fixed in 1.0.2-1.