This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 509180 - Openais segfaults at startup when the user ais does not exist.
Openais segfaults at startup when the user ais does not exist.
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: openais (Show other bugs)
5.3
All Linux
low Severity low
: rc
: ---
Assigned To: Jan Friesse
Cluster QE
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-07-01 12:55 EDT by Ben Turner
Modified: 2016-04-26 10:33 EDT (History)
7 users (show)

See Also:
Fixed In Version: openais-0.80.6-11.el5
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-03-30 03:48:27 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Core generated from crash. (11.53 MB, application/octet-stream)
2009-07-01 12:58 EDT, Ben Turner
no flags Details
openais strace (1.30 MB, application/octet-stream)
2009-07-01 12:59 EDT, Ben Turner
no flags Details
cman strace. (4.86 MB, application/octet-stream)
2009-07-01 13:00 EDT, Ben Turner
no flags Details
Proposed patch (993 bytes, patch)
2009-07-07 06:04 EDT, Jan Friesse
no flags Details | Diff

  None (edit)
Description Ben Turner 2009-07-01 12:55:03 EDT
Description of problem: Openais segfaults when starting without the group ais in /etc/group.

Version-Release number of selected component (if applicable): 
cman-2.0.98-1.el5_3.4
openais-0.80.3-22.el5_3.8

How reproducible: This is easily reproducible by removing the ais group from /etc/group


Steps to Reproduce:
1. Remove the group ais from /etc/group
2. restart cman
3.
  
Actual results:
Segfault.

Expected results:
An error message with no segfault.

Additional info:
Comment 1 Ben Turner 2009-07-01 12:58:39 EDT
Created attachment 350150 [details]
Core generated from crash.
Comment 2 Ben Turner 2009-07-01 12:59:22 EDT
Created attachment 350151 [details]
openais strace
Comment 3 Ben Turner 2009-07-01 13:00:03 EDT
Created attachment 350152 [details]
cman strace.
Comment 5 Adrien Kunysz 2009-07-02 03:22:54 EDT
The attached core file is from openais-0.80.3-22.el5_3.8-x86_64.

Core was generated by `aisexec'.
Program terminated with signal 11, Segmentation fault.
[New process 1397]
[New process 1404]
[New process 1398]
#0  main (argc=<value optimized out>, argv=<value optimized out>) at main.c:249
249                     log_printf (LOG_LEVEL_ERROR, "ERROR: The '%s' group is not found in /etc/group, please read the documentation.\n", group->gr_name);

Looking at the relevant function from main.c, the error is obvious:
static void aisexec_gid_determine (struct main_config *main_config)
{
        struct group *group;
        group = getgrnam (main_config->group);
        if (group == 0) {
                log_printf (LOG_LEVEL_ERROR, "ERROR: The '%s' group is not found in /etc/group, please read the documentation.\n", group->gr_name);
                openais_exit_error (AIS_DONE_GID_DETERMINE);
        }
        gid_valid = group->gr_gid;
        endgrent();
}

When the group pointer is NULL (because getgrnam() failed to retrieve the group by name for whatever reason) it will be dereferenced, thus crashing silently the daemon. As far as I can tell, this code is present in all 0.8x versions of openais (released or not).
Comment 6 Steven Dake 2009-07-06 22:48:04 EDT
Not a regression.

Honza, can you please work out a solution to this problem against whitetank branch?

Thanks
-steve
Comment 9 Jan Friesse 2009-07-07 06:04:44 EDT
Created attachment 350765 [details]
Proposed patch

This patch fixes problem by printing main_config->group rather then group->gr_name
Comment 12 Chris Ward 2010-02-11 05:07:38 EST
~~ Attention Customers and Partners - RHEL 5.5 Beta is now available on RHN ~~

RHEL 5.5 Beta has been released! There should be a fix present in this 
release that addresses your request. Please test and report back results 
here, by March 3rd 2010 (2010-03-03) or sooner.

Upon successful verification of this request, post your results and update 
the Verified field in Bugzilla with the appropriate value.

If you encounter any issues while testing, please describe them and set 
this bug into NEED_INFO. If you encounter new defects or have additional 
patch(es) to request for inclusion, please clone this bug per each request
and escalate through your support representative.
Comment 13 Adrien Kunysz 2010-02-13 10:59:09 EST
I confirm the program doesn't crash anymore but as far as I can tell it doesn't generate any log either. This makes the problem even harder to diagnose.

# cat /etc/ais/openais.conf 
# Please read the openais.conf.5 manual page

totem {
        version: 2
        secauth: off
        threads: 0
        interface {
                ringnumber: 0
                bindnetaddr: 192.168.2.0
                mcastaddr: 226.94.1.1
                mcastport: 5405
        }
}

logging {
        to_stderr: on
        debug: on
        timestamp: on
}

amf {
        mode: disabled
}
# grep ais /etc/group
openais:x:39:
# /usr/sbin/aisexec
#

We see the _log_printf() is called correctly but there is never leaves the process:
# stap -c /usr/sbin/aisexec -e 'probe process("/usr/sbin/aisexec").function("_log_printf") { printf("logging: %s", user_string($format)) } probe syscall.write { if (execname() == "aisexec") {printf("printing: %s\n", argstr) }}'
logging: AIS Executive Service RELEASE '%s'
logging: Copyright (C) 2002-2006 MontaVista Software, Inc and contributors.
logging: Copyright (C) 2006 Red Hat, Inc.
logging: AIS Executive Service: started and ready to provide service.
logging: AIS Executive Service RELEASE 'subrev 1887 version 0.80.6'
logging: Copyright (C) 2002-2006 MontaVista Software, Inc and contributors.
logging: Copyright (C) 2006 Red Hat, Inc.
logging: AIS Executive Service: started and ready to provide service.
logging: log setup
logging: ERROR: The '%s' group is not found in /etc/group, please read the documentation.
logging: AIS Executive exiting (reason: %s).

There is just nothing logged on stderr or in the logs. I suspect the program exits before the logger thread has been able to handle the messages.
Comment 15 Jan Friesse 2010-02-15 04:43:22 EST
Andrien,
with such config file, it is nothing strange, that it logs nothing.

Basically:
- running aisexec -f gives you output
- if you are running aisexec as daemon, configure to_file or to_syslog to get logs into file or into syslog.

Regards,
  Honza

(In reply to comment #13)
> I confirm the program doesn't crash anymore but as far as I can tell it doesn't
> generate any log either. This makes the problem even harder to diagnose.
> 
> # cat /etc/ais/openais.conf 
> # Please read the openais.conf.5 manual page
> 
> totem {
>         version: 2
>         secauth: off
>         threads: 0
>         interface {
>                 ringnumber: 0
>                 bindnetaddr: 192.168.2.0
>                 mcastaddr: 226.94.1.1
>                 mcastport: 5405
>         }
> }
> 
> logging {
>         to_stderr: on
>         debug: on
>         timestamp: on
> }
> 
> amf {
>         mode: disabled
> }
> # grep ais /etc/group
> openais:x:39:
> # /usr/sbin/aisexec
> #
> 
> We see the _log_printf() is called correctly but there is never leaves the
> process:
> # stap -c /usr/sbin/aisexec -e 'probe
> process("/usr/sbin/aisexec").function("_log_printf") { printf("logging: %s",
> user_string($format)) } probe syscall.write { if (execname() == "aisexec")
> {printf("printing: %s\n", argstr) }}'
> logging: AIS Executive Service RELEASE '%s'
> logging: Copyright (C) 2002-2006 MontaVista Software, Inc and contributors.
> logging: Copyright (C) 2006 Red Hat, Inc.
> logging: AIS Executive Service: started and ready to provide service.
> logging: AIS Executive Service RELEASE 'subrev 1887 version 0.80.6'
> logging: Copyright (C) 2002-2006 MontaVista Software, Inc and contributors.
> logging: Copyright (C) 2006 Red Hat, Inc.
> logging: AIS Executive Service: started and ready to provide service.
> logging: log setup
> logging: ERROR: The '%s' group is not found in /etc/group, please read the
> documentation.
> logging: AIS Executive exiting (reason: %s).
> 
> There is just nothing logged on stderr or in the logs. I suspect the program
> exits before the logger thread has been able to handle the messages.
Comment 19 Adrien Kunysz 2010-02-16 04:22:45 EST
(In reply to comment #15)
> Andrien,
> with such config file, it is nothing strange, that it logs nothing.
> 
> Basically:
> - running aisexec -f gives you output
> - if you are running aisexec as daemon, configure to_file or to_syslog to get
> logs into file or into syslog.

Running aisexec -f doesn't give me any output on stderr or in the logs. Even when setting to_stderr, debug and to_syslog. I also tried with only to_stderr and debug (no to_syslog).

# cat /etc/ais/openais.conf 
# Please read the openais.conf.5 manual page

totem {
        version: 2
        secauth: off
        threads: 0
        interface {
                ringnumber: 0
                bindnetaddr: 192.168.2.0
                mcastaddr: 226.94.1.1
                mcastport: 5405
        }
}

logging {
        to_stderr: on
        to_syslog: on
        debug: on
        timestamp: on
}

amf {
        mode: disabled
}
# /usr/sbin/aisexec -f
#

So, I insist it looks like something is wrong with the logging that will make the original issue (no "ais" group) harder to diagnose.
Comment 20 Adrien Kunysz 2010-02-16 05:22:47 EST
OK, I found the problem with logging:

to_file, to_syslog and to_stderr are yes/no options while debug and timestamp are on/off. The following configuration file works better:
# cat /etc/ais/openais.conf 
# Please read the openais.conf.5 manual page

totem {
        version: 2
        secauth: off
        threads: 0
        interface {
                ringnumber: 0
                bindnetaddr: 192.168.2.0
                mcastaddr: 226.94.1.1
                mcastport: 5405
        }
}

logging {
        to_stderr: yes
        to_syslog: yes
        debug: on
        timestamp: on
}

amf {
        mode: disabled
}
# /usr/sbin/aisexec -f
Feb 16  5:20:37.326249 [MAIN ] AIS Executive Service RELEASE 'subrev 1887 version 0.80.6'
Feb 16  5:20:37.326639 [MAIN ] Copyright (C) 2002-2006 MontaVista Software, Inc and contributors.
Feb 16  5:20:37.326782 [MAIN ] Copyright (C) 2006 Red Hat, Inc.
Feb 16  5:20:37.326864 [MAIN ] AIS Executive Service: started and ready to provide service.
Feb 16  5:20:37.326929 [print.c:0361] log setup
Feb 16  5:20:37.327580 [MAIN ] ERROR: The 'ais' group is not found in /etc/group, please read the documentation.
Feb 16  5:20:37.327609 [MAIN ] AIS Executive exiting (reason: could not determine the process GID).

As far as I can tell, this information cannot be found in the manual. Can you please improve the openais.conf(5) manual or make openais_main_config_read() accept both yes/no and on/off? Let me know if you want me to create a separate Bugzilla instead.
Comment 21 Steven Dake 2010-02-16 09:09:41 EST
The RFE you request is a separate issue and unrelated to this bugzilla issue.  Please file a separate RFE and we will address in the RHEL5.6 timeframe.  FWIW upstream we have already made this change regarding yes/no/on/off equivalent.

Regards
-ste e
Comment 23 errata-xmlrpc 2010-03-30 03:48:27 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2010-0180.html

Note You need to log in before you can comment on or make changes to this bug.