RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 907204 - SIGSEGV in tool_mm_flush()
Summary: SIGSEGV in tool_mm_flush()
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: sblim-sfcb
Version: 6.4
Hardware: i686
OS: Linux
high
high
Target Milestone: rc
: 6.0
Assignee: Vitezslav Crhonek
QA Contact: qe-baseos-daemons
Filip Hanzelka
URL:
Whiteboard:
Depends On:
Blocks: 947775 1070830 1159820 1269194 1359264
TreeView+ depends on / blocked
 
Reported: 2013-02-03 18:30 UTC by Vaughan Cao
Modified: 2019-12-16 04:25 UTC (History)
6 users (show)

Fixed In Version: sblim-sfcb-1.3.11-5.el6
Doc Type: Release Note
Doc Text:
The SFCB server now verifies that a WBEM port is available before attempting to use it The Small Footprint Common Information Model (CIM) Broker (SFCB) uses the same Web-based Enterprise Management (WBEM) port as the OpenPegasus CIM server and other programs. Previously, if the SFCB server was started when the WBEM port was already used by another program, the SFCB server terminated unexpectedly with a segmentation fault. With this update, the SFCB server verifies upon the start whether the desired WBEM port is free to use and proceeds only if the port is free. If the WBEM port is not free, the SFCB server finishes gracefully and logs the reason for its termination.
Clone Of:
Environment:
Last Closed: 2017-03-21 10:05:39 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
disable signals during startup (3.97 KB, patch)
2013-02-03 19:10 UTC, Vaughan Cao 		no flags Details | Diff
synchronize provider state (5.42 KB, patch)
2013-02-03 19:45 UTC, Vaughan Cao 		no flags Details | Diff
updated patch (6.97 KB, patch)
2016-03-03 14:31 UTC, Vitezslav Crhonek 		no flags Details | Diff
proposed patch (15.91 KB, patch)
2016-03-15 09:51 UTC, Vitezslav Crhonek 		no flags Details | Diff
Show Obsolete (1) View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:0648 0 normal SHIPPED_LIVE sblim-sfcb bug fix update 2017-03-21 12:32:26 UTC

Description Vaughan Cao 2013-02-03 18:30:11 UTC 
Description of problem:
start sblim-sfcb service while either httpPort(default 5988) or httpsPort(default 5989) were held by others. segfault will occur on sfcb.

Version-Release number of selected component (if applicable):
1.3.11-2

How reproducible:
Almost always

Steps to Reproduce:
1. service tog-pegasus start, let cimservermain hold port 5989.
2. sblim-sfcb default use sslMode and will try to bind 5989.
   service /etc/init.d/sblim-sfcb start. segfault occur in syslog.
  
Actual results:
log is like below:
Jan  8 21:38:47 ca-ostest361 sfcb[22014]: --- Cannot listen on port 5989 (Address already in use)
Jan  8 21:38:48 ca-ostest361 sfcb[22014]: -#- @�5#010ropProvider - 22031 provider exiting due to a SIGSEGV signal
Jan  8 21:38:48 ca-ostest361 kernel: sfcbd[22031]: segfault at 26828fbc ip 49ad67b5 sp bfaf6b70 error 4 in libsfcBrokerCore.so.0.0.0[49abd000+5c000]
Jan  8 21:38:48 ca-ostest361 kernel: sfcbd[22034]: segfault at 2d3447e7 ip 49ad67b5 sp bfaf6b70 error 4 in libsfcBrokerCore.so.0.0.0[49abd000+5c000]
Jan  8 21:38:48 ca-ostest361 sfcb[22014]: -#- ProfileProvider - 22034 provider exiting due to a SIGSEGV signal
Jan  8 21:38:49 ca-ostest361 abrt[22035]: Saved core dump of pid 22031 (/usr/sbin/sfcbd) to /var/spool/abrt/ccpp-2013-01-08-21:38:48-22031 (17854464 bytes)

Expected results:
sfcbd quit normally without segfault because port 5989 cannot be binded to.

Additional info:
Detected on both i686 and x86_64 at least.
It's a known bug in upstream. refs is http://sourceforge.net/tracker/index.php?func=detail&aid=3416054&group_id=128809&atid=712784.
Comment 2 Vaughan Cao 2013-02-03 19:10:20 UTC 
Created attachment 692459 [details]
disable signals during startup

(This is cherry-picked from upstream commit 3429a6d00ca9ec7e3c138ddbfcfb767b70bd4301)

If an interrupt occurs at startup, depending on the state of processes,
segfault can occur. The patch masks signals when processes are starting up
and enables it back again after that.
Comment 3 Vaughan Cao 2013-02-03 19:11:40 UTC 
Actually, http://sourceforge.net/tracker/index.php?func=detail&aid=3416054&group_id=128809&atid=712784 provides a patch for easy debug that forces an immediate provider exit and a core dump by calling abort() in handlerSigSegv() in providerDrv.c. Because When a provider causes a SIGSEGV, the provider is not immediately terminated. This can cause *many* repeating messages and not always ends in a core file for debugging.

Upstream bug report is in http://sourceforge.net/tracker/?func=detail&aid=3416060&group_id=128809&atid=712784. I clarify it a bit more here.
It's an race condition between a early pthread_key_delete(__mm_key) in uninitGarbageCollector() and pthread_getspecific(__mm_key) used by provider message handler functions. During initializing, sfcb fork a child to start httpdaemon and startup sort of providers, such as ClassProvider and ProfileProvider. If Http(s)Port cannot be bind, SIGQUIT will be sent to parent. CMPI_BrokerExt_Ftab->destroyThreadKey(__mm_key) will run by sighandler while children would use CMPI_BrokerExt_Ftab->getThreadSpecific(__mm_key) to check if provider is started normally. And an early destroying __mm_key leads segfault occur.

I have cherry-pick that patch from git commit 3429a6d00ca9ec7e3c138ddbfcfb767b70bd4301. This commit has been contained in sfcb-1.4.3. The patch just masks signals when processes are starting up
and enables it back again after that.
Comment 4 Vaughan Cao 2013-02-03 19:40:00 UTC 
If you fixed that segfault, A more serious bug would be easily detected. It's a race condition during service stop which is reported in upstream as "double free during stopProc - ID: 3497096" (http://sourceforge.net/tracker/?func=detail&aid=3497096&group_id=128809&atid=712784).

It's because uninit_sfcBroker() cannot invoked from so many places. I've cherry-picked the patch from upstream commit 4699356c2c7973994aa0ad280d1a848c984d9a0f. It's contained in sfcb-1.4.3. This patch ensures only stopBroker() is responsible for stoping the whole services and it can only be called once.
Comment 5 Vaughan Cao 2013-02-03 19:45:39 UTC 
Created attachment 692499 [details]
synchronize provider state

It's cherry-picked from upstream commit 4699356c2c7973994aa0ad280d1a848c984d9a0f.

Synchronize provider state, fix double free during stopProc().
Comment 6 Vaughan Cao 2013-02-03 20:01:10 UTC 
I've tested these two patches on both i686 and x86_64 El6.4 beta. It works.
Comment 7 RHEL Program Management 2013-10-13 23:59:32 UTC 
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated
in the current release, Red Hat is unable to address this
request at this time.

Red Hat invites you to ask your support representative to
propose this request, if appropriate, in the next release of
Red Hat Enterprise Linux.
Comment 8 Vaughan Cao 2013-10-15 06:24:10 UTC 
Hi, will it be addressed in 6.5?
Comment 10 shiwang 2016-03-01 04:22:51 UTC 
Hi team,

Will this bug be fixed in RHEL6.8?

Best regards,
Shiyu Wang
Technical Account Manager
Comment 11 Vitezslav Crhonek 2016-03-03 14:31:29 UTC 
Created attachment 1132811 [details]
updated patch

Hi,

I revised the original patches sent by the reporter again and updated them with latter upstream changes, see:

https://sourceforge.net/p/sblim/sfcb-tix/94/
https://sourceforge.net/p/sblim/sfcb-tix/95/

Resulting patch attached.

Still, I'm not very satisfied with it... It fixes the SIGSEGV indeed, but doesn't work as I would expected (and as latest upstream version works) all the time.

Sometimes it works fine (well... with a little disturbing message "-#- Serious provider id / provider process mismatch --- terminating process.") and sometimes, unfortunately much more often, sfcbd doesn't kill all it's processes at shutdown. Two processes are left. The first one hangs no wait in stopLogging (and I don't think that is a real problem - it should finish after the second one finishes too automatically). The second one hangs in spGetMsg at msgqueue.c:243. This seems to be somehow related to another upstream bug:

https://sourceforge.net/p/sblim/sfcb-tix/97/

Currently, I'm trying to backport the changes from that bug and test them.
Comment 12 Vitezslav Crhonek 2016-03-15 09:51:48 UTC 
Created attachment 1136496 [details]
proposed patch

Final patch. sfcb terminates correctly, reaps all it's processes.
Comment 13 Vitezslav Crhonek 2016-03-15 09:58:08 UTC 
Note that SELinux policy needs to be adjusted accordingly. Although it works fine running sfcb directly from terminal ('sfcbd' or 'sfcbd -d'), running it through 'service sblim-sfcb start' leads to errors with SELinux enforcing:

Mar 15 09:24:38 ibm-ls22-01 sfcb[24767]:
Mar 15 09:24:38 ibm-ls22-01 sfcb[24767]: -- Http Proc semaphore create key: 0x48036b34 failed: File exists
Mar 15 09:24:38 ibm-ls22-01 sfcb[24767]:      use "ipcrm -S 0x48036b34" to remove semaphore
Mar 15 09:24:38 ibm-ls22-01 sfcb[24767]: 
Mar 15 09:24:40 ibm-ls22-01 sfcb[24767]: -#- InternalProvider - 24783 provider exiting due to a SIGSEGV signal
Mar 15 09:24:40 ibm-ls22-01 kernel: sfcbd[24783] general protection ip:7f13392807b9 sp:7fff9d37d950 error:0 in libsfcBrokerCore.so.0.0.0[7f1339263000+59000]

Generated AVCs in SELinux permissive:
type=AVC msg=audit(1458031161.288:760): avc:  denied  { unix_read unix_write } for  pid=24831 comm="sfcbd" key=1208183604  scontext=unconfined_u:system_r:sblim_sfcbd_t:s0 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=sem
type=AVC msg=audit(1458031161.288:760): avc:  denied  { associate } for  pid=24831 comm="sfcbd" key=1208183604  scontext=unconfined_u:system_r:sblim_sfcbd_t:s0 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=sem
type=SYSCALL msg=audit(1458031161.288:760): arch=c000003e syscall=64 success=yes exit=98307 a0=48036b34 a1=1 a2=180 a3=7ffc9aa39550 items=0 ppid=24827 pid=24831 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=112 comm="sfcbd" exe="/usr/sbin/sfcbd" subj=unconfined_u:system_r:sblim_sfcbd_t:s0 key=(null)
type=AVC msg=audit(1458031161.289:761): avc:  denied  { destroy } for  pid=24831 comm="sfcbd" key=1208183604  scontext=unconfined_u:system_r:sblim_sfcbd_t:s0 tcontext=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 tclass=sem
type=SYSCALL msg=audit(1458031161.289:761): arch=c000003e syscall=66 success=yes exit=0 a0=18003 a1=0 a2=0 a3=0 items=0 ppid=24827 pid=24831 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=112 comm="sfcbd" exe="/usr/sbin/sfcbd" subj=unconfined_u:system_r:sblim_sfcbd_t:s0 key=(null)
type=AVC msg=audit(1458031161.288:762): avc:  denied  { unlink } for  pid=24830 comm="sfcbd" name="sfcbLocalSocket" dev=sda3 ino=4990472 scontext=unconfined_u:system_r:sblim_sfcbd_t:s0 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=sock_file
type=SYSCALL msg=audit(1458031161.288:762): arch=c000003e syscall=87 success=yes exit=0 a0=7f5be3015ba1 a1=7f5be3015bb8 a2=7f5be2da6c86 a3=7f5be2da69f0 items=0 ppid=1 pid=24830 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=112 comm="sfcbd" exe="/usr/sbin/sfcbd" subj=unconfined_u:system_r:sblim_sfcbd_t:s0 key=(null)
Comment 20 Alois Mahdal 2017-01-15 18:37:12 UTC 
Verified on all architectures with sblim-sfcb-1.3.11-5.el6.  sfcbd does not segfault anymore.
Comment 22 errata-xmlrpc 2017-03-21 10:05:39 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2017-0648.html
Note You need to log in before you can comment on or make changes to this bug.