Bug 1494182 - Segfault on race conditions. Fixed in imapproxy upstream
Summary: Segfault on race conditions. Fixed in imapproxy upstream
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora EPEL
Classification: Fedora
Component: up-imapproxy
Version: epel7
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Chris Adams
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-09-21 15:32 UTC by doldev
Modified: 2018-02-23 14:10 UTC (History)
4 users (show)

Fixed In Version: up-imapproxy-1.2.8-0.12.20171022svn14722.fc27 up-imapproxy-1.2.8-0.12.20171022svn14722.el7 up-imapproxy-1.2.8-0.12.20171022svn14722.fc26 up-imapproxy-1.2.8-0.12.20171022svn14722.fc25 up-imapproxy-1.2.8-0.12.20171022svn14722.el6
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-11-27 21:23:32 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description doldev 2017-09-21 15:32:02 UTC
Description of problem:

Imapproxy crashes on randomly when performing a connection cleanup.

Log:
-------
Sep 21 08:32:37 somehost in.imapproxyd[4068]: Expiring server sd [269]
Sep 21 08:32:37 somehost in.imapproxyd[4068]: Expiring server sd [-1]
Sep 21 08:32:37 somehost systemd[1]: imapproxy.service: main process exited, code=killed, status=11/SEGV
-------

This happens due to a race condition of multiple threads closing the SSL socket and Imapproxy tries to perform a SSL_write on a non existing socket.

gdb /tmp/core-in.imapproxyd.24766
------
(gdb) bt
#0  0x00007f4ad8bbee55 in SSL_write () from /lib64/libssl.so.10
#1  0x00007f4ad9011ff1 in IMAP_Write (ICD=<optimized out>, buf=<optimized out>, count=<optimized out>) at src/imapcommon.c:1371
#2  0x00007f4ad9011741 in _ICC_Recycle (Expiration=<optimized out>) at src/icc.c:139
#3  0x00007f4ad9011812 in ICC_Recycle_Loop () at src/icc.c:231
#4  0x00007f4ad8159dc5 in start_thread () from /lib64/libpthread.so.0
#5  0x00007f4ad7e8876d in clone () from /lib64/libc.so.6
------

This bug was fixed in the commit https://sourceforge.net/p/squirrelmail/code/14569 

The systemd unit file is also missing a `Restart=on-failure`. In our case the imapserver crashed and was not starting up due to a failure.

Comment 1 Chris Adams 2017-09-25 00:48:24 UTC
Thanks for finding this and tracking down the commit with the fix.

Right now, imapproxy won't build on current Fedora because of SSL issues; I plan to update to a new upstream snapshot (since there hasn't been an actual release in a long long time), and then go through and fix the SSL issues and make a new Fedora release.  Then we can push that to EPEL as well.

It's going to take me a little bit of time to set up a good test environment (I haven't been using the SSL support myself), but I hope to get this done before too long.

Comment 2 Chris Adams 2017-10-23 01:23:10 UTC
I believe I've addressed this (as well as the FTBFS against newer OpenSSL); I've pushed an update to Rawhide first.  I'm going to do a little more testing, then I'll submit it to updates-testing for Fedora 25/26/27 and EPEL 6/7.

Comment 3 Fedora Update System 2017-10-24 00:51:01 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-0e27e6c0ae

Comment 4 Fedora Update System 2017-10-24 00:51:37 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-4281b01423

Comment 5 Fedora Update System 2017-10-24 00:52:18 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-333e0f378f

Comment 6 Fedora Update System 2017-10-24 00:53:00 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-b4d51971d3

Comment 7 Fedora Update System 2017-10-24 00:53:40 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.el6 has been submitted as an update to Fedora EPEL 6. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-4380077af0

Comment 8 doldev 2017-10-24 07:30:09 UTC
  ㋡㋡㋡㋡㋡ ㋡   ㋡   ㋡   ㋡   ㋡ ㋡  ㋡
   ㋡   ㋡   ㋡  ㋡ ㋡  ㋡㋡  ㋡ ㋡ ㋡ 
   ㋡   ㋡㋡㋡㋡㋡ ㋡   ㋡ ㋡ ㋡ ㋡ ㋡㋡  
   ㋡   ㋡   ㋡ ㋡㋡㋡㋡㋡ ㋡  ㋡㋡ ㋡ ㋡ 
   ㋡   ㋡   ㋡ ㋡   ㋡ ㋡   ㋡ ㋡  ㋡

 ㋡    ㋡  ㋡㋡㋡  ㋡   ㋡
  ㋡  ㋡  ㋡   ㋡ ㋡   ㋡
   ㋡㋡   ㋡   ㋡ ㋡   ㋡
   ㋡    ㋡   ㋡ ㋡   ㋡
  ㋡      ㋡㋡㋡   ㋡㋡㋡ 

Will test it soon.

Comment 9 Fedora Update System 2017-10-25 15:23:54 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-0e27e6c0ae

Comment 10 Fedora Update System 2017-10-25 15:59:40 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-b4d51971d3

Comment 11 Fedora Update System 2017-10-26 01:10:36 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-333e0f378f

Comment 12 Fedora Update System 2017-10-26 01:31:55 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-4281b01423

Comment 13 Fedora Update System 2017-10-26 10:05:06 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.el6 has been pushed to the Fedora EPEL 6 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2017-4380077af0

Comment 14 Chris Adams 2017-11-11 18:35:44 UTC
Just checking in - anybody have a chance to test this?  It works for me, but I'd like a confirmation before I push to stable (at least make sure I didn't make any dumb packaging mistakes this time :) ).

Comment 15 Fedora Update System 2017-11-27 21:23:32 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.

Comment 16 Fedora Update System 2017-11-27 22:22:30 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.

Comment 17 Fedora Update System 2017-11-28 00:48:01 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.

Comment 18 Fedora Update System 2017-11-28 04:10:51 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.

Comment 19 Fedora Update System 2017-11-28 05:22:37 UTC
up-imapproxy-1.2.8-0.12.20171022svn14722.el6 has been pushed to the Fedora EPEL 6 stable repository. If problems still persist, please make note of it in this bug report.

Comment 20 doldev 2017-12-11 22:28:33 UTC
I just updated the package in production. I'll give you some feedback by the end of the week about possible issues.

For now, the only issue I found is the update process of the package. After updating the package the imapproxy was unable to connect to localhost:143. I killed the imapproxy process with kill because systemd had a failed status => inactive (dead).
After killing the dangling imapproxy process I was able to start the imapproxy with systemd.

Comment 21 doldev 2017-12-11 22:29:10 UTC
Btw: Our base system is CentOS 7 with EPEL.

Comment 22 doldev 2018-02-19 23:25:08 UTC
I just found out what caused the problem. We had a custom imapproxy.service file under /etc/systemd/system/imapproxy.service defined that was not compatible with the new version.

Removing the file fixed the problem.

Inspecting /usr/lib/systemd/system/imapproxy.service I encountered that the PIDFile points to /var/run/imapproxy.pid. Should this not be moved to /run/imapproxy.pid in the long run. https://fedoraproject.org/wiki/Packaging:Tmpfiles.d

While searching for the root cause of the problem I stumbled upon the Debian repository https://tracker.debian.org/pkg/up-imapproxy => https://github.com/rlaager/imapproxy-pkg .
They do a chroot preparation in the service startup => https://github.com/rlaager/imapproxy-pkg/blob/dc964f3d35f5ecbfc761fc83c32c8b7212b97288/debian/imapproxy.service#L9

This is quite neat. I had some service startup failures because to chroot folder was not present.

Comment 23 Chris Adams 2018-02-19 23:37:28 UTC
The /var/run vs. /run doesn't really matter, since /var/run is a symlink to /run. That unit setting will go away anyway, because I am planning to switch to running in the foreground with systemd Type=simple for F28 and above (got an updated package prepared today, just need to do a little more testing).

Also, I'm planning to enable the built-in chroot functionality by default (no prep needed because it chroots and then drops privileges, so will run in an empty directory).

Comment 24 doldev 2018-02-23 13:46:42 UTC
Ok, nice. I'll look forward to test the new releases.

About chroot. The mentioned Debian package only creates an empty folder and copies the localtime file into it. This might be a good preparation.
If you create an empty folder during the yum package installation and pointing the chroot option to it, this prepartion might be obsolete.
I thought it's kind of an handy to have a preparation script taking care of the chroot folder structure.

Comment 25 doldev 2018-02-23 14:10:31 UTC
This commit message explains the reason for changing to a preparation script. https://github.com/rlaager/imapproxy-pkg/commit/e84f29c9a8d150d13e2fc96f832d75481606d0b3#diff-9720c6b913f9b32ff7f2eeff2436d245


Note You need to log in before you can comment on or make changes to this bug.