Bug 1920461

Summary: cheroot (cherrypy) indefinitely hangs under a moderate rate of requests and never recovers
Product: [Fedora] Fedora EPEL Reporter: Ernesto Puerta <epuertat>
Component: python-cherootAssignee: Dan Radez <dradez>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: epel8CC: almartin, athakkar, dpivonka, dradez, igor.raits, jcaratza, jolmomar, kdreyer, nizamudeena
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: python-cheroot-8.5.2-1.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-01-28 02:18:06 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1920534    

Comment 1 Ken Dreyer (Red Hat) 2021-01-26 15:18:01 UTC
Dan and Justin, I've merged master into EPEL 8 here: https://src.fedoraproject.org/rpms/python-cheroot/pull-request/11

Would you please merge that?

Comment 2 Ken Dreyer (Red Hat) 2021-01-26 16:04:30 UTC
Steps to reproduce this bug:

1. Install python3-cherrypy and python3-cheroot-8.5.1 from EPEL

2. Generate HTTPS key and cert following the instructions at https://docs.cherrypy.org/en/latest/deploy.html#ssl-support
   openssl genrsa -out privkey.pem 2048
   openssl req -new -x509 -days 365 -key privkey.pem -out cert.pem

3. Create a "hello.py" app:

import cherrypy

cherrypy.server.ssl_certificate = "cert.pem"
cherrypy.server.ssl_private_key = "privkey.pem"

class HelloWorld(object):
    @cherrypy.expose
    def index(self):
        return "Hello World!"

cherrypy.quickstart(HelloWorld())

4. Run the app in a terminal

5. In another terminal, run "ab -c20 -n1000 https://localhost:8080/" until it hangs. In my experiments running this ab invocation three times was enough to get it to consistently hang. "-n10000" was less consistent in triggering this bug. Basically ab will report a hard timeout like "apr_pollset_poll: The timeout specified has expired (70007)", or an error like "SSL error".

I wonder how we could build this into Fedora CI (https://docs.fedoraproject.org/ar/ci/) so that we can run this kind of test on every CherryPy and Cheroot package build.

Comment 3 Ken Dreyer (Red Hat) 2021-01-26 16:07:49 UTC
I've built RPMs with https://src.fedoraproject.org/rpms/python-cheroot/pull-request/11 at https://fedorapeople.org/~ktdreyer/bz1920461/ . I've run the same test with this newer version and it passes.

Comment 4 Fedora Update System 2021-01-26 17:11:36 UTC
FEDORA-EPEL-2021-848a87b9dc has been submitted as an update to Fedora EPEL 8. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-848a87b9dc

Comment 5 Fedora Update System 2021-01-27 01:31:34 UTC
FEDORA-EPEL-2021-848a87b9dc has been pushed to the Fedora EPEL 8 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-848a87b9dc

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 6 Fedora Update System 2021-01-28 02:18:06 UTC
FEDORA-EPEL-2021-848a87b9dc has been pushed to the Fedora EPEL 8 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 7 Ken Dreyer (Red Hat) 2021-02-02 00:45:06 UTC
I took a stab at adding a CI test here: https://src.fedoraproject.org/fork/ktdreyer/rpms/python-cherrypy/blob/tests/f/tests

I have not tried running this at all yet. I've emailed ci.org for help.

Comment 10 Fedora Update System 2021-02-03 01:54:34 UTC
FEDORA-EPEL-2021-4b4520f49d has been pushed to the Fedora EPEL 8 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-4b4520f49d

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.