Bug 1685609

Summary: python35: Invent a workaround for when compat-openssl10 is gone
Product: [Fedora] Fedora Reporter: Miro Hrončok <mhroncok>
Component: python35Assignee: Victor Stinner <vstinner>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: cstratak, igor.raits, mhroncok, pviktori, python-sig, tmraz
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-05-14 13:53:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
build.log of python35 with openssl-devel 1.1.1 none

Description Miro Hrončok 2019-03-05 16:14:58 UTC 
Created attachment 1541030 [details]
build.log of python35 with openssl-devel 1.1.1

compat-openssl10 will likely get orphaned before Fedora 31.

Source: https://bugzilla.redhat.com/show_bug.cgi?id=1673419#c2

Since we don't want to deal with maintaining it ourselves, we need to invent a workaround.

python35 builds fine with openssl 1.1.1, however there are test failures:

BUILDSTDERR: test.test_asyncio.test_windows_utils (unittest.loader.ModuleSkipped) ... test test_asyncio failed
skipped 'Windows only'
======================================================================
ERROR: test_create_server_ssl_match_failed (test.test_asyncio.test_events.EPollEventLoopTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.5.7rc1/Lib/test/test_asyncio/test_events.py", line 1172, in test_create_server_ssl_match_failed
    proto.transport.close()
AttributeError: 'NoneType' object has no attribute 'close'
======================================================================
ERROR: test_create_server_ssl_match_failed (test.test_asyncio.test_events.PollEventLoopTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.5.7rc1/Lib/test/test_asyncio/test_events.py", line 1172, in test_create_server_ssl_match_failed
    proto.transport.close()
AttributeError: 'NoneType' object has no attribute 'close'
======================================================================
ERROR: test_create_server_ssl_match_failed (test.test_asyncio.test_events.SelectEventLoopTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.5.7rc1/Lib/test/test_asyncio/test_events.py", line 1172, in test_create_server_ssl_match_failed
    proto.transport.close()
AttributeError: 'NoneType' object has no attribute 'close'
----------------------------------------------------------------------
Ran 1024 tests in 16.193s
FAILED (errors=3, skipped=3)

test_wrong_cert (test.test_ssl.ThreadedTests)
BUILDSTDERR: Connecting when the server rejects the client's certificate ... test test_ssl failed
SSLError is SSLError(1, '[SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:728)')
ok
======================================================================
FAIL: test_options (test.test_ssl.ContextTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.5.7rc1/Lib/test/test_ssl.py", line 866, in test_options
    self.assertEqual(default, ctx.options)
AssertionError: 2181169236 != 2182217812
======================================================================
FAIL: test_default_ecdh_curve (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.5.7rc1/Lib/test/test_ssl.py", line 3064, in test_default_ecdh_curve
    self.assertIn("ECDH", s.cipher()[0])
AssertionError: 'ECDH' not found in 'TLS_AES_256_GCM_SHA384'
======================================================================
FAIL: test_shared_ciphers (test.test_ssl.ThreadedTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.5.7rc1/Lib/test/test_ssl.py", line 3382, in test_shared_ciphers
    self.fail(name)
AssertionError: TLS_AES_256_GCM_SHA384
----------------------------------------------------------------------
Ran 103 tests in 1.642s
FAILED (failures=3, skipped=8)

A workaround might be to skip those tests.
Comment 1 Igor Raits 2019-03-05 16:19:24 UTC 
I would say that we need to backport patches from 3.6 to support new openssl.
Comment 2 Miro Hrončok 2019-03-05 16:22:44 UTC 
That would defeat the purpose of the package.

If people use our python35 to test their code works on "general" 3.5, we cannot add features to 3.5.
Comment 3 Petr Viktorin (pviktori) 2019-03-19 14:50:22 UTC 
Let's skip the tests just before thy break.
If anyone wants a better solution, help is welcome.
Comment 4 Victor Stinner 2019-03-20 18:12:13 UTC 
Python 3.5.7 has been release with basic OpenSSL 1.1.1 support. Update python35 package to Python 3.5.7 should enough, no?

See also bz#1685612 for Python 3.4.
Comment 5 Victor Stinner 2019-03-20 18:15:44 UTC 
> Let's skip the tests just before they break.

I'm fine with skipping test_ssl and test_asyncio.

By the way, even on the master branch of Python upstream, test_asyncio fails randomly (likely because of TLS v1.3)... https://bugs.python.org/issue35998
Comment 6 Miro Hrončok 2019-03-20 18:18:30 UTC 
python35 package is Python 3.5.7.

When I've checked with 3.5.7rc1, I got the test failures attached here. I have not checked with 3.5.7 final. Were there any changes?
Comment 7 Victor Stinner 2019-03-21 14:54:14 UTC 
I'm sorry, I was confused by the issue requesting OpenSSL 1.1.1 support in Python 3.4. I know understand that a few test_ssl are failing with OpenSSL 1.1.1 and the question is how to fix them. Either skip test_ssl, skip the failing tests, or try to fix them.

As I wrote, I'm fine with skipping test_ssl and test_asyncio. python35 doesn't accept bugfixes anymore, and failures are mostly bugs in the tests rather than in Python itself (ssl and asyncio modules).
Comment 8 Victor Stinner 2019-04-02 15:19:16 UTC 
I created https://src.fedoraproject.org/rpms/python35/pull-request/23 to skip the 3 test_ssl tests which fail with OpenSSL 1.1.1.
Comment 9 Victor Stinner 2019-04-02 15:40:32 UTC 
With python35-3.5.7-1.fc31.x86_64 on Rawhide, all tests pass:

$ python3.5 -m test -j0 test_hashlib test_ssl test_asyncio
0:00:00 load avg: 0.29 [1/3] test_hashlib
0:00:01 load avg: 0.42 [2/3] test_ssl
0:00:15 load avg: 0.36 [3/3] test_asyncio
All 3 tests OK.
Tests result: SUCCESS

The _ssl module is linked to OpenSSL 1.0:

$ python3.5 -c 'import ssl; print(ssl.OPENSSL_VERSION)'
OpenSSL 1.0.2o-fips  27 Mar 2018

Similar output with python35-3.5.7-1.fc29.x86_64 on Fedora 29.

My PR https://src.fedoraproject.org/rpms/python35/pull-request/23 prepares the python35 package to replace "BuildRequires: compat-openssl10-devel" with "BuildRequires: openssl-devel".
Comment 10 Petr Viktorin (pviktori) 2019-05-14 13:53:16 UTC 
Fix has been merged; no build necessary.