Bug 1909674

Summary: python27 module: python-psycopg2 FTBFS
Product: Red Hat Enterprise Linux 8 Reporter: Tomas Orsava <torsava>
Component: python-psycopg2Assignee: Filip Januš <fjanus>
Status: CLOSED ERRATA QA Contact: Lukáš Zachar <lzachar>
Severity: unspecified Docs Contact:
Priority: urgent    
Version: 8.4CC: cstratak, databases-maint, hhorak, lbalhar, pkubat, praiskup, torsava
Target Milestone: rcKeywords: Reopened, Triaged
Target Release: 8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-11-14 15:36:08 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
build log 1
none
root log 1
none
build log 2
none
root log 2 none

Description Tomas Orsava 2020-12-21 11:26:28 UTC
Created attachment 1740933 [details]
build log 1

Description of problem:
Hi, I have tried to rebuild all packages in the python27 module in RHEL 8.4.0, and psycopg2 fails to build. I've run the build at least 3-4 times, so it's not a fluke. I'm including logs.


Version-Release number of selected component (if applicable):
python-psycopg2-2.7.5-7.scrmod+el8.4.0+9194+858efbc8
python27 module
RHEL 8.4.0


How reproducible:
Always

Steps to Reproduce:
1. rhpkg clone modules/python27
2. git checkout 2.7-rhel-8.4.0
3. rhpkg module-build --scratch --optional rebuild_strategy=all
4. rhpkg module-build-watch <module build number>

Actual results:

======================================================================
ERROR: test_external_close_async (psycopg2.tests.test_cursor.CursorTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILDROOT/python-psycopg2-2.7.5-7.scrmod+el8.4.0+9203+9523704e.s390x/usr/lib64/python2.7/site-packages/psycopg2/tests/testutils.py", line 390, in skip_if_no_superuser_
    return f(self)
  File "/builddir/build/BUILDROOT/python-psycopg2-2.7.5-7.scrmod+el8.4.0+9203+9523704e.s390x/usr/lib64/python2.7/site-packages/psycopg2/tests/testutils.py", line 434, in skip_if_windows_
    return f(self)
  File "/builddir/build/BUILDROOT/python-psycopg2-2.7.5-7.scrmod+el8.4.0+9203+9523704e.s390x/usr/lib64/python2.7/site-packages/psycopg2/tests/testutils.py", line 289, in skip_before_postgres__
    return f(self)
  File "/builddir/build/BUILDROOT/python-psycopg2-2.7.5-7.scrmod+el8.4.0+9203+9523704e.s390x/usr/lib64/python2.7/site-packages/psycopg2/tests/test_cursor.py", line 577, in test_external_close_async
    self._test_external_close(control_conn, connect_func, wait_func)
  File "/builddir/build/BUILDROOT/python-psycopg2-2.7.5-7.scrmod+el8.4.0+9203+9523704e.s390x/usr/lib64/python2.7/site-packages/psycopg2/tests/test_cursor.py", line 602, in _test_external_close
    self.assertRaises(psycopg2.OperationalError, f)
  File "/usr/lib64/python2.7/unittest/case.py", line 511, in assertRaises
    callableObj(*args, **kwargs)
  File "/builddir/build/BUILDROOT/python-psycopg2-2.7.5-7.scrmod+el8.4.0+9203+9523704e.s390x/usr/lib64/python2.7/site-packages/psycopg2/tests/test_cursor.py", line 600, in f
    wait_func(victim_conn)
ProgrammingError: close cannot be used while an asynchronous query is underway
----------------------------------------------------------------------


Expected results:
Successful build

Additional info:
Logs attached

Comment 1 Tomas Orsava 2020-12-21 11:27:21 UTC
Created attachment 1740934 [details]
root log 1

Comment 2 Tomas Orsava 2020-12-21 11:27:53 UTC
Created attachment 1740935 [details]
build log 2

Comment 3 Tomas Orsava 2020-12-21 11:28:16 UTC
Created attachment 1740936 [details]
root log 2

Comment 4 Honza Horak 2020-12-23 13:08:08 UTC
I suspected updated libpq being the cause, but when I tried in mock with libpq-12.1, it failed the same. So, libpq update to 13.1 is probably not the cause.

Comment 5 Honza Horak 2020-12-23 13:52:24 UTC
The failing test case is a regression test for this issue:
https://github.com/psycopg/psycopg2/issues/443

Added together with a fix in
https://github.com/psycopg/psycopg2/pull/446/files

I tried to increase the time used there, but it did not help.

Comment 6 Honza Horak 2020-12-23 15:13:28 UTC
Some slight progress -- it eventually is caused by libpq change, but it does not work with v12, only with v10.x when the last successful build was seen. I tried to rebuild with libpq v10.x and it worked.

It also works with newest version 2.8.6, so upstream already fixed the code to work fine with libpq 13.x.

There are more changes that can cause this, but this one looks the most suspicious: https://github.com/psycopg/psycopg2/commit/734845b79adc733bd9b7414602111824c55fdb2c#diff-11101e082302b2536010044ae80a15535d6c065885c8227886f21e46782275a8

Anyway, before trying to bisect what change fixed the compatibility with libpq, I'm more thinking about updating python-psycopg2 to either latest version or at least 2.8.4 that is also used in case of python38 module. The reason is that there might be more corner case issues with libpq 13.x.

Comment 14 RHEL Program Management 2022-06-21 07:27:30 UTC
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Comment 15 Lumír Balhar 2023-06-27 10:19:53 UTC
I'm reopening the bug because my last try to rebuild the python27 module failed on psycopg2 with the exact same error. Unfortunately, this time we cannot just skip it because all the components have some changes in dist-git which means that MBS will rebuild it no matter which strategy we choose.

Comment 17 Lumír Balhar 2023-06-27 10:21:44 UTC
And because we have to ship a CVE fix in one of the components of the module, this is kinda important for us.

The latest scratch build I have: https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=53466153

Comment 19 Filip Januš 2023-07-07 07:48:31 UTC
A test case is part of the backport. Also, the failure occurred during the check phase within the upstream test suite. So after the patch was applied, the error disappeared.

Comment 24 errata-xmlrpc 2023-11-14 15:36:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: python27:2.7 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:7042