Bug 1937719 - Test failures on s390x
Summary: Test failures on s390x
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-shapely
Version: rawhide
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Ben Beasley
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-03-11 12:28 UTC by Ben Beasley
Modified: 2021-03-27 00:31 UTC (History)
6 users (show)

Fixed In Version: python-shapely-1.7.1-7.fc34 python-shapely-1.7.1-7.fc33 python-shapely-1.7.1-9.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-03-19 20:13:57 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Ben Beasley 2021-03-11 12:28:30 UTC
Description of problem:


=================================== FAILURES ===================================
_______________________________ test_dumps_srid ________________________________
    def test_dumps_srid():
        p1 = Point(1.2, 3.4)
        result = dumps(p1)
>       assert bin2hex(result) == "0101000000333333333333F33F3333333333330B40"
E       AssertionError: assert '00000000013F...B333333333333' == '010100000033...3333333330B40'
E         - 0101000000333333333333F33F3333333333330B40
E         + 00000000013FF3333333333333400B333333333333
tests/test_wkb.py:23: AssertionError
____________________________ test_dumps_endianness _____________________________
    def test_dumps_endianness():
        p1 = Point(1.2, 3.4)
        result = dumps(p1)
>       assert bin2hex(result) == "0101000000333333333333F33F3333333333330B40"
E       AssertionError: assert '00000000013F...B333333333333' == '010100000033...3333333330B40'
E         - 0101000000333333333333F33F3333333333330B40
E         + 00000000013FF3333333333333400B333333333333
tests/test_wkb.py:31: AssertionError
________________________________ test_dumps_hex ________________________________
    def test_dumps_hex():
        p1 = Point(1.2, 3.4)
        result = dumps(p1, hex=True)
>       assert result == "0101000000333333333333F33F3333333333330B40"
E       AssertionError: assert '00000000013F...B333333333333' == '010100000033...3333333330B40'
E         - 0101000000333333333333F33F3333333333330B40
E         + 00000000013FF3333333333333400B333333333333
tests/test_wkb.py:41: AssertionError
_______________________________ test_loads_srid ________________________________
    def test_loads_srid():
        # load a geometry which includes an srid
        geom = loads(hex2bin("0101000020E6100000333333333333F33F3333333333330B40"))
        assert isinstance(geom, Point)
        assert geom.coords[:] == [(1.2, 3.4)]
        # by default srid is not exported
        result = dumps(geom)
>       assert bin2hex(result) == "0101000000333333333333F33F3333333333330B40"
E       AssertionError: assert '00000000013F...B333333333333' == '010100000033...3333333330B40'
E         - 0101000000333333333333F33F3333333333330B40
E         + 00000000013FF3333333333333400B333333333333
tests/test_wkb.py:51: AssertionError
_______________________________ test_point_empty _______________________________
    @requires_geos_39
    def test_point_empty():
        g = wkt.loads("POINT EMPTY")
>       assert g.wkb_hex == "0101000000000000000000F87F000000000000F87F"
E       AssertionError: assert '00000000017F...8000000000000' == '010100000000...000000000F87F'
E         - 0101000000000000000000F87F000000000000F87F
E         + 00000000017FF80000000000007FF8000000000000
tests/test_wkb.py:67: AssertionError
______________________________ test_point_z_empty ______________________________
    @requires_geos_39
    def test_point_z_empty():
        g = wkt.loads("POINT Z EMPTY")
>       assert g.wkb_hex == \
            "0101000080000000000000F87F000000000000F87F000000000000F87F"
E       AssertionError: assert '00800000017F...8000000000000' == '010100008000...000000000F87F'
E         - 0101000080000000000000F87F000000000000F87F000000000000F87F
E         ?  - ^                 ^  --              --            ----
E         + 00800000017FF80000000000007FF80000000000007FF8000000000000
E         ?   ^    ++++++             ^^              ++
tests/test_wkb.py:73: AssertionError
=========================== short test summary info ============================
FAILED tests/test_wkb.py::test_dumps_srid - AssertionError: assert '000000000...
FAILED tests/test_wkb.py::test_dumps_endianness - AssertionError: assert '000...
FAILED tests/test_wkb.py::test_dumps_hex - AssertionError: assert '0000000001...
FAILED tests/test_wkb.py::test_loads_srid - AssertionError: assert '000000000...
FAILED tests/test_wkb.py::test_point_empty - AssertionError: assert '00000000...
FAILED tests/test_wkb.py::test_point_z_empty - AssertionError: assert '008000...
============= 6 failed, 344 passed, 13 skipped, 2 xfailed in 1.95s =============

Version-Release number of selected component (if applicable): 1.7.1-4

How reproducible:


Steps to Reproduce:
1. Build using --with-failing_s390x_tests.

Actual results:

Errors shown above.

Expected results:

All tests pass—which they do on other architectures.

Additional info:

These tests are skipped for now. This has probably been a problem for some time, as it is mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=1853218. It was obscured by https://bugzilla.redhat.com/show_bug.cgi?id=1907955, which broke the build on all platforms.

It is not immediately apparent to me what the cause of these failures is—I am guessing platform-dependent floating-point implementation differences—nor is it obvious whether they represent a real problem or a trivial difference. It should be reported upstream, although they may not be able to help with this platform.

I have not had a chance to look into this too closely. Assistance is welcome.

Comment 1 Jerry James 2021-03-11 16:45:00 UTC
I'm looking at the first failing test, test_dumps_srid().  It looks like the geos output is endian-specific.  I'm looking at src/io/WKBWriter.cpp.  The WKBWriter::writePoint() method starts at line 138.  The first thing it does is:

    writeByteOrder();

That writes a 0 byte (geos::io::wkbXDR) on big endian machines and a 1 byte (geos::io::wkbNDR) on little endian machines.  Next:

    writeGeometryType(WKBConstants::wkbPoint, g.getSRID());

The value of geos::io::wkbPoint is 1, but it is written as a 32-bit integer in native byte order, so you get 1, 0, 0, 0 on little endian machines and 0, 0, 0, 1 on big endian machines.  Next:

    writeSRID(g.getSRID());

which does nothing in this test, since the SRID is zero.  Next:

    writeCoordinateSequence(*cs, false);

Now we are at line 285.  The value of sized is false, and the value of size is 1, so we make this call:

    writeCoordinate(cs, 0, false);

Now we are at line 303.  This method fetches the x and y coordinates and passes them both to:

    ByteOrderValues::putDouble(coordinate, buf, ByteOrderValues::ENDIAN_LITTLE or ByteOrderValues::ENDIAN_BIG);

Looking at src/io/ByteOrderValues.cpp, line 137, we see that the raw bytes of the double are written out in native byte order.  So for that first test_dumps_srid() test, the expected output from writing Point(1.2, 3.4) is this on little endian machines:

    "0101000000333333333333F33F3333333333330B40"

but is this on big endian machines:

    "00000000013FF3333333333333400B333333333333"

exactly as reported by the s390x build.  The bottom line is that geos output is endian-specific, but the python-shapely tests were written to assume a little endian architecture.

Comment 2 Jerry James 2021-03-11 16:49:38 UTC
A completely different matter, but has anyone noticed that python3-descartes is never installed now?  The line that reads:

%if 0%{?without bootstrap}

is wrong.  It should be:

%if %{without bootstrap}

i.e., no "0" and no "?".

Comment 3 Ben Beasley 2021-03-12 02:19:24 UTC
> I'm looking at the first failing test, test_dumps_srid().  It looks like the geos output is endian-specific. […]

Thanks, that saves a lot of time. I figured it would be something like that. That should be easy to patch and offer upstream, and I can reference your explanation.

> The line that reads: … is wrong.

Yes, it is, and I’ll fix it. Thanks also for reviewing these recent changes.

Comment 4 Ben Beasley 2021-03-12 15:28:49 UTC
https://github.com/Toblerity/Shapely/issues/1102

Comment 5 Fedora Update System 2021-03-12 16:11:49 UTC
FEDORA-EPEL-2021-31c75e514c has been submitted as an update to Fedora EPEL 8. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-31c75e514c

Comment 6 Fedora Update System 2021-03-12 16:30:01 UTC
FEDORA-2021-fac96509d3 has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-fac96509d3

Comment 7 Fedora Update System 2021-03-12 17:16:18 UTC
FEDORA-2021-2bd5f62a6f has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2021-2bd5f62a6f

Comment 8 Fedora Update System 2021-03-12 18:57:16 UTC
FEDORA-2021-fac96509d3 has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-fac96509d3`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-fac96509d3

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 9 Fedora Update System 2021-03-12 20:28:52 UTC
FEDORA-EPEL-2021-31c75e514c has been pushed to the Fedora EPEL 8 testing repository.

You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2021-31c75e514c

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 10 Fedora Update System 2021-03-12 20:35:19 UTC
FEDORA-2021-2bd5f62a6f has been pushed to the Fedora 33 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-2bd5f62a6f`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-2bd5f62a6f

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

Comment 11 Fedora Update System 2021-03-19 20:13:57 UTC
FEDORA-2021-fac96509d3 has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 12 Fedora Update System 2021-03-20 01:14:27 UTC
FEDORA-2021-2bd5f62a6f has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 13 Fedora Update System 2021-03-27 00:31:23 UTC
FEDORA-EPEL-2021-31c75e514c has been pushed to the Fedora EPEL 8 stable repository.
If problem still persists, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.