Bug 928927 - FTBFS: python-pillow-2.0.0 on ppc64
Summary: FTBFS: python-pillow-2.0.0 on ppc64
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: python-pillow
Version: 19
Hardware: powerpc
OS: Linux
urgent
urgent
Target Milestone: ---
Assignee: Sandro Mani
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: AcceptedFreezeException
Depends On:
Blocks: F19Alpha-accepted, F19AlphaFreezeException
TreeView+ depends on / blocked
 
Reported: 2013-03-28 18:19 UTC by David Aquilina
Modified: 2014-06-24 00:19 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-04-20 04:23:23 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Fix incorrectly assumed little-endian CPU (2.89 KB, patch)
2013-03-28 22:28 UTC, Sandro Mani
no flags Details | Diff

Description David Aquilina 2013-03-28 18:19:56 UTC
python-pillow-2.0.0-3.gitde210a2.fc19 (as well as -2.git2e88848.fc19 and -1.git2f4207c.fc19) fails to build on ppc(64) due to failing tests: 
 

Tests/test_image_array.py:13: assert_equal(test("I"), ((100, 128), '<i4', 51200)) # FIXME: wrong? failed:
- got ((100, 128), '>i4', 51200), expected ((100, 128), '<i4', 51200)
Tests/test_image_array.py:14: assert_equal(test("F"), ((100, 128), '<f4', 51200)) # FIXME: wrong? failed:
- got ((100, 128), '>f4', 51200), expected ((100, 128), '<f4', 51200)
Tests/test_lib_pack.py:28: assert_equal(pack("I", "I"), [1, 0, 0, 0]) failed:
- got [0, 0, 0, 1], expected [1, 0, 0, 0]
Tests/test_lib_pack.py:30: assert_equal(pack("F", "F"), [0, 0, 128, 63]) failed:
- got [63, 128, 0, 0], expected [0, 0, 128, 63]
Tests/test_mode_i16.py:90: assert_equal(tobytes("I"), b"\x01\x00\x00\x00") failed:
- got '\x00\x00\x00\x01', expected '\x01\x00\x00\x00'

Full log: http://ppc.koji.fedoraproject.org/kojifiles/work/tasks/4514/1014514/build.log

Comment 1 Sandro Mani 2013-03-28 20:44:26 UTC
I'll investigate

Comment 2 Sandro Mani 2013-03-28 22:28:32 UTC
Created attachment 717867 [details]
Fix incorrectly assumed little-endian CPU

All these are endianness issues. The attached patch should fix the issues, please give it a try and I'll do a pull request upstream and commit it to the fedora repo.

Comment 3 Dan Horák 2013-03-31 11:44:24 UTC
the results for our big-endian platforms are:
ppc (32bit) - all tests are passed
s390 (32bit) - all tests expect webp are passed

...
running test_file_webp ...
Tests/test_file_webp.py:21: assert_image_equal(im, target) failed:
- got different content
Tests/test_file_webp.py:42: assert_image_equal(im, target) failed:
- got different content
...

ppc64 (64bit) - test_image_getcolors and test_image_quantize are failing
...
running test_image_getcolors ...
Tests/test_image_getcolors.py:48: assert_equal(A, None) failed:
- got [(3249, (0, 0, 0)), (13135, (255, 255, 255))], expected None
Tests/test_image_getcolors.py:52: assert_equal(A, expected) failed:
- got [(3249, (0, 0, 0)), (13135, (255, 255, 255))], expected [(3236, (227, 183, 147)), (6297, (143, 84, 81)), (6851, (208, 143, 112))]
Tests/test_image_getcolors.py:56: assert_equal(A, expected) failed:
- got [(3249, (0, 0, 0)), (13135, (255, 255, 255))], expected [(3236, (227, 183, 147)), (6297, (143, 84, 81)), (6851, (208, 143, 112))]
Tests/test_image_getcolors.py:60: assert_equal(A, expected) failed:
- got [(3249, (0, 0, 0)), (13135, (255, 255, 255))], expected [(3236, (227, 183, 147)), (6297, (143, 84, 81)), (6851, (208, 143, 112))]
Tests/test_image_getcolors.py:64: assert_equal(A, expected) failed:
- got [(3249, (0, 0, 0)), (13135, (255, 255, 255))], expected [(3236, (227, 183, 147)), (6297, (143, 84, 81)), (6851, (208, 143, 112))]
...
running test_image_quantize ...
Tests/test_image_quantize.py:22: assert len(im.getcolors()) == 100 failed:
Traceback (most recent call last):
  File "Tests/test_image_quantize.py", line 22, in test_octree_quantize
    assert len(im.getcolors()) == 100
AssertionError
...

s390x (64bit) - same as ppc64 + the webp failure from s390

scratch build on ppc/ppc64 - http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=1020659

Comment 4 Dan Horák 2013-03-31 11:45:17 UTC
please fix a typo in the previous comment
s390 (32bit) - all tests except webp are passed

Comment 5 Sandro Mani 2013-03-31 18:39:24 UTC
The webp test is fixed upstream, the image_get_colors failures are probably a consequence of the quantize code misbehaving, and as for why the quantization code is failing, I'll need to investigate...

Comment 6 Sandro Mani 2013-04-01 21:15:09 UTC
I've tried running a fedora ppc64 iso via qemu but all I get is a yellowish screen when running "qemu-system-ppc64 -cdrom /path/to/iso -m 1024M"... Any way I can get a ppc environment running without corresponding hardware?

Comment 7 Dan Horák 2013-04-02 07:01:42 UTC
(In reply to comment #6)
> I've tried running a fedora ppc64 iso via qemu but all I get is a yellowish
> screen when running "qemu-system-ppc64 -cdrom /path/to/iso -m 1024M"... Any
> way I can get a ppc environment running without corresponding hardware?

you can get an account on a real ppc64 hardware, please check #fedora-ppc IRC channel on FreeNode in the afternoon EU time or wait for a contact via email. I'll ask the right people to act.

Comment 8 Sandro Mani 2013-04-03 19:42:44 UTC
This will require some effort, since overall the quantization code looks very fragile (void* to int casts, assumptions that an unsigned long is 4 bytes, etc - actually a miracle that it works at all on other platforms). But I'm working on it.

Comment 9 Dan Horák 2013-04-04 15:48:31 UTC
Proposing as F19 Alpha blocker, right now we have a workaround in form of python-pillow-2.0.0-3.1.gitde210a2.fc19 which disables the test-suite for ppc/ppc64 and s390/s390x and has no change on x86/x86_64. We need this build to be tagged to f19.

Comment 10 Adam Williamson 2013-04-04 21:26:14 UTC
For what reason? Why should this block the release? What is the practical result of the bug?

Comment 11 Dan Horák 2013-04-05 09:55:13 UTC
The secondary arches build packages within buildroots that contains ideally only the same NVR as the buildroots in primary, practically there can be few newer NVRs for packages that fail to build and must be fixed. But definitely no older NVRs are allowed. Not having python-pillow-2.0.0-3.1.gitde210a2.fc19 (and latest pygobject3) tagged in f19 means secondary arches can't build cca 300 packages that existed in f19 before the alpha freeze eg. anaconda or the latest GNOME stuff. The release engineering tag syncing script ensure that same NVRs are tagged between primary and secondaries effectively meaning no newer NVR is possible in secondary. This won't allow to make an Alpha compose on secondaries that matches primary.

Risk for inclusion of python-pillow-2.0.0-3.1.gitde210a2.fc19 instead of python-pillow-2.0.0-3.gitde210a2.fc19 in primary is zero as there is no difference in these 2 packages for primary.

Comment 12 Dan Horák 2013-04-05 10:51:41 UTC
switching to Freeze Exception which is more appropriate

Comment 13 Sandro Mani 2013-04-07 16:40:54 UTC
Sorry for the silence, illness kept me in bed the past few days...

From what I understand, the version disparity is the biggest problem atm - I'll update pillow in F19 and rawhide in primary.

As for the issue itself, my efforts on narrowing down the issue have so far been unsuccessful. Attempting to replace various variable types with types from stdint.h to match the assumptions made did not help.

Comment 14 Sandro Mani 2013-04-07 23:15:16 UTC
Got it! Patches commited (see [1]) and sent upstream. ppc scratch-build here [2].

[1] http://pkgs.fedoraproject.org/cgit/python-pillow.git/commit/?id=da3613d08948dda9e6597bc47cb1b0ffc641b582
[2] http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=1033901

Comment 15 Fedora Update System 2013-04-08 10:49:19 UTC
python-pillow-2.0.0-5.git93a488e.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/python-pillow-2.0.0-5.git93a488e.fc19

Comment 16 Fedora Update System 2013-04-08 15:45:17 UTC
Package python-pillow-2.0.0-5.git93a488e.fc19:
* should fix your issue,
* was pushed to the Fedora 19 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing python-pillow-2.0.0-5.git93a488e.fc19'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-5105/python-pillow-2.0.0-5.git93a488e.fc19
then log in and leave karma (feedback).

Comment 17 Adam Williamson 2013-04-08 17:00:29 UTC
Discussed at 2013-04-08 freeze exception review meeting: http://meetbot.fedoraproject.org/fedora-blocker-review/2013-04-08/f19alpha-blocker-review-5.2013-04-08-16.01.log.txt . This is accepted as a freeze exception as a secondary arch blocker, but we are *only* granting a freeze exception to the 'workaround' build python-pillow-2.0.0-3.1.gitde210a2.fc19 , not the 'fix' build python-pillow-2.0.0-5.git93a488e.fc19 . The fix build looks like too big and untested a change to be worth endangering the primary arches for.

Comment 18 Dan Horák 2013-04-08 18:09:20 UTC
Adam, thanks, the workaround build is what was requested and is sufficient for our purposes.

Comment 19 Fedora Update System 2013-04-08 19:02:55 UTC
python-pillow-2.0.0-3.1.gitde210a2.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/python-pillow-2.0.0-3.1.gitde210a2.fc19

Comment 20 Fedora Update System 2013-04-19 17:42:37 UTC
python-pillow-2.0.0-6.gitd1c6db8.fc19 has been submitted as an update for Fedora 19.
https://admin.fedoraproject.org/updates/python-pillow-2.0.0-6.gitd1c6db8.fc19

Comment 21 Fedora Update System 2013-04-20 04:23:26 UTC
python-pillow-2.0.0-3.1.gitde210a2.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 22 Fedora Update System 2013-04-25 14:03:19 UTC
python-pillow-2.0.0-6.gitd1c6db8.fc19 has been pushed to the Fedora 19 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.