Bug 1271230 - libssh agent support fails on big endian architectures
libssh agent support fails on big endian architectures
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: libssh (Show other bugs)
22
ppc64 Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Andreas Schneider
Fedora Extras Quality Assurance
RejectedBlocker AcceptedFreezeException
:
: 1275229 (view as bug list)
Depends On:
Blocks: ZedoraTracker PPCTracker F23BetaFreezeException F23FinalFreezeException
  Show dependency treegraph
 
Reported: 2015-10-13 08:34 EDT by Sinny Kumari
Modified: 2015-11-01 16:49 EST (History)
13 users (show)

See Also:
Fixed In Version: libssh-0.7.2-2.fc23 libssh-0.7.2-2.fc22
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-10-31 12:08:09 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Sinny Kumari 2015-10-13 08:34:23 EDT
Description of problem:
I tried building latest cockpit version 0.79 for ppc64 arch. Build hangs for around 1 day at the end of test and fails with message:
PASS: test-webservice 40 /web-service/resource/gzip-encoding
EXCEPTION: Timeout(86400) expired for command:
 # bash --login -c /usr/bin/rpmbuild -bb --target ppc64 --nodeps  /builddir/build/SPECS/cockpit.spec 

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/mockbuild/trace_decorator.py", line 84, in trace
    result = func(*args, **kw)
  File "/usr/lib/python2.7/site-packages/mockbuild/util.py", line 487, in do
    raise commandTimeoutExpired("Timeout(%s) expired for command:\n # %s\n%s" % (timeout, command, output))
commandTimeoutExpired: Timeout(86400) expired for command:
 # bash --login -c /usr/bin/rpmbuild -bb --target ppc64 --nodeps  /builddir/build/SPECS/cockpit.spec 

LEAVE do --> EXCEPTION RAISED

In test log, I see one of the test failing with message:

./test-sshtransport terminated with SIGABRT
FAIL: test-sshtransport 8 /ssh-transport/key-auth-message

Detailed log is available at http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=2808237

Version-Release number of selected component (if applicable): cockpit-0.79-1.fc22

This issue also occur on Fedora 23 which can be verified from build log http://ppc.koji.fedoraproject.org/koji/taskinfo?taskID=2808218
Comment 1 Sinny Kumari 2015-10-13 08:55:28 EDT
I complied and ran test locally on a ppc64 box to find reason of test failure. So far I found following minute details which may be useful:
Comment lines 857 and 858 of source file https://github.com/cockpit-project/cockpit/blob/0.79/src/ws/test-sshtransport.c to avoid running sub-tests key-auth-message and key-auth-failed .

After commenting them, all tests passes and F22 sub-packages build successfully too.

I will look further and will update reason of these tests failure.
Comment 2 Dan Horák 2015-10-13 09:05:54 EDT
also worth mentioning that only big endian arches seems affected, ppc64 and s390/s390x
Comment 5 Peter 2015-10-13 23:04:40 EDT
Tracked this down to a bug with libssh. While we are waiting for a fix do we want to disable key auth in cockpit on these arches. Or are we ok holding up these builds till it gets fixed.
Comment 6 Dan Horák 2015-10-14 02:54:52 EDT
(In reply to Peter from comment #5)
> Tracked this down to a bug with libssh. While we are waiting for a fix do we
> want to disable key auth in cockpit on these arches. Or are we ok holding up
> these builds till it gets fixed.

We would prefer a new cockpit release (can be just patched Fedora package) until there is fixed libssh (is the issue already tracked in separate bugzilla?). We don't have a new cockpit build for quite some time and cockpit is part of the Fedora Server flavour we produce for ppc64 and s390x.
Comment 7 Stef Walter 2015-10-14 03:05:46 EDT
Reassigning to libssh. Peter do you have more details of the bug?
Comment 8 Peter 2015-10-14 08:56:23 EDT
upstream bug report. https://red.libssh.org/issues/204
Comment 9 Sinny Kumari 2015-10-16 03:51:02 EDT
I applied provided patch for libssh (tried on both 0.7.1 and 0.7.2) https://git.libssh.org/users/asn/libssh.git/commit/?h=fix&id=dbe409ccdd59e25f5ea3639d5f4f91ed3c570a36 (additionally modifying #define bswap_32 to #define bswap_32(x) to avoid compilation error) and installed locally on ppc64 box.

Further tried building again cockpit-0.79, but I still see test failing:
FAIL: test-sshtransport 8 /ssh-transport/key-auth-message

For reference, latest build.log file is available at https://sinnykumari.fedorapeople.org/cockpit/build.log
Comment 10 Peter 2015-10-16 11:13:57 EDT
Just to be clear, the patch upstream does not fix the issue. I know it's confusing because the error message contains the word "Success" but it's still a failure.
Comment 11 Sinny Kumari 2015-10-20 03:27:32 EDT
With the latest libssh patch https://git.libssh.org/users/asn/libssh.git/commit/?h=fix&id=d60cfa24861394b1f5a8d80e26dadb41b3d3829e , cockpit-0.79 build failure has been resolved.

Successful local build log of cockpit-0.79 is available at https://sinnykumari.fedorapeople.org/cockpit/fixed_build/build.log .

I think we can now build cockpit-0.79 on big endian with newer libssh build(having "fix agent auth on big endian machines" patch included).
Comment 12 Fedora Update System 2015-10-22 05:46:19 EDT
libssh-0.7.2-2.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-66ba300501
Comment 13 Fedora Update System 2015-10-22 05:47:22 EDT
libssh-0.7.2-2.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-b242195d7f
Comment 14 Fedora Update System 2015-10-24 08:07:46 EDT
libssh-0.7.2-2.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update libssh'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-66ba300501
Comment 15 Fedora Blocker Bugs Application 2015-10-26 09:05:58 EDT
Proposed as a Blocker and Freeze Exception for 23-final by Fedora user karsten using the blocker tracking app because:

 This is blocking cockpit builds on bigendian secondary archs.
Requesting freeze exception as cockpit is required for the Server edition.
Comment 16 Peter Robinson 2015-10-26 11:36:47 EDT
*** Bug 1275229 has been marked as a duplicate of this bug. ***
Comment 17 Fedora Blocker Bugs Application 2015-10-26 13:23:33 EDT
Proposed as a Freeze Exception for 23-beta by Fedora user pbrobinson using the blocker tracking app because:

 The libssh breaks builds of cockpit on big endian architectures where cockpit is a core feature of the Server spins we ship. 

Proposing as a exception as it's for secondary arches
Comment 18 Petr Schindler 2015-10-26 13:25:45 EDT
Discussed at 2015-10-26 blocker review meeting: [1]. 

This bug was accepted as Final blocker: This bug doesn't qualify for blocker status, but we'd consider pulling in a tested fix despite freeze.

[1] https://meetbot.fedoraproject.org/fedora-blocker-review/2015-10-26/f23-blocker-review.2015-10-26-16.00.html
Comment 19 Petr Schindler 2015-10-26 13:30:12 EDT
Sorry for the confusion. I have really a bad day. This bug was rejected as blocker but accepted as Freeze Exception.
Comment 20 Fedora Update System 2015-10-26 14:28:58 EDT
libssh-0.7.2-2.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update libssh'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-b242195d7f
Comment 21 Adam Williamson 2015-10-26 18:54:06 EDT
For the record, I'm leaving this out of the RC4 compose, as it's big-endian only there's no point pulling it into the compose for primary arches, all of which are little-endian. The point of the FE is to get it pushed stable, and that will happen when it has sufficient karma.
Comment 22 Peter Robinson 2015-10-26 18:56:48 EDT
(In reply to awilliam@redhat.com from comment #21)
> For the record, I'm leaving this out of the RC4 compose, as it's big-endian
> only there's no point pulling it into the compose for primary arches, all of
> which are little-endian. The point of the FE is to get it pushed stable, and
> that will happen when it has sufficient karma.

We need it stable for the GA f23 tag
Comment 23 Kalev Lember 2015-10-27 01:24:09 EDT
Isn't libssh on the Server DVD? I don't think it can go stable if it wasn't part of the compose because we have to make sure that the version that's on the images matches with the version that's in the stable repo.
Comment 24 Adam Williamson 2015-10-27 02:18:05 EDT
Oh, yes, sorry - you're right. I missed that, was trying to avoid changing anything potentially risky for RC4. Well, if we get to RC5 we'll pull it in. my bad!
Comment 25 Fedora Update System 2015-10-31 12:08:02 EDT
libssh-0.7.2-2.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.
Comment 26 Fedora Update System 2015-11-01 16:49:22 EST
libssh-0.7.2-2.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.