Bug 539183 - FTBFS nss-3.12.4-13.1.fc13
FTBFS nss-3.12.4-13.1.fc13
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: nss (Show other bugs)
rawhide
All Linux
high Severity high
: ---
: ---
Assigned To: Elio Maldonado Batiz
Fedora Extras Quality Assurance
http://linux.dell.com/files/fedora/Fi...
: Reopened, Triaged
Depends On:
Blocks: 517000 F13FTBFS
  Show dependency treegraph
 
Reported: 2009-11-19 12:08 EST by FTBFS
Modified: 2011-05-23 14:55 EDT (History)
6 users (show)

See Also:
Fixed In Version: nss-3.12.9-8.fc13
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-02-28 23:22:57 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
root.log (33.09 KB, text/plain)
2009-11-19 12:08 EST, FTBFS
no flags Details
build.log (1.88 MB, text/plain)
2009-11-19 12:08 EST, FTBFS
no flags Details
mock.log (952 bytes, text/plain)
2009-11-19 12:08 EST, FTBFS
no flags Details
root.log (43.77 KB, text/plain)
2009-11-19 12:08 EST, FTBFS
no flags Details
build.log (1.88 MB, text/plain)
2009-11-19 12:08 EST, FTBFS
no flags Details
mock.log (960 bytes, text/plain)
2009-11-19 12:08 EST, FTBFS
no flags Details
Short-term fix for ssl test suites hangs on ipv6 type connections (1.20 KB, patch)
2011-02-24 18:03 EST, Elio Maldonado Batiz
no flags Details | Diff


External Trackers
Tracker ID Priority Status Summary Last Updated
Mozilla Foundation 617723 None None None Never

  None (edit)
Description FTBFS 2009-11-19 12:08:44 EST
nss-3.12.4-13.1.fc13.src.rpm Failed To Build From Source against the rawhide tree.  See http://fedoraproject.org/wiki/FTBFS for more information.
Comment 1 FTBFS 2009-11-19 12:08:47 EST
Setting to ASSIGNED per Fedora Bug Triage workflow.  https://fedoraproject.org/wiki/BugZappers/BugStatusWorkFlow
Comment 2 FTBFS 2009-11-19 12:08:50 EST
Created attachment 372046 [details]
root.log

root.log for i386
Comment 3 FTBFS 2009-11-19 12:08:52 EST
Created attachment 372047 [details]
build.log

build.log for i386
Comment 4 FTBFS 2009-11-19 12:08:54 EST
Created attachment 372048 [details]
mock.log

mock.log for i386
Comment 5 FTBFS 2009-11-19 12:08:55 EST
Created attachment 372049 [details]
root.log

root.log for x86_64
Comment 6 FTBFS 2009-11-19 12:08:57 EST
Created attachment 372050 [details]
build.log

build.log for x86_64
Comment 7 FTBFS 2009-11-19 12:08:59 EST
Created attachment 372051 [details]
mock.log

mock.log for x86_64
Comment 8 Elio Maldonado Batiz 2009-11-19 17:35:47 EST
mock.log shows : "ERROR: Exception(nss-3.12.4-13.1.fc13.src.rpm) 
Config(fedora-development-x86_64) 360 minutes 15 seconds"
It timed out after 10 hours?

Without making any changes I just did a scratch build 
http://koji.fedoraproject.org/koji/taskinfo?taskID=1818268
The i686 build, for example, shows
Created	Thu, 19 Nov 2009 21:41:29 UTC
Completed Thu, 19 Nov 2009 21:57:07 UTC
or about 16 minutes
Comment 9 Matt Domsch 2009-11-19 20:11:16 EST
There are lots of messages like this during the build:

selfserv_9669 starting at Wed Nov 18 14:40:05 CST 2009
trying to connect to selfserv_9669 at Wed Nov 18 14:40:05 CST 2009
retrying to connect to selfserv_9669 at Wed Nov 18 14:41:11 CST 2009
selfserv_9669 with PID 9142 found at Wed Nov 18 14:42:12 CST 2009
selfserv_9669 with PID 9142 started at Wed Nov 18 14:42:12 CST 2009
trying to kill selfserv_9669 with PID 9142 at Wed Nov 18 14:42:12 CST 2009
selfserv_9669 with PID 9142 killed at Wed Nov 18 14:42:12 CST 2009
selfserv_9669 starting at Wed Nov 18 14:42:12 CST 2009
trying to connect to selfserv_9669 at Wed Nov 18 14:42:12 CST 2009
retrying to connect to selfserv_9669 at Wed Nov 18 14:43:17 CST 2009


indicating that it is unable to set up the socket to listen on properly, fails to connect to that socket after a minute or two each time; repeat.  Eventually the build is killed after 6 hours of such failures.
Comment 10 Elio Maldonado Batiz 2009-11-20 13:22:55 EST
Re-tagged and made a new build which completed fine
http://koji.fedoraproject.org/koji/taskinfo?taskID=1820054

By the way, change log should have been release -17 not -14.1. I'll fix that in next build.
Comment 11 Elio Maldonado Batiz 2010-01-06 13:56:06 EST
The problem persists and it's reproducible in f13, f12, and f11. Similar builds done for RHEL 5 on RHEL build machines and via brew do not have this problem. 
RHEL builds done on my F-12 system also fail.

By tracing selvserv in the debugger with a client that repeatedly tries to connect to it and  I see that selvserv call 
PR_Accept(listen_sock, &addr, PR_INTERVAL_NO_TIMEOUT);
never returns. This NSPR method is a one-liner 
return((fd->methods->accept)(fd,addr,timeout));
that calls into libSSL accept callback 
ssl_Accept(PRFileDesc *fd, PRNetAddr *sockaddr, PRIntervalTime timeout)
which in turns calls NSPR's thread support
pt_Accept(PRFileDesc *fd, PRNetAddr *addr, PRIntervalTime timeout)
in mozilla/nsprpub/pr/src/pthreads/ptio.c.\

It never leaves this function as far as I can tell.

Th calls to osfd = accept(fd->secret->md.osfd, (struct sockaddr*)addr, &addr_len);
keep failing but the error handling code after the call never finds a reason to make it bail out and it continues in some sort of polling.
Comment 12 FTBFS 2010-02-13 22:36:10 EST
FTBFS fixed in nss-3.12.5-9.fc13.  Closing.
Comment 13 Elio Maldonado Batiz 2010-03-06 15:48:43 EST
The problem is back. Denis Gregorovic mentioned that he suspects it may be a Firewall configuration problem on the Fedora Koji build machines. This never happens with brew.
Comment 14 Elio Maldonado Batiz 2010-03-07 11:59:55 EST
Don't know why it failed that time. It's consistently working again. I built nss- 3.12.6 for F-14, F-13, F-12 and F-11 with all tests enabled. I'll close it.
Comment 15 Elio Maldonado Batiz 2010-08-14 13:47:07 EDT
The problems was never fixed. It was masked by an improper commenting out of lines in the spec file
#  For example, to disable the ssl test suites
#  you would uncomment the following lines
#%global nss_ssl_tests " "
#%global nss_ssl_run " "
The last two lines should have been
#%%global nss_ssl_tests " "
#%%global nss_ssl_run " "
Comment 16 Elio Maldonado Batiz 2010-08-14 13:50:55 EDT
The nightly Tinderbox upstream builds 
http://tinderbox.mozilla.org/showbuilds.cgi?tree=NSS
which are run on many platforms, but not on Fedora, don't have this problem.
Neither do similar build on RHEL-5.
Comment 17 Elio Maldonado Batiz 2010-08-14 14:01:15 EDT
Let's also look at recent RHEL-6 builds. 
https://brewweb.devel.redhat.com/buildinfo?buildID=134530

Examining a particulat log
http://download.devel.redhat.com/brewroot/packages/nss/3.12.6/3.el6/data/logs/x86_64/build.log

we find
+ cd ./mozilla/security/nss/tests/
+ HOST=localhost
+ DOMSUF=localdomain
+ PORT=9999
+ NSS_CYCLES=
+ NSS_TESTS=
+ NSS_SSL_TESTS=normal_fips
+ NSS_SSL_RUN='cov auth'
+ ./all.sh
At least a subset of the ssl tests were enabled and ran okay.
Comment 18 Elio Maldonado Batiz 2010-08-24 11:08:56 EDT
The problem has also been reported on RHEL-6
https://bugzilla.redhat.com/show_bug.cgi?id=589636

The ssl tests were conpleting fine for me on RHEL-6 a week or so ago.

Back to fedora F-13, I changed my /etc/hosts as follows:
[emaldona@localhost /]$ diff /etc/hosts.backup /etc/hosts
2c2
< ::1	localhost6.localdomain6	localhost6
---
> #::1	localhost6.localdomain6	localhost6
and the ssl tests completed. It's hardly a solution.
Comment 19 Bob Relyea 2010-08-24 15:03:41 EDT
In order for the NSS tests to complete, it needs to have DOMSUF set correctly. all.sh tries to discover what the appropriate domain suffix (DOMSUF) is from the host name, but that determination does not always work (depending on how hostname is set up on the local machine). You can usually get all.sh to run by manually setting DOMSUF or tweaking with your local machine configuration.

This problem can be fixed by 
 1) improving all.sh's ability to correctly choose DOMSUF, (preferred, but difficult).
 2) updating the configuration on the build machines. (yucky, but perhaps quick -- until the build machine environement changes again.).
 3) hand setting DOMSUF to whatever magic makes the build machines work. (not preferred as it makes the rpm very specific to our current build environment).

Clearly long term 1) is the preferred solution, 2 and 3 are usable if some build or compose is being held up.

bob
Comment 20 Elio Maldonado Batiz 2010-08-24 16:21:21 EDT
(In reply to comment #19)
> This problem can be fixed by 
>  1) improving all.sh's ability to correctly choose DOMSUF, (preferred, but
> difficult).
Desirable long term.

>  2) updating the configuration on the build machines. (yucky, but perhaps quick
> -- until the build machine environement changes again.).
Yucky indeed, unwise and likely unacceptable to the infrastrcture team.

>  3) hand setting DOMSUF to whatever magic makes the build machines work. (not
> preferred as it makes the rpm very specific to our current build environment).

We are actually doing that in nss.spec. (Added \'s for readabilty)
HOST=localhost DOMSUF=localdomain PORT=$MYRAND \
NSS_CYCLES=%{?nss_cycles} NSS_TESTS=%{?nss_tests} \
NSS_SSL_TESTS=%{?nss_ssl_tests} NSS_SSL_RUN=%{?nss_ssl_run}\
 ./all.sh

A scratch build with all tests enabled on RHEL-6 completed fine on all platforms. Another one for f13 at http://koji.fedoraproject.org/koji/taskinfo?taskID=2424201
seems okay so far. The problem goes away and comes back.

I wish we had RHEL-6 on the upstream Tinderbox runs.
Comment 21 Elio Maldonado Batiz 2010-08-24 16:44:57 EDT
Spoke to soon the problem still on f13.
Comment 22 Fedora Admin XMLRPC Client 2010-09-07 16:44:05 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 23 Elio Maldonado Batiz 2011-01-04 14:29:00 EST
A patch to the ssl test tools and scripts has been submitted upstream for review
https://bugzilla.mozilla.org/attachment.cgi?id=499383&action=diff
I have tested this with fedora scratch builds and it does solve the reported problems. Waiting for it to be accepted upstream before we can pick it up.
Comment 24 Elio Maldonado Batiz 2011-02-24 18:01:00 EST
A better short term fix has been proposed upstream and for use in Linux and Mac OS X builds and testing, see https://bugzilla.mozilla.org/show_bug.cgi?id=617723
Comment 25 Elio Maldonado Batiz 2011-02-24 18:03:46 EST
Created attachment 480879 [details]
Short-term fix for ssl test suites hangs on ipv6 type connections

Change selfserv to use a dual-stack IPv6 listening socket, which can accept connections from both IPv4 and IPv6 clients. NSPR's IPv6 sockets have the IPV6_V6ONLY socket option default to false.
Comment 26 Fedora Update System 2011-02-24 19:23:34 EST
nss-softokn-3.12.9-6.fc15,nss-3.12.9-13.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/nss-softokn-3.12.9-6.fc15,nss-3.12.9-13.fc15
Comment 27 Fedora Update System 2011-02-25 11:13:15 EST
nss-softokn-3.12.9-4.fc14,nss-3.12.9-8.fc14 has been submitted as an update for Fedora 14.
https://admin.fedoraproject.org/updates/nss-softokn-3.12.9-4.fc14,nss-3.12.9-8.fc14
Comment 28 Fedora Update System 2011-02-25 18:18:27 EST
nss-softokn-3.12.9-6.fc15, nss-3.12.9-13.fc15 has been pushed to the Fedora 15 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update nss-softokn nss'.  You can provide feedback for this update here: https://admin.fedoraproject.org/updates/nss-softokn-3.12.9-6.fc15,nss-3.12.9-13.fc15
Comment 29 Fedora Update System 2011-02-27 16:34:24 EST
Package nss-3.12.9-8.fc13,nss-softokn-3.12.9-5.fc13,nss-util-3.12.9-1.fc13,nspr-4.8.7-1.fc13:
* should fix your issue,
* was pushed to the Fedora 13 updates-testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing nss-3.12.9-8.fc13,nss-softokn-3.12.9-5.fc13,nss-util-3.12.9-1.fc13,nspr-4.8.7-1.fc13'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/nss-3.12.9-8.fc13,nss-softokn-3.12.9-5.fc13,nss-util-3.12.9-1.fc13,nspr-4.8.7-1.fc13
then log in and leave karma (feedback).
Comment 30 Fedora Update System 2011-02-28 23:22:34 EST
nss-softokn-3.12.9-5.fc14, nss-3.12.9-8.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 31 Fedora Update System 2011-03-02 22:22:16 EST
nss-softokn-3.12.9-7.fc15, nss-3.12.9-13.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 32 Fedora Update System 2011-03-07 15:55:17 EST
nss-3.12.9-8.fc13, nss-softokn-3.12.9-5.fc13, nss-util-3.12.9-1.fc13, nspr-4.8.7-1.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.