RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1402001 - messages about skipped tests when running upstream test suite
Summary: messages about skipped tests when running upstream test suite
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: nss
Version: 7.3
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: rc
: ---
Assignee: Daiki Ueno
QA Contact: Hubert Kario
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-12-06 14:43 UTC by Hubert Kario
Modified: 2018-04-10 09:25 UTC (History)
2 users (show)

Fixed In Version: nss-3.34.0-0.1.beta1.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-10 09:23:57 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Mozilla Foundation 1395588 0 None None None 2017-08-31 14:54:11 UTC
Red Hat Product Errata RHEA-2018:0679 0 None None None 2018-04-10 09:25:17 UTC

Description Hubert Kario 2016-12-06 14:43:35 UTC 
Description of problem:
When the NSS package is rebuilt, and upstream test suite is ran, the log includes mentions of skipped test cases and stuff that looks like misinterpreted data

Version-Release number of selected component (if applicable):
nss-3.21.0-17.el7.x86_64

How reproducible:
always

Steps to Reproduce:
1. rpmbuild nss
2. inspect output

Actual results:
selfserv_9853 -b -p 9853 2>/dev/null;
selfserv_9853 with PID 13333 killed at Mon Dec  5 22:04:11 EST 2016
ssl.sh: SSL Cert Status (OCSP Stapling)  - server normal/client fips  - with ECC ===============================
ssl.sh: skipping  OCSP stapling, signed response, good status (non-FIPS only)
ssl.sh: skipping  OCSP stapling, signed response, revoked status (non-FIPS only)
ssl.sh: skipping  OCSP stapling, signed response, unknown status (non-FIPS only)
ssl.sh: skipping  OCSP stapling, unsigned failure response (non-FIPS only)
ssl.sh: skipping  OCSP stapling, good status, bad signature (non-FIPS only)
ssl.sh: skipping  OCSP stapling, invalid cert status data (non-FIPS only)
ssl.sh: skipping  Valid cert, Server doesn't staple (non-FIPS only)
ssl.sh: skipping  Stress OCSP stapling, server uses random status (non-FIPS only)
ssl.sh: SSL Stress Test  - server normal/client fips  - with ECC ===============================
ssl.sh: skipping  Form is subject to the terms of the Mozilla Public (non-FIPS only)
ssl.sh: skipping  If a copy of the MPL was not distributed with this (non-FIPS only)
ssl.sh: skipping  obtain one at http://mozilla.org/MPL/2.0/. (non-FIPS only)
ssl.sh: skipping   (non-FIPS only)
ssl.sh: skipping  the stress tests for SSL/TLS. (non-FIPS only)
ssl.sh: skipping   (non-FIPS only)
ssl.sh: skipping   (non-FIPS only)
ssl.sh: skipping  client                         Test Case name (non-FIPS only)
ssl.sh: skipping  params (non-FIPS only)
ssl.sh: skipping  ------                         --------------- (non-FIPS only)
ssl.sh: skipping  -c_1000_-C_A                  Stress SSL2 RC4 128 with MD5 (non-FIPS only)
ssl.sh: skipping  Stress SSL3 RC4 128 with MD5 (non-FIPS only)
ssl.sh: skipping  Stress TLS  RC4 128 with MD5 (non-FIPS only)
ssl.sh: skipping  Stress TLS  RC4 128 with MD5 (false start) (non-FIPS only)
ssl.sh: skipping  Stress TLS  RC4 128 with MD5 (session ticket) (non-FIPS only)
ssl.sh: skipping  Stress TLS  RC4 128 with MD5 (compression) (non-FIPS only)
ssl.sh: skipping  Stress TLS  RC4 128 with MD5 (session ticket, compression) (non-FIPS only)
ssl.sh: skipping  Stress TLS  RC4 128 with MD5 (session ticket, compression, false start) (non-FIPS only)
ssl.sh: skipping  Stress TLS RC4 128 with MD5 (session ticket, SNI) (non-FIPS only)
ssl.sh: skipping   (non-FIPS only)
ssl.sh: skipping  versions here... (non-FIPS only)
ssl.sh: skipping   (non-FIPS only)


ssl.sh: skipping   (non-FIPS only)
ssl.sh: skipping  ############################ (non-FIPS only)
ssl.sh: skipping   (non-FIPS only)
ssl.sh: skipping  Stress TLS DHE_RSA_WITH_3DES_EDE_CBC_SHA (no reuse) (non-FIPS only)
ssl.sh: skipping  Stress TLS DHE_RSA_WITH_AES_128_CBC_SHA (non-FIPS only)
ssl.sh: skipping  Stress TLS DHE_RSA_WITH_AES_256_CBC_SHA (no reuse) (non-FIPS only)
ssl.sh: skipping  Stress TLS DHE_DSS_WITH_AES_128_CBC_SHA256 (no reuse) (non-FIPS only)
ssl.sh: skipping  -V_ssl3:_-c_1000_-C_:0038_-u Stress TLS DHE_DSS_WITH_AES_256_CBC_SHA (session ticket) (non-FIPS only)
ssl.sh: skipping  session ticket test, once session tickets with DHE_DSS are working (non-FIPS only)
ssl.sh: skipping  Stress TLS DHE_DSS_WITH_AES_256_CBC_SHA (no reuse) (non-FIPS only)
ssl.sh: skipping  -V_ssl3:_-c_1000_-C_:006A    Stress TLS DHE_DSS_WITH_AES_256_CBC_SHA256 (non-FIPS only)
ssl.sh: skipping  reuse test, once the session cache with DHE_DSS is working (non-FIPS only)
ssl.sh: skipping  Stress TLS DHE_DSS_WITH_AES_256_CBC_SHA256 (no reuse (non-FIPS only)
ssl.sh: skipping  Stress TLS DHE_RSA_WITH_AES_256_CBC_SHA256 (no reuse) (non-FIPS only)
ssl.sh: skipping  Stress TLS DHE_RSA_WITH_AES_128_GCM_SHA256 (no reuse) (non-FIPS only)
ssl.sh: skipping   (non-FIPS only)
ssl.sh: skipping  versions here... (non-FIPS only)
ssl.sh: skipping   (non-FIPS only)


selfserv_9853 -b -p 9853 2>/dev/null;
selfserv_9853 with PID 18315 killed at Mon Dec  5 22:04:44 EST 2016
ssl.sh: SSL Client Authentication Extended Test - server normal/client fips  - with ECC ===============================
ssl.sh: skipping  TLS Request don't require client auth (client does not provide auth) (non-FIPS only)
ssl.sh: skipping  TLS Request don't require client auth (bad password) (non-FIPS only)
ssl.sh: skipping  TLS Request don't require client auth (client auth) (non-FIPS only)
ssl.sh: TLS Require client auth (client does not provide auth) ----
selfserv_9853 starting at Mon Dec  5 22:04:44 EST 2016

Expected results:
no skipped tests that test major functionality

Additional info:
Comment 1 Kai Engert (:kaie) (inactive account) 2017-02-24 14:11:51 UTC 
Sounds like a request to fix NSS tests as part of the RPM packing.

Does this also happen when the test is executed outside the scope of an RPM build? If yes, we need an upstream bug.

Could this potentially be a deliberate decision of the upstream test suite, e.g. related to its multiple cycles, and when some tests aren't appropriate?

Does this still happen with the newer 3.28.x package?

Is this a regression (from packages older than 3.21.x)?
Comment 2 Hubert Kario 2017-02-24 17:50:03 UTC 
(In reply to Kai Engert (:kaie) from comment #1)
> Sounds like a request to fix NSS tests as part of the RPM packing.

well, it is a RHEL bug... :)
 
> Does this also happen when the test is executed outside the scope of an RPM
> build? If yes, we need an upstream bug.

haven't tested
 
> Could this potentially be a deliberate decision of the upstream test suite,
> e.g. related to its multiple cycles, and when some tests aren't appropriate?

maybe, but the formatting of the errors doesn't fell like this to me

> Does this still happen with the newer 3.28.x package?

not tested

> Is this a regression (from packages older than 3.21.x)?

I would have to double check, but I did in the past go through full test script results and didn't notice them before, so yes, I'd bet on "it's a regression"
Comment 4 Kai Engert (:kaie) (inactive account) 2017-06-01 09:01:45 UTC 
I suggest that as a first step for this bug, it should be evaluated, how much our logs differ from the upstream logs.

Does upstream skip the same messages? Then this needs to be fixed upstream first.
Comment 11 errata-xmlrpc 2018-04-10 09:23:57 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:0679
Note You need to log in before you can comment on or make changes to this bug.