Bug 1450677 - Fail to execute neutron functional metadata_proxy tests
Summary: Fail to execute neutron functional metadata_proxy tests
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron
Version: 12.0 (Pike)
Hardware: Unspecified
OS: Unspecified
Target Milestone: Upstream M2
: 12.0 (Pike)
Assignee: Daniel Alvarez Sanchez
QA Contact: Toni Freger
Depends On:
TreeView+ depends on / blocked
Reported: 2017-05-14 14:53 UTC by Arie Bregman
Modified: 2018-02-05 19:07 UTC (History)
9 users (show)

Fixed In Version: openstack-neutron-11.0.0-0.20170521040619.3f2e22a.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2017-12-13 21:27:05 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
RDO 6708 0 None None None 2017-05-19 14:28:48 UTC
Red Hat Product Errata RHEA-2017:3462 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 12.0 Enhancement Advisory 2018-02-16 01:43:25 UTC

Description Arie Bregman 2017-05-14 14:53:19 UTC
Description of problem:

Neutron functional metadata_proxy tests fail:



Traceback (most recent call last):
  File "neutron/tests/base.py", line 115, in func
    return f(self, *args, **kwargs)
  File "neutron/tests/base.py", line 115, in func
    return f(self, *args, **kwargs)
  File "neutron/tests/functional/agent/l3/test_metadata_proxy.py", line 174, in test_haproxy_migration_path
    self.assertIn('haproxy', pm.cmdline)
  File "/usr/lib/python2.7/site-packages/testtools/testcase.py", line 356, in assertIn
    self.assertThat(haystack, Contains(needle), message)
  File "/usr/lib/python2.7/site-packages/testtools/testcase.py", line 435, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 'haproxy' not in None




Traceback (most recent call last):
  File "neutron/tests/base.py", line 115, in func
    return f(self, *args, **kwargs)
  File "neutron/tests/base.py", line 115, in func
    return f(self, *args, **kwargs)
  File "neutron/tests/functional/agent/l3/test_metadata_proxy.py", line 120, in test_access_to_metadata_proxy
    firstline = self._query_metadata_proxy(machine)
  File "neutron/tests/functional/agent/l3/test_metadata_proxy.py", line 85, in _query_metadata_proxy
    self.fail('Timed out waiting metadata proxy to become available')
  File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 666, in fail
    raise self.failureException(msg)
AssertionError: Timed out waiting metadata proxy to become available

Version-Release number of selected component (if applicable): openstack-neutron-11.0.0-0.20170503012423.617fc60

How reproducible: 100%

Steps to Reproduce:
1. Install openstack-neutron
2. Run dsvm-functional tests

Actual results: The above failures

Expected results: Tests passed successfully

Comment 2 Red Hat Bugzilla Rules Engine 2017-05-14 14:56:58 UTC
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.

Comment 3 Assaf Muller 2017-05-15 10:36:04 UTC
Is 'haproxy' dependency not installed in functional testing? It is a dep in the spec file.

Comment 5 Ihar Hrachyshka 2017-05-18 16:35:43 UTC
In test log, we see:

InvalidUserOrGroupException: Invalid user/uid: '65534'

Apparently the nobody user is not there, or it does have a different number.

Comment 6 Daniel Alvarez Sanchez 2017-05-18 17:00:19 UTC
Current neutron functional tests assumes that there's a user with uid 65534 [0].
This, apparently, is always true upstream for the ubuntu xenial deployment in the gate. However, it's not in RHEL as the uuid 65534 is created by package nfs-utils which is not installed by default in our CI [1]:

$ rpm -q --scripts nfs-utils 
# Create nfsnobody uid as long as it does not already exist.
cat /etc/passwd | cut -d':' -f 1 | grep --quiet nfsnobody 2>/dev/null
if [ "$?" -eq 1 ]; then
    /usr/sbin/useradd -l -c "Anonymous NFS User" -r -g 65534 \
                -s /sbin/nologin -u 65534 -d /var/lib/nfs nfsnobody >/dev/null 2>&1 || :
   /usr/sbin/usermod -u 65534 -g 65534 nfsnobody >/dev/null 2>&1 || :

We should modify the functional tests so that it doesn't make that assumption.
I'll open a bug in launchpad and try to come up with a fix.

[0] https://github.com/openstack/neutron/blob/stable/ocata/neutron/tests/functional/agent/l3/test_metadata_proxy.py#L133
[1] https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/network/view/neutron/job/DFG-network-neutron-dsvm-functional-rhos/240/artifact/infrared/9795f228d997b171ef31983a4c7bf782-tester-0/home/cloud-user/logs/installed-rpms.log

Comment 7 Daniel Alvarez Sanchez 2017-05-19 07:58:57 UTC
Apparently, nfs-utils package gets installed as a dependency for libvirt:

$ repoquery --requires --recursive --resolve libvirt | grep nfs-utils

Once nfs-utils is installed it creates the nfsnobody user with uid 65534 that allows functional tests to pass. In our CI we're not installing libvirt for functional tests [0] and since it's not a dependency either I guess we shouldn't relay on this uid to exist on the system for the tests.

[0] https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/network/view/neutron/job/DFG-network-neutron-dsvm-functional-rhos/240/artifact/infrared/9795f228d997b171ef31983a4c7bf782-tester-0/home/cloud-user/logs/installed-rpms.log

Comment 8 Daniel Alvarez Sanchez 2017-05-23 13:35:05 UTC
nfs-utils has been added as a requirement to python-neutron-tests [0] and it's already in RDO promotion that happened today. It should be auto-imported to OSP in a few days. I'll update the fields accordingly.

[0] https://review.rdoproject.org/r/#/c/6708/

Comment 9 Ihar Hrachyshka 2017-05-26 18:19:10 UTC
The fix landed in RDO, we are waiting for OSP sync to happen to close the bug.

Comment 15 Assaf Muller 2017-07-14 19:52:52 UTC
Verified per comment 12.

Comment 18 errata-xmlrpc 2017-12-13 21:27:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.