Bug 1436580

+++ This bug was initially created as a clone of Bug #1436576 +++

Description of problem:
The failing test is "test_add_list_remove_router_on_l3_agent" in neutron/tests/tempest/api/admin/test_l3_agent_scheduler.py

The test suite's setUp creates a router and if it's a dvr router (which it is in this case) it also uses 'router-gateway-set' to ensure it's scheduled. Then, the test itself tries to schedule the router to an agent. This will fail if the agent that was scheduled to during setUp is different than the one that's attempted to be scheduled to during the test itself.

In other words, the test is faulty in that it should know that the router is already scheduled.

The failure happens in Mitaka and Neuton version when running on DVR setup
Stacktrace

Traceback (most recent call last):
testtools.testresult.real._StringException: Empty attachments:
  stderr
  stdout

pythonlogging:'': {{{
2017-03-24 21:34:26,554 7724 INFO     [tempest.lib.common.rest_client] Request (L3AgentSchedulerTestJSON:test_add_list_remove_router_on_l3_agent): 409 POST http://10.0.0.107:9696/v2.0/agents/397fef47-1243-485e-9189-38842e0b2d77/l3-routers 0.353s
2017-03-24 21:34:26,555 7724 DEBUG    [tempest.lib.common.rest_client] Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: {"router_id": "3c6003e9-d81e-4287-8a77-603f22b3b27b"}
    Response - Headers: {'status': '409', u'content-length': '205', 'content-location': 'http://10.0.0.107:9696/v2.0/agents/397fef47-1243-485e-9189-38842e0b2d77/l3-routers', u'date': 'Sat, 25 Mar 2017 01:34:27 GMT', u'content-type': 'application/json', u'connection': 'close', u'x-openstack-request-id': 'req-0f7b896c-7a3a-4c9e-9155-c2faa610d5ff'}
        Body: {"NeutronError": {"message": "The router 3c6003e9-d81e-4287-8a77-603f22b3b27b has been already hosted by the L3 Agent 9e974922-85c2-457c-a791-8fa4c97cf25e.", "type": "RouterHostedByL3Agent", "detail": ""}}
}}}

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/neutron/tests/tempest/api/admin/test_l3_agent_scheduler.py", line 91, in test_add_list_remove_router_on_l3_agent
    self.router['id'])
  File "/usr/lib/python2.7/site-packages/neutron/tests/tempest/services/network/json/network_client.py", line 471, in add_router_to_l3_agent
    resp, body = self.post(uri, body)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 276, in post
    return self.request('POST', url, extra_headers, headers, body, chunked)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 665, in request
    self._error_checker(resp, resp_body)
  File "/usr/lib/python2.7/site-packages/tempest/lib/common/rest_client.py", line 777, in _error_checker
    raise exceptions.Conflict(resp_body, resp=resp)
tempest.lib.exceptions.Conflict: An object with that identifier already exists
Details: {u'message': u'The router 3c6003e9-d81e-4287-8a77-603f22b3b27b has been already hosted by the L3 Agent 9e974922-85c2-457c-a791-8fa4c97cf25e.', u'type': u'RouterHostedByL3Agent', u'detail': u''}


Version-Release number of selected component (if applicable):
python-neutron-10.0.0-8.el7ost.noarch
openstack-neutron-10.0.0-8.el7ost.noarch
python-neutronclient-6.1.0-1.el7ost.noarch
puppet-neutron-10.3.0-1.el7ost.noarch
python-neutron-lib-1.1.0-1.el7ost.noarch
How reproducible:
 Failed 4 times in the last 7 runs. Stability: 42 %

Steps to Reproduce:
1.run ci gate: https://rhos-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/RHOS/view/RHOS11/job/qe-DFG-neutron-11_director-rhel-7.3-virthost-3cont_2comp-ipv4-vxlan-lvm-lbaas-dvr
2.
3.

Actual results:


Expected results:


Additional info: