Description of problem: In the previous cycle a change added to tempest which allows to use rhel7 image instead of very small test only image. Unfortunately the change is not compatible with RHEL8, first of all, the 'serivce network restart' Does not exists. The change for RHEL7 in tempest: https://github.com/openstack/tempest/commit/c5c7ba997e tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet.* tests are failing. Additional info: RHEL guest image does not have the legacy part of the initscripts, network-scripts was splinted out from initscripts (package build time) to a dedicated package since Jun 14 2018 9.83-1 .
For the record steps how to reproduce: generate tempest.conf using rhel8 $ discover-tempest-config --create --debug --image <path/to/rhel-8> then use the generated tempest.conf to run tempest (either packaged or cloned from master): $ tempest run --regex tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet
Finally I got back to this BZ, steps how to reproduce: $ source ~/overcloudrc && openstack flavor create --id 200 --ram 1024 --disk 10 --vcpus 1 rhel $ source ~/overcloudrc && openstack flavor create --id 201 --ram 2048 --disk 11 --vcpus 2 rhel_alt $ discover-tempest-config --debug --create --image <path/to/rhel8> --deployer-input ../ir-tempest-deployer-input.conf compute.flavor_ref 200 compute.flavor_ref_alt 201 compute.image_ssh_user cloud-user validation.image_ssh_user cloud-user scenario.dhcp_client dhclient volume.volume_size 12 I executed one of the tests from tempest/scenario/test_network_v6.py: $ tempest run --regex tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_dhcp6_stateless_from_os <output omitted> 2019-04-26 09:21:50,876 89245 ERROR [tempest.lib.common.utils.linux.remote_client] (TestGettingAddress:test_dualnet_dhcp6_stateless_from_os) Executing command on 10.0.0.214 failed. Error: Command 'set -eu -o pipefail; PATH=$PATH:/sbin; echo -e "DEVICE=eth1\nIPV6INIT=yes" | sudo tee /etc/sysconfig/network-scripts/ifcfg-eth1; sudo /sbin/service network restart', exit status: 5, stderr: Redirecting to /bin/systemctl restart network.service Failed to restart network.service: Unit network.service not found. stdout: DEVICE=eth1 IPV6INIT=yes 2019-04-26 09:21:50,876 89245 DEBUG [tempest.scenario.test_network_v6] Error during restarting eth1 interface on instance. Error message: Command 'set -eu -o pipefail; PATH=$PATH:/sbin; echo -e "DEVICE=eth1\nIPV6INIT=yes" | sudo tee /etc/sysconfig/network-scripts/ifcfg-eth1; sudo /sbin/service network restart', exit status: 5, stderr: Redirecting to /bin/systemctl restart network.service Failed to restart network.service: Unit network.service not found. <output omitted> So the first problem is that network can't be restarted on RHEL-8 the way as it used to be on RHEL-7 and older, https://access.redhat.com/discussions/3791081 Therefore the test should either restart the network a unified way (it works on RHEL-8 and the older versions as well) or the test should check what kind of image is used. I like more the first idea, I'm gonna have a look.
[1] in the test can be changed for 'nmcli connection reload', which will work also for RHEL8 and anyway the tests should not restart the whole network service at all. The second problem with the tests is that RHEL8 doesn't calculate ipv6 addresses based on the mac address of the interface anymore, therefore, the tests can't assume what ipv6 address will be given to the interface (ips_from_api_* variable set by prepare_server method [2]). We need to modify the tests and instead of assuming what address will be given to the interface, the test should rather check if any ipv6 address was given to it and maybe ping it. [1] https://github.com/openstack/tempest/blob/c248f5962ec4abfd716822c3b3ba496fe0e9cfbb/tempest/scenario/test_network_v6.py#L171 [2] https://github.com/openstack/tempest/blob/c248f5962ec4abfd716822c3b3ba496fe0e9cfbb/tempest/scenario/test_network_v6.py#L192-L193
$ openstack server create --image <rhel8> --flavor <flavor> --network <network1> --network <network2> --key-name <key_name> instance_name +-------------------------------------+--------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------------------+--------------------------------------------------------------------------------------------------------+ <output omitted> | accessIPv4 | | | accessIPv6 | | | addresses | tempest-network-smoke--884367252=10.100.0.5; tempest-network-smoke--18828977=2003::f816:3eff:febb:7456 | <output omitted> The test expects that eth1 interface of the instance will have ipv6 address equal to 2003::f816:3eff:febb:7456. This address is returned by 'openstack server create' command and I guess it's calculated based on the mac address of eth1 interface which is fa:16:3e:bb:74:56, however, when I ssh to the instance, I see the following output of 'ip a' command: 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc fq_codel state UP group default qlen 1000 link/ether fa:16:3e:48:e8:b5 brd ff:ff:ff:ff:ff:ff inet 10.100.0.3/28 brd 10.100.0.15 scope global dynamic noprefixroute eth0 valid_lft 86363sec preferred_lft 86363sec inet6 fe80::f816:3eff:fe48:e8b5/64 scope link valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc fq_codel state UP group default qlen 1000 link/ether fa:16:3e:bb:74:56 brd ff:ff:ff:ff:ff:ff inet6 2003::b47f:f400:ecca:2a55/64 scope global dynamic noprefixroute valid_lft 86385sec preferred_lft 14385sec inet6 fe80::7615:8d57:775d:fae/64 scope link noprefixroute valid_lft forever preferred_lft forever Notice that eth1 interface has an ipv6 address which seems not to be derived from its mac address. If the calculation of ipv6 addresses has changed in rhel8, 'openstack server create' should know about it and should return the proper address.
(In reply to Martin Kopec from comment #6) > [1] in the test can be changed for 'nmcli connection reload', which will > work also for RHEL8 and anyway the tests should not restart the whole > network service at all. But if you use systemctl restart network, shouldn't that work also on the other supported distributions and avoid a distribution-specific branch?
(In reply to Luigi Toscano from comment #8) > (In reply to Martin Kopec from comment #6) > > [1] in the test can be changed for 'nmcli connection reload', which will > > work also for RHEL8 and anyway the tests should not restart the whole > > network service at all. > > But if you use systemctl restart network, shouldn't that work also on the > other supported distributions and avoid a distribution-specific branch? Based on the comments in this [1] commit, the distribution-specific branch was created because a config file in a certain location needs to be changed. The restart command is just a part of the process of changing the config file. The problem which this BZ describes is that RHEL8 doesn't have network service as it used to have in the previous version [2] so restarting of the network service will not work there (either via systemctl or service command). [1] https://github.com/openstack/tempest/commit/c5c7ba997e [2] https://access.redhat.com/discussions/3791081
Created attachment 1580192 [details] verification output The fixed in version package openstack-tempest-19.0.0-4.el7ost contains a fix for the issue and it's available in RHOS-14 repository via latest symlink.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:1671