RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1964342 - Using nmstate inside a pod prevents any OVS operation
Summary: Using nmstate inside a pod prevents any OVS operation
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: nmstate
Version: 8.5
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: beta
: ---
Assignee: Gris Ge
QA Contact: Mingyu Shi
URL:
Whiteboard:
Depends On:
Blocks: 1964439
TreeView+ depends on / blocked
 
Reported: 2021-05-25 09:20 UTC by Quique Llorente
Modified: 2023-12-26 09:42 UTC (History)
5 users (show)

Fixed In Version: nmstate-1.1.0-0.1.el8
Doc Type: No Doc Update
Doc Text:
Clone Of:
: 1964439 (view as bug list)
Environment:
Last Closed: 2021-11-09 17:43:51 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github nmstate nmstate pull 1583 0 None closed ovs: Fix `is_ovs_running()` in container environment. 2021-05-25 13:27:34 UTC
Red Hat Issue Tracker NMT-1012 0 None None None 2023-12-26 09:42:49 UTC
Red Hat Product Errata RHBA-2021:4157 0 None None None 2021-11-09 17:44:23 UTC

Description Quique Llorente 2021-05-25 09:20:45 UTC 
Description of problem:
When nmstate is used inside a pod delegating the OVS operation to the node trying to apply some desired state with OVS involved fail since nmstate try to run "systemctl status ovs" but systemctl is not working at those containers.


Version-Release number of selected component (if applicable):


How reproducible:

Steps to Reproduce:
1. git clone https://github.com/nmstate/nmstate
2. git reset --hard d67ba7f6b7c3036ceeca62545c95a1b10bdea513~1 # Before the fix
1. ./automation/run-tests.sh --k8s --centos-stream --test-type=integ_tier1 --pytest-args="-k ovs"


Actual results:
=================================== FAILURES ===================================
___________ test_create_and_remove_ovs_bridge_with_min_desired_state ___________

    @pytest.mark.tier1
    def test_create_and_remove_ovs_bridge_with_min_desired_state():
>       with Bridge(BRIDGE1).create() as state:

tests/integration/ovs_test.py:96: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib64/python3.6/contextlib.py:81: in __enter__
    return next(self.gen)
tests/integration/testlib/ovslib.py:126: in create
    libnmstate.apply(desired_state)
/usr/lib/python3.6/site-packages/libnmstate/netapplier.py:74: in apply
    desired_state, plugins_capabilities(plugins)
/usr/lib/python3.6/site-packages/libnmstate/validator.py:46: in validate_capabilities
    validate_interface_capabilities(state.get(Interface.KEY, []), capabilities)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

ifaces_state = [{'bridge': {}, 'name': 'br1', 'state': 'up', 'type': 'ovs-bridge'}]
capabilities = ['team']

    def validate_interface_capabilities(ifaces_state, capabilities):
        ifaces_types = {iface_state.get("type") for iface_state in ifaces_state}
        has_ovs_capability = NmstatePlugin.OVS_CAPABILITY in capabilities
        has_team_capability = NmstatePlugin.TEAM_CAPABILITY in capabilities
        ovs_is_running = is_ovs_running()
        for iface_type in ifaces_types:
            is_ovs_type = iface_type in (
                InterfaceType.OVS_BRIDGE,
                InterfaceType.OVS_INTERFACE,
                InterfaceType.OVS_PORT,
            )
            if is_ovs_type and not has_ovs_capability:
                if not ovs_is_running:
                    raise NmstateDependencyError(
>                       "openvswitch service is not started."
                    )
E                   libnmstate.error.NmstateDependencyError: openvswitch service is not started.


Expected results:
Test passing


Additional info:
Comment 7 Mingyu Shi 2021-06-20 05:05:00 UTC 
Verified with versions:
nmstate-1.1.0-0.3.alpha3.el8.noarch
nispor-1.1.0-2.el8.x86_64
NetworkManager-1.32.0-0.5.el8.x86_64
openvswitch2.13-2.13.0-105.el8fdp.x86_64

1. Configuring ovs inside a container works.

2. If OVS is unusable(for example, stopping the openvswitch daemon in the host), nmstate will raise an error when trying to update anything of ovs:
  File "/usr/lib/python3.6/site-packages/libnmstate/nm/context.py", line 213, in wait_all_finish
    raise tmp_error
libnmstate.error.NmstateDependencyError: Activate profile uuid:ae19ff6a-2c17-4e0e-a021-3e2f16e3390e iface:ovsbr1 type: ovs-interface failed: failed to communicating with Open vSwitch database, <enum NM_DEVICE_STATE_REASON_OVSDB_FAILED of type NM.DeviceStateReason>

3. Even though openvswitch daemon stopped, the existed ovs interfaces(created by nmstate or NM) still can be shown:
# nmstatectl show ovsbr0
Unhandled IFLA_INFO_DATA for iface type Other("Ipoib")
Unhandled IFLA_INFO_DATA for iface type Other("Ipoib")
2021-06-20 04:55:46,358 root         DEBUG    NetworkManager version 1.31.90
2021-06-20 04:55:46,359 root         DEBUG    Async action: Retrieve applied config: ethernet eno1 started
2021-06-20 04:55:46,359 root         DEBUG    Async action: Retrieve applied config: ovs-port ovs-port-ovsbr0 started
2021-06-20 04:55:46,360 root         DEBUG    Async action: Retrieve applied config: ovs-interface ovsbr0 started
2021-06-20 04:55:46,360 root         DEBUG    Async action: Retrieve applied config: ovs-bridge ovsbr0 started
2021-06-20 04:55:46,362 root         DEBUG    Async action: Retrieve applied config: ethernet eno1 finished
2021-06-20 04:55:46,363 root         DEBUG    Async action: Retrieve applied config: ovs-port ovs-port-ovsbr0 finished
2021-06-20 04:55:46,363 root         DEBUG    Async action: Retrieve applied config: ovs-interface ovsbr0 finished
2021-06-20 04:55:46,364 root         DEBUG    Async action: Retrieve applied config: ovs-bridge ovsbr0 finished
2021-06-20 04:55:46,368 root         DEBUG    Interface ovs-interface.br0 found. Merging the interface information.
2021-06-20 04:55:46,368 root         DEBUG    Interface ethernet.eno1 found. Merging the interface information.
2021-06-20 04:55:46,368 root         DEBUG    Interface ethernet.eno2 found. Merging the interface information.
2021-06-20 04:55:46,368 root         DEBUG    Interface ethernet.eno3 found. Merging the interface information.
2021-06-20 04:55:46,368 root         DEBUG    Interface ethernet.eno4 found. Merging the interface information.
2021-06-20 04:55:46,368 root         DEBUG    Interface ethernet.ens1f0 found. Merging the interface information.
2021-06-20 04:55:46,368 root         DEBUG    Interface ethernet.ens1f1 found. Merging the interface information.
2021-06-20 04:55:46,368 root         DEBUG    Interface ovs-interface.ovsbr0 found. Merging the interface information.
Unhandled IFLA_INFO_DATA for iface type Other("Ipoib")
Unhandled IFLA_INFO_DATA for iface type Other("Ipoib")
Unhandled IFLA_INFO_DATA for iface type Other("Ipoib")
Unhandled IFLA_INFO_DATA for iface type Other("Ipoib")
---
dns-resolver:
  config: {}
  running:
    search:
    - rhts.eng.pek2.redhat.com
    server:
    - 10.73.2.107
    - 10.73.2.108
    - 10.66.127.10
route-rules:
  config: []
routes:
  config: []
  running: []
interfaces:
- name: ovsbr0
  type: ovs-interface
  state: up
  accept-all-mac-addresses: false
  ethtool:
    feature:
      highdma: true
      rx-gro: true
      rx-gro-list: false
      tx-checksum-ip-generic: true
      tx-generic-segmentation: true
      tx-gre-csum-segmentation: true
      tx-gre-segmentation: true
      tx-ipxip4-segmentation: true
      tx-ipxip6-segmentation: true
      tx-nocache-copy: false
      tx-scatter-gather-fraglist: true
      tx-sctp-segmentation: true
      tx-tcp-ecn-segmentation: true
      tx-tcp-mangleid-segmentation: true
      tx-tcp-segmentation: true
      tx-tcp6-segmentation: true
      tx-udp_tnl-csum-segmentation: true
      tx-udp_tnl-segmentation: true
      tx-vlan-hw-insert: true
      tx-vlan-stag-hw-insert: true
  ipv4:
    enabled: false
    address: []
    dhcp: false
  ipv6:
    enabled: false
    address: []
    autoconf: false
    dhcp: false
  lldp:
    enabled: false
  mac-address: 82:51:53:76:59:49
  mtu: 1500
  ovs-db:
    external_ids: {}
- name: ovsbr0
  type: ovs-bridge
  state: up
  bridge:
    options:
      fail-mode: ''
      mcast-snooping-enable: false
      rstp: false
      stp: false
    port:
    - name: ovsbr0
  lldp:
    enabled: false
  ovs-db:
    external_ids: {}
Comment 9 errata-xmlrpc 2021-11-09 17:43:51 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (nmstate bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4157
Note You need to log in before you can comment on or make changes to this bug.