Bug 2111586
| Summary: | Export OVS metrics | |||
|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Tim Rozet <trozet> | |
| Component: | Networking | Assignee: | Tim Rozet <trozet> | |
| Networking sub component: | ovn-kubernetes | QA Contact: | Weibin Liang <weliang> | |
| Status: | CLOSED ERRATA | Docs Contact: | ||
| Severity: | high | |||
| Priority: | unspecified | CC: | amorenoz, anusaxen, mkennell, zzhao | |
| Version: | 4.7 | |||
| Target Milestone: | --- | |||
| Target Release: | 4.12.0 | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 2111587 (view as bug list) | Environment: | ||
| Last Closed: | 2023-01-17 19:53:34 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 2111587 | |||
|
Description
Tim Rozet
2022-07-27 14:52:22 UTC
Tested and verified in 4.12.0-0.nightly-2022-08-12-053438
[weliang@weliang ~]$ oc exec ovnkube-master-7bb7r -- curl 127.0.0.1:29105/metrics | grep -i "ovs_db\|ovs_vswitch"
Defaulted container "northd" out of: northd, nbdb, kube-rbac-proxy, sbdb, ovnkube-master, ovn-dbchecker
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0# HELP ovs_db_process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE ovs_db_process_cpu_seconds_total counter
ovs_db_process_cpu_seconds_total 2.48
# HELP ovs_db_process_max_fds Maximum number of open file descriptors.
# TYPE ovs_db_process_max_fds gauge
ovs_db_process_max_fds 1024
# HELP ovs_db_process_open_fds Number of open file descriptors.
# TYPE ovs_db_process_open_fds gauge
ovs_db_process_open_fds 27
# HELP ovs_db_process_resident_memory_bytes Resident memory size in bytes.
# TYPE ovs_db_process_resident_memory_bytes gauge
ovs_db_process_resident_memory_bytes 3.2530432e+07
# HELP ovs_db_process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE ovs_db_process_start_time_seconds gauge
ovs_db_process_start_time_seconds 1.66057123164e+09
# HELP ovs_db_process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE ovs_db_process_virtual_memory_bytes gauge
ovs_db_process_virtual_memory_bytes 9.6534528e+07
# HELP ovs_db_process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
# TYPE ovs_db_process_virtual_memory_max_bytes gauge
ovs_db_process_virtual_memory_max_bytes 1.8446744073709552e+19
# HELP ovs_vswitchd_bridge A metric with a constant '1' value labeled by bridge name present on the instance.
# TYPE ovs_vswitchd_bridge gauge
ovs_vswitchd_bridge{bridge="br-ex"} 1
ovs_vswitchd_bridge{bridge="br-int"} 1
# HELP ovs_vswitchd_bridge_flows_total Represents the number of OpenFlow flows on the OVS bridge.
# TYPE ovs_vswitchd_bridge_flows_total gauge
ovs_vswitchd_bridge_flows_total{bridge="br-ex"} 35
ovs_vswitchd_bridge_flows_total{bridge="br-int"} 2207
# HELP ovs_vswitchd_bridge_ports_total Represents the number of OVS ports on the bridge.
# TYPE ovs_vswitchd_bridge_ports_total gauge
ovs_vswitchd_bridge_ports_total{bridge="br-ex"} 3
ovs_vswitchd_bridge_ports_total{bridge="br-int"} 27
# HELP ovs_vswitchd_bridge_reconfigure Number of times OVS bridges were reconfigured.
# TYPE ovs_vswitchd_bridge_reconfigure gauge
ovs_vswitchd_bridge_reconfigure 387
# HELP ovs_vswitchd_bridge_total Represents total number of OVS bridges on the system.
# TYPE ovs_vswitchd_bridge_total gauge
ovs_vswitchd_bridge_total 2
# HELP ovs_vswitchd_dp A metric with a constant '1' value labeled by datapath name present on the instance.
# TYPE ovs_vswitchd_dp gauge
ovs_vswitchd_dp{datapath="ovs-system",type="system"} 1
# HELP ovs_vswitchd_dp_flows_lookup_hit Represents number of packets matching the existing flows while processing incoming packets in the datapath.
# TYPE ovs_vswitchd_dp_flows_lookup_hit gauge
ovs_vswitchd_dp_flows_lookup_hit{datapath="ovs-system"} 9.383964e+06
# HELP ovs_vswitchd_dp_flows_lookup_lost number of packets destined for user space process but subsequently dropped before reaching userspace.
# TYPE ovs_vswitchd_dp_flows_lookup_lost gauge
ovs_vswitchd_dp_flows_lookup_lost{datapath="ovs-system"} 0
# HELP ovs_vswitchd_dp_flows_lookup_missed Represents the number of packets not matching any existing flow and require user space processing.
# TYPE ovs_vswitchd_dp_flows_lookup_missed gauge
ovs_vswitchd_dp_flows_lookup_missed{datapath="ovs-system"} 46338
# HELP ovs_vswitchd_dp_flows_total Represents the number of flows in datapath.
# TYPE ovs_vswitchd_dp_flows_total gauge
ovs_vswitchd_dp_flows_total{datapath="ovs-system"} 405
# HELP ovs_vswitchd_dp_if_total Represents the number of ports connected to the datapath.
# TYPE ovs_vswitchd_dp_if_total gauge
ovs_vswitchd_dp_if_total{datapath="ovs-system"} 25
# HELP ovs_vswitchd_dp_masks_hit Represents the total number of masks visited for matching incoming packets.
# TYPE ovs_vswitchd_dp_masks_hit gauge
ovs_vswitchd_dp_masks_hit{datapath="ovs-system"} 2.9584965e+07
# HELP ovs_vswitchd_dp_masks_hit_ratio Represents the average number of masks visited per packet the ratio between hit and total number of packets processed by the datapath.
# TYPE ovs_vswitchd_dp_masks_hit_ratio gauge
ovs_vswitchd_dp_masks_hit_ratio{datapath="ovs-system"} 3.14
# HELP ovs_vswitchd_dp_masks_total Represents the number of masks in a datapath.
# TYPE ovs_vswitchd_dp_masks_total gauge
ovs_vswitchd_dp_masks_total{datapath="ovs-system"} 59
# HELP ovs_vswitchd_dp_packets_total Represents the total number of packets datapath processed which is the sum of hit and missed.
# TYPE ovs_vswitchd_dp_packets_total gauge
ovs_vswitchd_dp_packets_total{datapath="ovs-system"} 9.430302e+06
# HELP ovs_vswitchd_dp_total Represents total number of datapaths on the system.
# TYPE ovs_vswitchd_dp_total gauge
ovs_vswitchd_dp_total 1
# HELP ovs_vswitchd_dpif_execute Number of times the OpenFlow actions were executed in userspace on behalf of the datapath.
# TYPE ovs_vswitchd_dpif_execute gauge
ovs_vswitchd_dpif_execute 44197
# HELP ovs_vswitchd_dpif_flow_del Number of times flows were deleted from the datapath (Linux kernel datapath module).
# TYPE ovs_vswitchd_dpif_flow_del gauge
ovs_vswitchd_dpif_flow_del 37704
# HELP ovs_vswitchd_dpif_flow_flush Number of times flows were flushed from the datapath (Linux kernel datapath module).
# TYPE ovs_vswitchd_dpif_flow_flush gauge
ovs_vswitchd_dpif_flow_flush 1
# HELP ovs_vswitchd_dpif_flow_get Number of times flows were retrieved from the datapath (Linux kernel datapath module).
# TYPE ovs_vswitchd_dpif_flow_get gauge
ovs_vswitchd_dpif_flow_get 25
# HELP ovs_vswitchd_dpif_flow_put Number of times flows were added to the datapath (Linux kernel datapath module).
# TYPE ovs_vswitchd_dpif_flow_put gauge
ovs_vswitchd_dpif_flow_put 39907
100 51063 0 5106# HELP ovs_vswitchd_dpif_port_add Number of times a netdev was added as a port to the dpif.
3 0 # TYPE ovs_vswitchd_dpif_port_add gauge
0 ovs_vswitchd_dpif_port_add 65
977k # HELP ovs_vswitchd_dpif_port_del Number of times a netdev was removed from the dpif.
0 --:--:-- -# TYPE ovs_vswitchd_dpif_port_del gauge
-:--:-- --:--:--ovs_vswitchd_dpif_port_del 78
977k
# HELP ovs_vswitchd_handlers_total Represents the number of handlers thread. This thread reads upcalls from dpif, forwards each upcall's packet and possibly sets up a kernel flow as a cache.
# TYPE ovs_vswitchd_handlers_total gauge
ovs_vswitchd_handlers_total 4
# HELP ovs_vswitchd_hw_offload Represents whether netdev flow offload to hardware is enabled or not -- false(0) and true(1).
# TYPE ovs_vswitchd_hw_offload gauge
ovs_vswitchd_hw_offload 0
# HELP ovs_vswitchd_interface_collisions_total The total number of packet collisions transmitted by Open vSwitch interface(s).
# TYPE ovs_vswitchd_interface_collisions_total gauge
ovs_vswitchd_interface_collisions_total 0
# HELP ovs_vswitchd_interface_resets_total The number of link state changes observed by Open vSwitch interface(s).
# TYPE ovs_vswitchd_interface_resets_total gauge
ovs_vswitchd_interface_resets_total 4
# HELP ovs_vswitchd_interface_rx_dropped_total The total number of received packets dropped by Open vSwitch interface(s).
# TYPE ovs_vswitchd_interface_rx_dropped_total gauge
ovs_vswitchd_interface_rx_dropped_total 0
# HELP ovs_vswitchd_interface_rx_errors_total The total number of received packets with errors by Open vSwitch interface(s).
# TYPE ovs_vswitchd_interface_rx_errors_total gauge
ovs_vswitchd_interface_rx_errors_total 0
# HELP ovs_vswitchd_interface_tx_dropped_total The total number of transmitted packets dropped by Open vSwitch interface(s).
# TYPE ovs_vswitchd_interface_tx_dropped_total gauge
ovs_vswitchd_interface_tx_dropped_total 1
# HELP ovs_vswitchd_interface_tx_errors_total The total number of transmitted packets with errors by Open vSwitch interface(s).
# TYPE ovs_vswitchd_interface_tx_errors_total gauge
ovs_vswitchd_interface_tx_errors_total 0
# HELP ovs_vswitchd_netlink_overflow Netlink messages dropped by the daemon due to buffer overflow.
# TYPE ovs_vswitchd_netlink_overflow gauge
ovs_vswitchd_netlink_overflow 0
# HELP ovs_vswitchd_netlink_received Number of netlink messages received by the kernel.
# TYPE ovs_vswitchd_netlink_received gauge
ovs_vswitchd_netlink_received 176596
# HELP ovs_vswitchd_netlink_recv_jumbo Number of netlink messages that were received fromthe kernel were more than the allocated buffer.
# TYPE ovs_vswitchd_netlink_recv_jumbo gauge
ovs_vswitchd_netlink_recv_jumbo 49483
# HELP ovs_vswitchd_netlink_sent Number of netlink message sent to the kernel.
# TYPE ovs_vswitchd_netlink_sent gauge
ovs_vswitchd_netlink_sent 204061
# HELP ovs_vswitchd_ofproto_dpif_expired Number of times the flows were removed for reasons - idle timeout, hard timeout, flow delete, group delete, meter delete, or eviction.
# TYPE ovs_vswitchd_ofproto_dpif_expired gauge
ovs_vswitchd_ofproto_dpif_expired 0
# HELP ovs_vswitchd_ofproto_flush Number of times the flows from all of ofproto's flow tables were flushed.
# TYPE ovs_vswitchd_ofproto_flush gauge
ovs_vswitchd_ofproto_flush 1
# HELP ovs_vswitchd_ofproto_packet_out Number of times the controller injected the packet into the kernel datapath.
# TYPE ovs_vswitchd_ofproto_packet_out gauge
ovs_vswitchd_ofproto_packet_out 33
# HELP ovs_vswitchd_ofproto_recv_openflow Number of times an OpenFlow message was handled.
# TYPE ovs_vswitchd_ofproto_recv_openflow gauge
ovs_vswitchd_ofproto_recv_openflow 15798
# HELP ovs_vswitchd_ofproto_reinit_ports Number of times all the OpenFlow ports were reinitialized.
# TYPE ovs_vswitchd_ofproto_reinit_ports gauge
ovs_vswitchd_ofproto_reinit_ports 0
# HELP ovs_vswitchd_packet_in Specifies the number of times ovs-vswitchd has handled the packet-ins on behalf of kernel datapath.
# TYPE ovs_vswitchd_packet_in gauge
ovs_vswitchd_packet_in 43753
# HELP ovs_vswitchd_packet_in_drop Specifies the number of times the ovs-vswitchd has dropped the packet-ins due to resource constraints.
# TYPE ovs_vswitchd_packet_in_drop gauge
ovs_vswitchd_packet_in_drop 0
# HELP ovs_vswitchd_process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE ovs_vswitchd_process_cpu_seconds_total counter
ovs_vswitchd_process_cpu_seconds_total 35.95
# HELP ovs_vswitchd_process_max_fds Maximum number of open file descriptors.
# TYPE ovs_vswitchd_process_max_fds gauge
ovs_vswitchd_process_max_fds 65535
# HELP ovs_vswitchd_process_open_fds Number of open file descriptors.
# TYPE ovs_vswitchd_process_open_fds gauge
ovs_vswitchd_process_open_fds 56
# HELP ovs_vswitchd_process_resident_memory_bytes Resident memory size in bytes.
# TYPE ovs_vswitchd_process_resident_memory_bytes gauge
ovs_vswitchd_process_resident_memory_bytes 2.00040448e+08
# HELP ovs_vswitchd_process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE ovs_vswitchd_process_start_time_seconds gauge
ovs_vswitchd_process_start_time_seconds 1.66057123192e+09
# HELP ovs_vswitchd_process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE ovs_vswitchd_process_virtual_memory_bytes gauge
ovs_vswitchd_process_virtual_memory_bytes 7.07035136e+08
# HELP ovs_vswitchd_process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
# TYPE ovs_vswitchd_process_virtual_memory_max_bytes gauge
ovs_vswitchd_process_virtual_memory_max_bytes 1.8446744073709552e+19
# HELP ovs_vswitchd_pstream_open Specifies the number of time passive connections were opened for the remote peer to connect.
# TYPE ovs_vswitchd_pstream_open gauge
ovs_vswitchd_pstream_open 5
# HELP ovs_vswitchd_rconn_discarded Specifies the number of messages that have been dropped because the send queue had to be flushed because of reconnection.
# TYPE ovs_vswitchd_rconn_discarded gauge
ovs_vswitchd_rconn_discarded 0
# HELP ovs_vswitchd_rconn_overflow Specifies the number of messages that have been dropped because of the queue overflow.
# TYPE ovs_vswitchd_rconn_overflow gauge
ovs_vswitchd_rconn_overflow 0
# HELP ovs_vswitchd_rconn_queued Specifies the number of messages that have been queued because it couldn’t be sent using the underlying virtual connection to OpenFlow devices.
# TYPE ovs_vswitchd_rconn_queued gauge
ovs_vswitchd_rconn_queued 7972
# HELP ovs_vswitchd_rconn_sent Specifies the number of messages that have been sent to the underlying virtual connection (unix, tcp, or ssl) to OpenFlow devices.
# TYPE ovs_vswitchd_rconn_sent gauge
ovs_vswitchd_rconn_sent 7972
# HELP ovs_vswitchd_revalidators_total Represents the number of revalidators thread. This thread processes datapath flows, updates OpenFlow statistics, and updates or removes them if necessary.
# TYPE ovs_vswitchd_revalidators_total gauge
ovs_vswitchd_revalidators_total 2
# HELP ovs_vswitchd_stream_open Specifies the number of attempts to connect to a remote peer (active connection).
# TYPE ovs_vswitchd_stream_open gauge
ovs_vswitchd_stream_open 1
# HELP ovs_vswitchd_tc_policy Represents the policy used with HW offloading -- none(0), skip_sw(1), and skip_hw(2).
# TYPE ovs_vswitchd_tc_policy gauge
ovs_vswitchd_tc_policy 0
# HELP ovs_vswitchd_txn_aborted Specifies the number of times the OVSDB transaction has been aborted.
# TYPE ovs_vswitchd_txn_aborted gauge
ovs_vswitchd_txn_aborted 0
# HELP ovs_vswitchd_txn_error Specifies the number of times the OVSDB transaction has errored out.
# TYPE ovs_vswitchd_txn_error gauge
ovs_vswitchd_txn_error 0
# HELP ovs_vswitchd_txn_incomplete Specifies the number of times the OVSDB transaction did not complete and the client had to re-try.
# TYPE ovs_vswitchd_txn_incomplete gauge
ovs_vswitchd_txn_incomplete 692
# HELP ovs_vswitchd_txn_success Specifies the number of times the OVSDB transaction has successfully completed.
# TYPE ovs_vswitchd_txn_success gauge
ovs_vswitchd_txn_success 522
# HELP ovs_vswitchd_txn_try_again Specifies the number of times the OVSDB transaction failed and the client had to re-try.
# TYPE ovs_vswitchd_txn_try_again gauge
ovs_vswitchd_txn_try_again 0
# HELP ovs_vswitchd_txn_unchanged Specifies the number of times the OVSDB transaction resulted in no change to the database.
# TYPE ovs_vswitchd_txn_unchanged gauge
ovs_vswitchd_txn_unchanged 883
# HELP ovs_vswitchd_txn_uncommitted Specifies the number of times the OVSDB transaction were uncommitted.
# TYPE ovs_vswitchd_txn_uncommitted gauge
ovs_vswitchd_txn_uncommitted 0
# HELP ovs_vswitchd_vconn_open Specifies the number of attempts to connect to an OpenFlow Device.
# TYPE ovs_vswitchd_vconn_open gauge
ovs_vswitchd_vconn_open 0
# HELP ovs_vswitchd_vconn_received Specifies the number of messages received from the OpenFlow Device.
# TYPE ovs_vswitchd_vconn_received gauge
ovs_vswitchd_vconn_received 16621
# HELP ovs_vswitchd_vconn_sent Specifies the number of messages sent to the OpenFlow Device.
# TYPE ovs_vswitchd_vconn_sent gauge
ovs_vswitchd_vconn_sent 9884
# HELP ovs_vswitchd_xlate_actions Number of times an OpenFlow actions were translated into datapath actions.
# TYPE ovs_vswitchd_xlate_actions gauge
ovs_vswitchd_xlate_actions 366622
# HELP ovs_vswitchd_xlate_actions_oversize Number of times the translated OpenFlow actions into a datapath actions were too big for a netlink attribute.
# TYPE ovs_vswitchd_xlate_actions_oversize gauge
ovs_vswitchd_xlate_actions_oversize 0
# HELP ovs_vswitchd_xlate_actions_too_many_output Number of times the number of datapath actions were more than what the kernel can handle reliably.
# TYPE ovs_vswitchd_xlate_actions_too_many_output gauge
ovs_vswitchd_xlate_actions_too_many_output 0
[weliang@weliang ~]$
[weliang@weliang ~]$ oc get clusterversion
NAME VERSION AVAILABLE PROGRESSING SINCE STATUS
version 4.12.0-0.nightly-2022-08-12-053438 True False 15m Cluster version is 4.12.0-0.nightly-2022-08-12-053438
[weliang@weliang ~]$
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.12.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:7399 The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days |