Bug 1420182 - [3.5] conntrack executable not found on $PATH during cluster horizontal run
Summary: [3.5] conntrack executable not found on $PATH during cluster horizontal run
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.5.0
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
: ---
Assignee: Giuseppe Scrivano
QA Contact: Johnny Liu
URL:
Whiteboard:
Depends On:
Blocks: 1420393 1420395
TreeView+ depends on / blocked
 
Reported: 2017-02-08 04:05 UTC by Mike Fiedler
Modified: 2017-07-24 14:11 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
OCP 3.4 and 3.3 introduced a requirement on the conntrack executable but this dependency was not enforced at install time so service proxy management may have failed post installation. The installer now ensures that conntrack is installed.
Clone Of:
: 1420393 1420395 (view as bug list)
Environment:
Last Closed: 2017-04-12 18:49:56 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2017:0903 0 normal SHIPPED_LIVE OpenShift Container Platform atomic-openshift-utils bug fix and enhancement 2017-04-12 22:45:42 UTC

Description Mike Fiedler 2017-02-08 04:05:46 UTC
Description of problem:

While scaling a 300 node cluster up to 1000 projects and 4000 running pods, the following node Error level message popped a few times in the log on the master.   I need to check other nodes to see if they hit it, too.

Feb  7 22:56:58 192 atomic-openshift-node: E0207 22:56:58.045551    3115 proxier.go:755] conntrack return with error: Error looking for path of conntrack: exec: "conntrack": executable file not found in $PATH
Feb  7 22:56:58 192 atomic-openshift-node: E0207 22:56:58.045600    3115 proxier.go:755] conntrack return with error: Error looking for path of conntrack: exec: "conntrack": executable file not found in $PATH



Version-Release number of selected component (if applicable): 3.5.0.17


How reproducible:   Unknown,  saw a few instances during this test and will investigate other node logs


Expected:  No Error or Fatal messages.

Comment 2 Mike Fiedler 2017-02-08 04:07:45 UTC
This cluster was installed with the latest (as of 7 Feb) master branch of openshift-ansible/playbooks/byo/config.yml

Comment 3 Ben Bennett 2017-02-08 14:14:36 UTC
We need to make sure that the libnetfilter_conntrack RPM is installed on any node running openshift-node.

Comment 4 Scott Dodson 2017-02-08 14:39:51 UTC
/usr/sbin/conntrack is provided by conntrack-tools need to ensure that's installed on all nodes.

Comment 5 Scott Dodson 2017-02-08 14:41:40 UTC
Including verifying availability on atomic host and/or the containerized node image. Currently libnetfilter_conntrack is required there but not conntrack-tools

Comment 6 Giuseppe Scrivano 2017-02-09 10:43:54 UTC
change for the container version:

https://github.com/openshift/origin/pull/12885

not containerized version:

https://github.com/openshift/openshift-ansible/pull/3305

Comment 8 Johnny Liu 2017-02-22 14:13:29 UTC
Verified this bug with openshift-ansible-3.5.13-1.git.0.562e91d.el7.noarch, and PASS.

For rpm install,the following step is shown in node install.
TASK [openshift_node : Install conntrack-tools package] ************************

After installation, check rpm package.
# rpm -q conntrack-tools
conntrack-tools-1.4.3-1.el7.x86_64

For containerized install:
# docker run --rm -ti --entrypoint rpm openshift3/node:v3.5.0.32  -q conntrack-tools
conntrack-tools-1.4.3-1.el7.x86_64

Comment 10 errata-xmlrpc 2017-04-12 18:49:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:0903


Note You need to log in before you can comment on or make changes to this bug.