Bug 1679372

Summary: kubernetes 1.13.2 added a dependency on conntrack-tools which is missing in rhel7 repos
Product: Red Hat Enterprise Linux 7 Reporter: Alex Osadchyy <lockerua>
Component: conntrack-toolsAssignee: Phil Sutter <psutter>
Status: CLOSED ERRATA QA Contact: Jianwen Ji <jiji>
Severity: high Docs Contact:
Priority: high    
Version: 7.6CC: bbreard, bmcclain, bperkins, dcbw, dhorak, dustymabe, imcleod, jiji, jligon, nstielau, psutter, rkhan, shuali, toneata, yselkowi
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: s390x   
OS: Linux   
Whiteboard:
Fixed In Version: conntrack-tools-1.4.4-7.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1749286 (view as bug list) Environment:
Last Closed: 2020-03-31 20:06:38 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1730456, 1734478, 1749286, 1750744    

Description Alex Osadchyy 2019-02-21 02:16:31 UTC
Description of problem:
kubernetes 1.13.2 cannot be installed on RHEL7 due to an unresolvable dependency on conntrack-tools on s390x architecture (didn't check others). It is missing in rhel7 repos. 
See related issue and PR on github.com:
https://github.com/kubernetes/kubeadm/issues/1287 

The workaround is to install those manually:
yum install libnetfilter_queue
wget http://rpmfind.net/linux/fedora-secondary/development/rawhide/Everything/s390x/os/Packages/l/libnetfilter_cthelper-1.0.0-15.fc30.s390x.rpm 
rpm -i libnetfilter_cthelper-1.0.0-15.fc30.s390x.rpm 
wget https://rpmfind.net/linux/fedora-secondary/development/rawhide/Everything/s390x/os/Packages/l/libnetfilter_cttimeout-1.0.0-13.fc30.s390x.rpm
rpm -i libnetfilter_cttimeout-1.0.0-13.fc30.s390x.rpm
wget https://rpmfind.net/linux/fedora-secondary/development/rawhide/Everything/s390x/os/Packages/c/conntrack-tools-1.4.5-3.fc30.s390x.rpm
rpm -i conntrack-tools-1.4.5-3.fc30.s390x.rpm 

Then proceed with k8s installation
yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes


Version-Release number of the following components:
rpm -q openshift-ansible
rpm -q ansible
ansible --version

How reproducible:

Steps to Reproduce:
1.Install kubernetes by following standard instructions on RHEL7.6 and openshift.
2.https://kubernetes.io/docs/setup/independent/install-kubeadm/
3.

Actual results:
yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
fails with unresolved dependency on conntrack missing in rhel7 base and optional repos

Expected results:
yum install resolves and installs all dependencied from the base rhel7 repo
Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 1 Steve Milner 2019-03-25 21:52:26 UTC
Alex,

Thanks for the report and sorry for the delay. Can you provide the version of RHCOS you are using?

Comment 2 Ben Breard 2019-03-25 22:21:54 UTC
I'm moving this BZ to RHEL 7, as it has nothing to do with OpenShift or RHCOS. 

This looks like a request to include conntrack-tools with s390x. Looking at the package browser: https://access.redhat.com/downloads/content/conntrack-tools/1.4.4-4.el7/x86_64/fd431d51/package

We have already included this in RHEL8's build for s390x, but not RHEL 7's.

The decision will be up to the engineering team and PM on if this makes sense to include. Thanks!

Comment 5 Alex Osadchyy 2019-03-26 22:48:05 UTC
Correct. The request is to add contrack-tools in RHEL 7.6.

Comment 8 Steve Almy 2019-03-27 20:07:07 UTC
(In reply to Alex Osadchyy from comment #5)
> Correct. The request is to add contrack-tools in RHEL 7.6.

Hi Alex,

Could you reach out to your support contact in Red Hat support and ask them to associate this request with your account.

That will help us on the tracking side.

Thanks!

Comment 9 Steve Almy 2019-03-28 13:54:52 UTC
^^^ Thanks, Alex.

Comment 10 Dan Williams 2019-06-17 21:08:07 UTC
Li Shuang/Jianwen Ji are you able to confirm the package is in s390x arch per https://bugzilla.redhat.com/show_bug.cgi?id=1679372#c7 ? Is there any more testing detail Phil can give? Thanks!

Comment 13 Phil Sutter 2019-07-17 11:18:37 UTC
*** Bug 1730460 has been marked as a duplicate of this bug. ***

Comment 34 Phil Sutter 2019-09-02 16:48:10 UTC
Fix sent upstream: https://lore.kernel.org/netfilter-devel/20190902164431.18398-1-phil@nwl.cc/T/#u

With that patch applied, upstream testsuite passes for me on an s390x machine from beaker.

Comment 35 Yaakov Selkowitz 2019-09-02 17:07:45 UTC
is that a code fix or testsuite fix?

Comment 36 Phil Sutter 2019-09-02 17:21:18 UTC
(In reply to Yaakov Selkowitz from comment #35)
> is that a code fix or testsuite fix?

It's a code fix.

Comment 39 Phil Sutter 2019-09-04 10:20:46 UTC
Upstream commit to backport:

commit f27901afb038b07532b4c31cb77bbc0bd8068253
Author: Phil Sutter <phil>
Date:   Mon Sep 2 18:39:51 2019 +0200

    conntrack: Fix CIDR to mask conversion on Big Endian
    
    Code assumed host architecture to be Little Endian. Instead produce a
    proper mask by pushing the set bits into most significant position and
    apply htonl() on the result.
    
    Fixes: 3f6a2e90936bb ("conntrack: add support for CIDR notation")
    Signed-off-by: Phil Sutter <phil>
    Acked-by: Pablo Neira Ayuso <pablo>

Comment 43 Phil Sutter 2019-09-10 13:14:54 UTC
As reported in Bug 1750626, conntrack-tools on Big Endian is still problematic. Fix submitted upstream already:

https://lore.kernel.org/netfilter-devel/20190910120631.20817-1-phil@nwl.cc/T/#u

Comment 44 Phil Sutter 2019-09-10 13:18:13 UTC
*** Bug 1750626 has been marked as a duplicate of this bug. ***

Comment 45 Phil Sutter 2019-09-10 20:45:51 UTC
Backported my fix from upstream:

commit 7c5f4b390f4b8dc02aceb0a18ed7c59ff14f392c
Author: Phil Sutter <phil>
Date:   Tue Sep 10 14:02:30 2019 +0200

    nfct: helper: Fix NFCTH_ATTR_PROTO_L4NUM size
    
    Kernel defines NFCTH_TUPLE_L4PROTONUM as of type NLA_U8. When adding a
    helper, NFCTH_ATTR_PROTO_L4NUM attribute is correctly set using
    nfct_helper_attr_set_u8(), though when deleting
    nfct_helper_attr_set_u32() was incorrectly used. Due to alignment, this
    causes trouble only on Big Endian.
    
    Fixes: 5e8f64f46cb1d ("conntrackd: add cthelper infrastructure (+ example FTP helper)")
    Signed-off-by: Phil Sutter <phil>
    Acked-by: Pablo Neira Ayuso <pablo>

Comment 50 errata-xmlrpc 2020-03-31 20:06:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:1147