Bug 1786218 - [OSP][Kuryr] Could not install iscsi-target image
Summary: [OSP][Kuryr] Could not install iscsi-target image
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.3.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium
Target Milestone: ---
: 4.3.z
Assignee: Luis Tomas Bolivar
QA Contact: GenadiC
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-12-24 02:48 UTC by Chao Yang
Modified: 2020-06-17 20:28 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-06-17 20:27:11 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:2436 0 None None None 2020-06-17 20:28:20 UTC

Description Chao Yang 2019-12-24 02:48:16 UTC
Description of problem:
Could not install iscsi-target image 

Version-Release number of selected component (if applicable):
4.3.0-0.nightly-2019-12-22-054650

How reproducible:
Always

Steps to Reproduce:
1.Create scc
https://raw.githubusercontent.com/openshift-qe/docker-iscsi/master/scc.yml
2.Create target
https://github.com/openshift-qe/docker-iscsi/blob/master/iscsi-target.json
3.
NAME               READY   STATUS              RESTARTS   AGE
pod/iscsi-target   0/1     ContainerCreating   0          17m

Events:
  Type     Reason                  Age        From                                Message
  ----     ------                  ----       ----                                -------
  Normal   Scheduled               <unknown>  default-scheduler                   Successfully assigned storage/iscsi-target to ostest-xlw6j-worker-8mttb
  Warning  FailedCreatePodSandBox  60s        kubelet, ostest-xlw6j-worker-8mttb  Failed create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_iscsi-target_storage_5d49022a-1a64-4807-9390-f4156b928ea6_0(de286fdb0151f237d1490138b4a18ea3038ee6a3fefe24d1cecb5624ed0dfaaf): Multus: error adding pod to network "kuryr": delegateAdd: error invoking DelegateAdd - "kuryr-cni": error in getting result from AddNetwork: CNI Daemon returned error 500
  Warning  FailedCreatePodSandBox  43s        kubelet, ostest-xlw6j-worker-8mttb  Failed create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_iscsi-target_storage_5d49022a-1a64-4807-9390-f4156b928ea6_0(07dfcfe456ba68345c0faa2455fbe3e5d7ce5ddc1a95cac8f2edb56395c06f1b): Multus: error adding pod to network "kuryr": delegateAdd: error invoking DelegateAdd - "kuryr-cni": error in getting result from AddNetwork: CNI Daemon returned error 500
  Warning  FailedCreatePodSandBox  26s        kubelet, ostest-xlw6j-worker-8mttb  Failed create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_iscsi-target_storage_5d49022a-1a64-4807-9390-f4156b928ea6_0(077e1419e8edd64642e58b6540a4959f9dccd7ce91e5ad4953cdd013ea7fff1f): Multus: error adding pod to network "kuryr": delegateAdd: error invoking DelegateAdd - "kuryr-cni": error in getting result from AddNetwork: CNI Daemon returned error 500
  Warning  FailedCreatePodSandBox  9s         kubelet, ostest-xlw6j-worker-8mttb  Failed create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_iscsi-target_storage_5d49022a-1a64-4807-9390-f4156b928ea6_0(e19251b85615a96bd83eceb70c752f73cdd2e864d0bd59cd68d3a327d5459eae): Multus: error adding pod to network "kuryr": delegateAdd: error invoking DelegateAdd - "kuryr-cni": error in getting result from AddNetwork: CNI Daemon returned error 500

Actual results:
pod/iscsi-target could not be running

Expected results:
pod/iscsi-target should be running

Additional info:

Comment 1 Luis Tomas Bolivar 2019-12-24 16:05:32 UTC
A bit more information on this may be helpful. Any error/logs on the kuryr-controller? Any erros on the kuryr-cni on the worker ostest-xlw6j-worker-8mttb? What is this test doing?

Comment 2 Chao Yang 2019-12-25 08:30:08 UTC
I can see some logs from kuryr-cni
2019-12-25 08:25:45.383 56 ERROR kuryr_kubernetes.cni.daemon.service [-] Error when processing addNetwork request. CNI Params: {'CNI_IFNAME': 'eth0', 'CNI_NETNS': '/proc/1392981/ns/net', 'CNI_PATH': '/opt/multus/bin:/var/lib/cni/bin', 'CNI_COMMAND': 'ADD', 'CNI_CONTAINERID': '531d21b76015f7afc46e2641d30c00354f89ed534ea326cbb75b4aae8aeb9be3', 'CNI_ARGS': 'IgnoreUnknown=true;K8S_POD_NAMESPACE=storage;K8S_POD_NAME=iscsi-target;K8S_POD_INFRA_CONTAINER_ID=531d21b76015f7afc46e2641d30c00354f89ed534ea326cbb75b4aae8aeb9be3'}: pyroute2.netlink.exceptions.NetlinkError: (17, 'File exists')

Comment 4 Luis Tomas Bolivar 2020-01-02 08:03:18 UTC
Ohh, ok, then this is most probably a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1786298.

Closing it as duplicate of the other one, please reopen if hit it again with newer versions. That is already fixed

*** This bug has been marked as a duplicate of bug 1786298 ***

Comment 8 Chao Yang 2020-01-08 09:18:10 UTC
Iscsi function is passed on osp13 with kuryr enabled.
cluster version is 4.3.0-0.nightly-2020-01-06-185654
1.Create iscsi target
2.Config iscsi initiator
3.Create pv, pvc, pod
4.Pod is running
pod/iscsi   1/1     Running   0          5h25m
Name:         iscsi
Namespace:    default
Priority:     0
Node:         ostest-vqq5d-worker-6jcw4/10.196.0.14
Start Time:   Tue, 07 Jan 2020 22:17:21 -0500
Labels:       name=iscsi
Annotations:  k8s.v1.cni.cncf.io/networks-status:
                [{
                    "name": "kuryr",
                    "interface": "eth0",
                    "ips": [
                        "10.128.82.3"
                    ],
                    "mac": "fa:16:3e:a6:94:3d",
                    "dns": {}
                }]
              openstack.org/kuryr-pod-label: {"name": "iscsi"}
              openstack.org/kuryr-vif:
                {"versioned_object.changes": ["default_vif"], "versioned_object.data": {"additional_vifs": {}, "default_vif": {"versioned_object.changes":...
Status:       Running
IP:           10.128.82.3
IPs:
  IP:  10.128.82.3
Containers:
  iscsi:
    Container ID:   cri-o://9b8c2189b1b3c0aa985fe971f719d0825f9ec69b4f37f96c0ddcae1496e777af
    Image:          jhou/hello-openshift
    Image ID:       docker.io/jhou/hello-openshift@sha256:72a676ac8fe20f2506502d128b021a64f92d00bf444c64e000121b36011655aa
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Tue, 07 Jan 2020 22:18:00 -0500
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /mnt/iscsi from iscsi (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-rxfcf (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  iscsi:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  pvc-iscsi
    ReadOnly:   false
  default-token-rxfcf:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-rxfcf
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>

Comment 9 Chao Yang 2020-01-09 02:39:13 UTC
Update to verified since iscsi function works on osp with kuryr enabled env. 
Issue "NetlinkError: (17, 'File exists')" is tacking by https://bugzilla.redhat.com/show_bug.cgi?id=1786298

Comment 12 errata-xmlrpc 2020-06-17 20:27:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2436


Note You need to log in before you can comment on or make changes to this bug.