Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1566238 - upgrade from v3.7 to v3.9 fails with openshift-ansible-3.9.20-1.git.0.f99fb43.el7
upgrade from v3.7 to v3.9 fails with openshift-ansible-3.9.20-1.git.0.f99fb43...
Status: CLOSED CURRENTRELEASE
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer (Show other bugs)
3.9.0
Unspecified Unspecified
unspecified Severity unspecified
: ---
: 3.9.z
Assigned To: Vadim Rutkovsky
Weihua Meng
:
: 1566435 (view as bug list)
Depends On:
Blocks: 1561247
  Show dependency treegraph
 
Reported: 2018-04-11 16:08 EDT by Jason Montleon
Modified: 2018-06-18 14:18 EDT (History)
7 users (show)

See Also:
Fixed In Version: openshift-ansible-3.9.24-1.git.0.d0289ea.el7
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2018-06-18 14:18:27 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:2013 normal SHIPPED_LIVE Important: OpenShift Container Platform 3.9 security, bug fix, and enhancement update 2018-06-27 18:01:43 EDT

  None (edit)
Description Jason Montleon 2018-04-11 16:08:54 EDT
Description of problem:
upgrade from v3.7 to v3.9 fails with openshift-ansible-3.9.20-1.git.0.f99fb43.el7

Version-Release number of the following components:
openshift-ansible-3.9.20-1.git.0.f99fb43.el7

How reproducible:
Always

Steps to Reproduce:
1. Install v3.7
2. Try to upgrade to v3.9 with openshift-ansible-3.9.20-1.git.0.f99fb43.el7

Actual results:
TASK [etcd : Generate etcd backup] *********************************************
    fatal: [192.168.120.4.nip.io]: FAILED! => {"changed": false, "cmd": "runc exec etcd etcdctl backup --data-dir=/var/lib/etcd/ --backup-dir=/var/lib/etcd//openshift-backup-pre-upgrade-20180411080955", "msg": "[Errno 2] No such file or directory", "rc": 2}

Expected results:
Upgrade worls

Additional info:
Works with the previous version:
openshift-ansible-3.9.19-1.git.0.34f4090
Comment 2 Weihua Meng 2018-04-11 21:55:56 EDT
I did an upgrade with openshift-ansible-3.9.20-1.git.0.f99fb43.el7.noarch
Not meet this error

<debug-wmengah37-master-etcd-3.0411-l0i.qe.rhcloud.com> (0, '\r\n{"changed": true, "end": "2018-04-11 08:36:21.802860", "stdout": "", "cmd": ["docker", "exec", "etcd_container", "etcdctl", "backup", "--data-dir=/var/lib/etcd/", "--backup-dir=/var/lib/etcd//openshift-backup-pre-upgrade-20180411043609"], "rc": 0, "start": "2018-04-11 08:36:20.701460", "stderr": "", "delta": "0:00:01.101400", "invocation": {"module_args": {"warn": true, "executable": null, "_uses_shell": false, "_raw_params": "docker exec etcd_container etcdctl backup --data-dir=/var/lib/etcd/ --backup-dir=/var/lib/etcd//openshift-backup-pre-upgrade-20180411043609", "removes": null, "creates": null, "chdir": null, "stdin": null}}}\r\n', 'Shared connection to debug-wmengah37-master-etcd-3.0411-l0i.qe.rhcloud.com closed.\r\n')
changed: [debug-wmengah37-master-etcd-3.0411-l0i.qe.rhcloud.com] => {
    "changed": true,
    "cmd": [
        "docker",
        "exec",
        "etcd_container",
        "etcdctl",
        "backup",
        "--data-dir=/var/lib/etcd/",
        "--backup-dir=/var/lib/etcd//openshift-backup-pre-upgrade-20180411043609"
    ],
    "delta": "0:00:01.101400",
    "end": "2018-04-11 08:36:21.802860",
    "failed": false,
    "invocation": {
        "module_args": {
            "_raw_params": "docker exec etcd_container etcdctl backup --data-dir=/var/lib/etcd/ --backup-dir=/var/lib/etcd//openshift-backup-pre-upgrade-20180411043609",
            "_uses_shell": false,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "stdin": null,
            "warn": true
        }
    },
    "rc": 0,
    "start": "2018-04-11 08:36:20.701460",
    "stderr": "",
    "stderr_lines": [],
    "stdout": "",
    "stdout_lines": []
}

PLAY RECAP ********************************************************************************************************************************************************************************************************
debug-wmengah37-lb-1.0411-l0i.qe.rhcloud.com : ok=59   changed=2    unreachable=0    failed=0   
debug-wmengah37-master-etcd-1.0411-l0i.qe.rhcloud.com : ok=774  changed=149  unreachable=0    failed=0   
debug-wmengah37-master-etcd-2.0411-l0i.qe.rhcloud.com : ok=523  changed=72   unreachable=0    failed=0   
debug-wmengah37-master-etcd-3.0411-l0i.qe.rhcloud.com : ok=523  changed=72   unreachable=0    failed=0   
debug-wmengah37-node-primary-1.0411-l0i.qe.rhcloud.com : ok=94   changed=17   unreachable=0    failed=0   
debug-wmengah37-node-primary-2.0411-l0i.qe.rhcloud.com : ok=94   changed=17   unreachable=0    failed=0   
debug-wmengah37-nrri-1.0411-l0i.qe.rhcloud.com : ok=94   changed=17   unreachable=0    failed=0   
debug-wmengah37-nrri-2.0411-l0i.qe.rhcloud.com : ok=94   changed=17   unreachable=0    failed=0   
localhost                  : ok=38   changed=0    unreachable=0    failed=0
Comment 3 Scott Dodson 2018-04-12 17:37:59 EDT
Can you confirm this is a system container based install?
Comment 4 Jason Montleon 2018-04-13 08:38:53 EDT
It is an RPM install.
Comment 5 Weihua Meng 2018-04-16 05:53:39 EDT
reproduced by RPM install upgrade.
containerized install (RHEL and Atomic Host) upgrade does not have this issue.

openshift-ansible-3.9.22-1.git.0.2e15102.el7.noarch
Comment 6 Vadim Rutkovsky 2018-04-16 06:16:53 EDT
(In reply to Jason Montleon from comment #4)
> It is an RPM install.

It seems the code incorrectly decides its a system container installation.

Could you attach the full update log?
Comment 7 Vadim Rutkovsky 2018-04-16 07:28:49 EDT
Nvm, can reproduce on any update.

Created https://github.com/openshift/openshift-ansible/pull/7974 to fix it
Comment 8 Russell Teague 2018-04-17 09:17:10 EDT
*** Bug 1566435 has been marked as a duplicate of this bug. ***
Comment 9 Vadim Rutkovsky 2018-04-19 03:53:03 EDT
Fix is available in openshift-ansible-3.9.24-1
Comment 10 Weihua Meng 2018-04-19 10:44:01 EDT
Fixed.
openshift-ansible-3.9.24-1.git.0.d0289ea.el7.noarch

###  RPM  ###
  Operating System: Red Hat Enterprise Linux Server 7.5 (Maipo)
       CPE OS Name: cpe:/o:redhat:enterprise_linux:7.5:GA:server
            Kernel: Linux 3.10.0-862.el7.x86_64
      Architecture: x86-64

changed: [qe-wmengrpm37-master-etcd-1.0419-u40.qe.rhcloud.com] => {
    "changed": true,
    "cmd": [
        "etcdctl",
        "backup",
        "--data-dir=/var/lib/etcd/",
        "--backup-dir=/var/lib/etcd//openshift-backup-pre-upgrade-20180419095653"
    ],


###  containerized  ###
  Operating System: Red Hat Enterprise Linux Atomic Host 7.5.0
       CPE OS Name: cpe:/o:redhat:enterprise_linux:7.5:GA:atomic-host
            Kernel: Linux 3.10.0-862.el7.x86_64
      Architecture: x86-64

changed: [qe-wmengah75-372-master-etcd-1.0419-q09.qe.rhcloud.com] => {
    "changed": true,
    "cmd": [
        "docker",
        "exec",
        "etcd_container",
        "etcdctl",
        "backup",
        "--data-dir=/var/lib/etcd/",
        "--backup-dir=/var/lib/etcd//openshift-backup-pre-upgrade-20180419100212"
    ],

Note You need to log in before you can comment on or make changes to this bug.