Bug 1472441 - Upgrade from 3.5 to 3.6 fails due to missing directory
Upgrade from 3.5 to 3.6 fails due to missing directory
Status: CLOSED NOTABUG
Product: OpenShift Container Platform
Classification: Red Hat
Component: Upgrade (Show other bugs)
3.6.0
Unspecified Unspecified
unspecified Severity medium
: ---
: ---
Assigned To: Jan Chaloupka
liujia
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-07-18 14:06 EDT by Vikas Laad
Modified: 2017-07-19 09:22 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-07-19 09:22:52 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Vikas Laad 2017-07-18 14:06:59 EDT
Description of problem:
Upgrade fails due to wrong directory used for backup, I am using interactive installation.

..........................................fatal: [172.31.47.154]: FAILED! => {"changed": true, "cmd": ["stat", "-c", "%C", "/var/lib/origin/openshift.local.etcd"], "delta": "0:00:00.002798", "end": "2017-07-18 13:51:43.112438", "failed": true, "rc": 1, "start": "2017-07-18 13:51:43.109640", "stderr": "stat: cannot stat ‘/var/lib/origin/openshift.local.etcd’: No such file or directory", "stderr_lines": ["stat: cannot stat ‘/var/lib/origin/openshift.local.etcd’: No such file or directory"], "stdout": "", "stdout_lines": []}


Version-Release number of selected component (if applicable):
3.5 version
openshift v3.5.5.31.5
kubernetes v1.5.2+43a9be4
etcd 3.1.0

atomic-openshift-utils.noarch 3.6.153-1.git.0.5a6bf7d.el7

Steps to Reproduce:
1. Install ocp 3.5 
2. create installer config
3. run upgrade from https://docs.openshift.com/container-platform/3.5/install_config/upgrading/automated_upgrades.html#upgrading-using-the-installation-utility-to-upgrade

Actual results:
Upgrade fails

Expected results:
Upgrade completes

Additional info:
will attach installer config file.

Description of problem:

Version-Release number of the following components:
rpm -q openshift-ansible
rpm -q ansible
ansible --version

How reproducible:

Steps to Reproduce:
1.
2.
3.

Actual results:
Please include the entire output from the last TASK line through the end of output if an error is generated

Expected results:

Additional info:
Please attach logs from ansible-playbook with the -vvv flag
Comment 2 liujia 2017-07-18 22:29:24 EDT
@Vikas Laad

According to QE's verify, it should not be a bug.

1. Checked above enviroment, an external etcd was installed on the master host.
# rpm -qa|grep etcd
etcd-3.1.9-2.el7.x86_64

 # cat /etc/origin/master/master-config.yaml |grep etcdClient -A 5
etcdClientInfo:
  ca: master.etcd-ca.crt
  certFile: master.etcd-client.crt
  keyFile: master.etcd-client.key
  urls:
    - https://ip-172-31-47-154.x.x.x.x:2379

2. Checked installer.cfg.yml in attachment, which shows that the cluster used  embedded etcd because no etcd role specified in installer.cfg.yml file.
<--snip-->
 roles: 
    master:
    node:

With this installer.cfg.yml file, run "atomic-openshift-installer -c /tmp/bug/installer.cfg.yml upgrade", a hosts file generated in the same directory.
[OSEv3:children]
nodes
masters

[OSEv3:vars]
...

When use external etcd, the data dir should be /var/lib/etcd/ and if use embedded etcd the data dir should be /var/lib/origin/. The error happened because it could not find /var/lib/origin/ when you specify a config file without etcd role.

So this issue should be caused by a wrong installer.cfg.yml file which is not fit with your worked cluster.

BTW, maybe you need not create installer.cfg.yml manually, if you install old version ocp with advanced install method, then you can continue upgrade work with hosts file with advanced way. If you install old cluster with quick install method, then you can find correct installer.cfg.yml file placed in /root/.config/openshift/.
Comment 3 Vikas Laad 2017-07-19 09:03:13 EDT
@liujia I am sorry I attached wrong file, I modified that file later and added etcd role. Upgrade still failed, by mistake I attached old config where role was missing. I still have the env, you can have a look. Attaching the latest config file.
Comment 4 Vikas Laad 2017-07-19 09:22:52 EDT
Please disregard comment #3, after adding etcd to roles list upgrade works fine.

Note You need to log in before you can comment on or make changes to this bug.