Bug 1876835 - master ignitions fails in UPI install with Traceback (most recent call last): File "<string>", line 3, in <module> KeyError: 'storage'
Summary: master ignitions fails in UPI install with Traceback (most recent call last):...
Keywords:
Status: CLOSED DUPLICATE of bug 1868439
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.6
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
: ---
Assignee: Pierre Prinetti
QA Contact: David Sanz
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-09-08 10:17 UTC by Jatan Malde
Modified: 2020-09-18 10:45 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-09-08 15:05:25 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Jatan Malde 2020-09-08 10:17:41 UTC
Description of problem:

Using the latest feature build and following this section for OpenShift on OpenStack UPI installation. 

https://docs.openshift.com/container-platform/4.6/installing/installing_openstack/installing-openstack-user.html#installation-osp-creating-control-plane-ignition_installing-openstack-user

(overcloud) [stack@undercloud-0 bugtest]$ ./openshift-install version
./openshift-install 4.6.0-fc.4
built from commit 878562e15ce6c31148b4cff758ea70e685e6b6e3
release image quay.io/openshift-release-dev/ocp-release@sha256:960ec73733150827076cbb5fa2c1f5aaa9a94bfbce1b4897e46432a56ac976c1
(overcloud) [stack@undercloud-0 bugtest]$
(overcloud) [stack@undercloud-0 bugtest]$ ./openshift-install create manifests
INFO Credentials loaded from file "/home/stack/shift/upi/bugtest/clouds.yaml" 
INFO Consuming Install Config from target directory 
WARNING Making control-plane schedulable by setting MastersSchedulable to true for Scheduler cluster settings 
INFO Manifests created in: manifests and openshift 
(overcloud) [stack@undercloud-0 bugtest]$ rm -f openshift/99_openshift-cluster-api_master-machines-*.yaml openshift/99_openshift-cluster-api_worker-machineset-*.yaml
(overcloud) [stack@undercloud-0 bugtest]$ ./openshift-install create ignition-configs
INFO Consuming Worker Machines from target directory 
INFO Consuming Common Manifests from target directory 
INFO Consuming Master Machines from target directory 
INFO Consuming OpenShift Install (Manifests) from target directory 
INFO Consuming Openshift Manifests from target directory 
INFO Ignition-Configs created in: . and auth      
(overcloud) [stack@undercloud-0 bugtest]$ export INFRA_ID=$(jq -r .infraID metadata.json)
(overcloud) [stack@undercloud-0 bugtest]$ for index in $(seq 0 2); do
>     MASTER_HOSTNAME="$INFRA_ID-master-$index\n"
>     python -c "import base64, json, sys;
> ignition = json.load(sys.stdin);
> files = ignition['storage'].get('files', []);
> files.append({'path': '/etc/hostname', 'mode': 420, 'contents': {'source': 'data:text/plain;charset=utf-8;base64,' + base64.standard_b64encode(b'$MASTER_HOSTNAME').decode().strip()}});
> ignition['storage']['files'] = files;
> json.dump(ignition, sys.stdout)" <master.ign >"$INFRA_ID-master-$index-ignition.json"
> done
Traceback (most recent call last):
  File "<string>", line 3, in <module>
KeyError: 'storage'
Traceback (most recent call last):
  File "<string>", line 3, in <module>
KeyError: 'storage'
Traceback (most recent call last):
  File "<string>", line 3, in <module>
KeyError: 'storage'
(overcloud) [stack@undercloud-0 bugtest]$

The steps mention running the above command to create 3 master ignitions but since the above messages are reported the files which created are empty. 

Ideally the above error should be more self explanatory and the stacktrace is not helpful.

Version-Release number of the following components:
rpm -q openshift-ansible
rpm -q ansible
ansible --version

How reproducible:

Steps to Reproduce:
1. Use openshift-install binary as listed above.
2. Follow the steps mentioned in the UPI documentation and run the command as mentioned to create 3 master ignition file.
3. 

Actual results:
Please include the entire output from the last TASK line through the end of output if an error is generated

From the build I see the ignitions created are of v3.1.0 and the python script does not fail for bootstrap since the storage section is available. 
Take a look at the master.ign file and I do not see the storage section and so is the above stacktrace. 

(overcloud) [stack@undercloud-0 bugtest]$ cat master.ign | jq .
{
  "ignition": {
    "config": {
      "merge": [
        {
          "source": "https://30.0.0.5:22623/config/master"
        }
      ]
    },
    "security": {
      "tls": {
        "certificateAuthorities": [
          {
            "source": "data:text/plain;charset=utf-8;base64,LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFRENDQWZpZ0F3SUJBZ0lJSXg4QWY4dUU0N1F3RFFZSktvWklodmNOQVFFTEJRQXdKakVTTUJBR0ExVUUKQ3hNSmIzQmxibk5vYVdaME1SQXdEZ1lEVlFRREV3ZHliMjkwTFdOaE1CNFhEVEl3TURrd09EQTVOVFV6TmxvWApEVE13TURrd05qQTVOVFV6Tmxvd0pqRVNNQkFHQTFVRUN4TUpiM0JsYm5Ob2FXWjBNUkF3RGdZRFZRUURFd2R5CmIyOTBMV05oTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF2V2FQUGxmOWg4RWoKSmxEZjloaldCZ1pSVDRycDdXOEcyeS9PMXRabFBhdkErNkcxTHFmNFd4MVZ1OWVJeFBiQkl1bytVMkZ4MzNkWQp0SUx4T1BGTmlkRlcybTMrM3RzTUtiaHEvbnJ0Q0U5L0lBNUovYzhQSFRkaVhaV2RxUHZJYjBPQnFKbUl3Yk03CllIQUJNNnhsVjFqYUJ6YTczQ0E4SWVnallLcjZtVVlHV3pteXptanRjTTRDM2t1dXJ0WmhLZUZSSzBKK3FtM1oKVzZJZC9wazNSMmlsZytHYkl1NG9EakVPVjJXWWZzZ0hzenRQSkFQYm05T3VhbUN6VWhkSnpYWjBQU2RnbzhkQgpaT0NhSzRVZGJDRFVCbFo5SUxxemRaMGVpajJYWk1kM1NCdFNydTFsZnJSbk1WdFMwaExIa3A1V2MzVmdaTzFvCk5pS0JnVDkzendJREFRQUJvMEl3UURBT0JnTlZIUThCQWY4RUJBTUNBcVF3RHdZRFZSMFRBUUgvQkFVd0F3RUIKL3pBZEJnTlZIUTRFRmdRVWNsRHBlTHJxY3pzUzRoSFpxd3h1SmNHa093QXdEUVlKS29aSWh2Y05BUUVMQlFBRApnZ0VCQUxXMzNRcC83N3crL1U2T1lHS1loUnBPenh1eWY2M280cVlFNUM5WU4wNVlsOGk5S2wydWo3UWxTZk41CnR6cWFlaU1CSHBhTHZzb1dSU0ZycWRqc0VsU2ZJM29aeElFSDN3VGJEaVB2ckxpU3NmSEc5SUxEZDNmNnY4b1kKU3dKMDNwY3hscGtwb24zdEpsQlV2NHlSTlJ2Nno4Q2x5V3BGOHZUengyZGdNdDlwTkVKOEpGSmphRW92bzdFLwozT1hmVHlmM0FNVVRUS0swTzBLd2VHQ0hyS0pROVdiMEZyVmtreDVPVUI0c1JBVngxTGpMWjZvMGRYczVpUGwvCk9wWk9QM0hTcXgyVnI0S210Mk1jTndZcGQrZmhOM0xIZFUwcitKTlYxcW1FMjEvQ2hLaFRtWlFMK0IwcnhWY2kKWmFCR0x4WUFVTGtwUmxwUTZmNXh4VXRTRlY0PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg=="
          }
        ]
      }
    },
    "version": "3.1.0"
  }
}

Expected results:

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

Comment 2 Pierre Prinetti 2020-09-08 15:05:25 UTC
This should be fixed with the current upstream documentation[1].


[1]: https://github.com/openshift/installer/blob/master/docs/user/openstack/install_upi.md#master-ignition

*** This bug has been marked as a duplicate of bug 1868439 ***


Note You need to log in before you can comment on or make changes to this bug.