Bug 1704478

Moving to validations team to take a look.

Alexander,

Could you please run this?

$ export ANSIBLE_KEEP_REMOTE_FILES=1
$ ansible-playbook -vvvv -i /usr/bin/tripleo-ansible-inventory /usr/share/openstack-tripleo-validations/validations/switch-vlans.yaml

Take note of the directory Ansible used to store modules on the host.
In the log you pasted in your comment #1, it should be this line,

<localhost> PUT /home/stack/.ansible/tmp/ansible-local-84625ucc885v0/tmp6vm33zfo TO **/tmp/ansible-stack/ansible-tmp-1556573739.0759747-104081225904581/AnsiballZ_switch_vlans.py**

Navigate to the directory you noted,

$ cd /tmp/ansible-stack/ansible-tmp-1556573739.0759747-104081225904581/

Run the wrapper’s explode command to turn the string into some python files that I can work with for debugging:

$ python3 /tmp/ansible-stack/ansible-tmp-1556573739.0759747-104081225904581/AnsiballZ_switch_vlans.py explode
$ cd debug_dir
$ ll
total 5412
drwxr-xr-x. 3 root root      65 Apr 30 13:41 ansible
-rw-r--r--. 1 root root    8179 Apr 30 14:24 ansible_module_switch_vlans.py
-rw-r--r--. 1 root root 5532748 Apr 30 13:40 args

Now that you should have a debug_dir created, could you please make a tarball and adding it as an attachment to this BZs

Thanks for your help,
Gaël

Created attachment 1561680 [details]
debug_dir

Hi Alexander,

You did the right process but on the wrong ansible task.

Could you please repeat the same process but with the right path coming from the failing task which call the switch_vlans.py module (ie TASK [Check that switch vlans are present if used in nic-config files])?

Thanks
Gaël

This is the error:
TASK [Check that switch vlans are present if used in nic-config files] ***********************************************************************************************************************
task path: /usr/share/openstack-tripleo-validations/validations/switch-vlans.yaml:25
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: stack
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /tmp/ansible-${USER}/ansible-tmp-1557330056.4472654-224543919452709 `" && echo ansible-tmp-1557330056.4472654-224543919452709="` echo /tmp/ansible-${USER}/ansible-tmp-1557330056.4472654-224543919452709 `" ) && sleep 0'                                                                                                      
Using module file /usr/share/openstack-tripleo-validations/validations/library/switch_vlans.py
<localhost> PUT /home/stack/.ansible/tmp/ansible-local-6143332xsiz3bj/tmpow8oae8u TO /tmp/ansible-stack/ansible-tmp-1557330056.4472654-224543919452709/AnsiballZ_switch_vlans.py
<localhost> EXEC /bin/sh -c 'chmod u+x /tmp/ansible-stack/ansible-tmp-1557330056.4472654-224543919452709/ /tmp/ansible-stack/ansible-tmp-1557330056.4472654-224543919452709/AnsiballZ_switch_vlans.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python3 /tmp/ansible-stack/ansible-tmp-1557330056.4472654-224543919452709/AnsiballZ_switch_vlans.py && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible-stack/ansible-tmp-1557330056.4472654-224543919452709/AnsiballZ_switch_vlans.py", line 262, in <module>
    _ansiballz_main()
  File "/tmp/ansible-stack/ansible-tmp-1557330056.4472654-224543919452709/AnsiballZ_switch_vlans.py", line 252, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/tmp/ansible-stack/ansible-tmp-1557330056.4472654-224543919452709/AnsiballZ_switch_vlans.py", line 120, in invoke_module
    imp.load_module('__main__', mod, module, MOD_DESC)
  File "/usr/lib64/python3.6/imp.py", line 235, in load_module
    return load_source(name, filename, file)
  File "/usr/lib64/python3.6/imp.py", line 170, in load_source
    module = _exec(spec, sys.modules[name])
  File "<frozen importlib._bootstrap>", line 618, in _exec
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/tmp/ansible_switch_vlans_payload_t9qfd9o2/__main__.py", line 225, in <module>
  File "/tmp/ansible_switch_vlans_payload_t9qfd9o2/__main__.py", line 209, in main
  File "/tmp/ansible_switch_vlans_payload_t9qfd9o2/__main__.py", line 208, in <dictcomp>
ValueError: too many values to unpack (expected 2)

fatal: [undercloud]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible-stack/ansible-tmp-1557330056.4472654-224543919452709/AnsiballZ_switch_vlans.py\", line 262, in <module>\n    _ansiballz_main()\n  File \"/tmp/ansible-stack/ansible-tmp-1557330056.4472654-224543919452709/AnsiballZ_switch_vlans.py\", line 252, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/tmp/ansible-stack/ansible-tmp-1557330056.4472654-224543919452709/AnsiballZ_switch_vlans.py\", line 120, in invoke_module\n    imp.load_module('__main__', mod, module, MOD_DESC)\n  File \"/usr/lib64/python3.6/imp.py\", line 235, in load_module\n    return load_source(name, filename, file)\n  File \"/usr/lib64/python3.6/imp.py\", line 170, in load_source\n    module = _exec(spec, sys.modules[name])\n  File \"<frozen importlib._bootstrap>\", line 618, in _exec\n  File \"<frozen importlib._bootstrap_external>\", line 678, in exec_module\n  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n  File \"/tmp/ansible_switch_vlans_payload_t9qfd9o2/__main__.py\", line 225, in <module>\n  File \"/tmp/ansible_switch_vlans_payload_t9qfd9o2/__main__.py\", line 209, in main\n  File \"/tmp/ansible_switch_vlans_payload_t9qfd9o2/__main__.py\", line 208, in <dictcomp>\nValueError: too many values to unpack (expected 2)\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

PLAY RECAP ***********************************************************************************************************************************************************************************
undercloud                 : ok=3    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   





So going to use:
/tmp/ansible-stack/ansible-tmp-1557330056.4472654-224543919452709/AnsiballZ_switch_vlans.py

Created attachment 1565719 [details]
new debug_dir

Here is a first statement about what happens here.

The culprit is the custom ansible lookup plugin called tht.py which should return
a beautiful list of tuples containing each files and their contents from the overcloud
swift container.

It well returns a list of something but not a list of tuples, that's why the error 
"ValueError: too many values to unpack (expected 2)"

It seems to be related to Python3 on RHEL 8. Trying to debug it and finding out a
proper solution ASAP.

FailedQA

Environment:
openstack-tripleo-validations-10.4.1-0.20190608000407.ff86911.el8ost.noarch


  source /home/stack/stackrc
  export TRIPLEO_PLAN_NAME=my-templates
  ansible-playbook -vvv -i /usr/bin/tripleo-ansible-inventory /usr/share/openstack-tripleo-validations/validations/switch-vlans.yaml


TASK [Check that switch vlans are present if used in nic-config files] ********************************************************************************************************************************************
task path: /usr/share/openstack-tripleo-validations/validations/switch-vlans.yaml:25
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: stack
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /tmp/ansible-${USER}/ansible-tmp-1560779449.07787-35710612521861 `" && echo ansible-tmp-1560779449.07787-35710612521861="` echo /tmp/ansible-${USER}/ansible-tmp-1560779449.07787-35710612521861 `" ) && sleep 0'                                                                                                                                                         
Using module file /usr/share/openstack-tripleo-validations/validations/library/switch_vlans.py
<localhost> PUT /home/stack/.ansible/tmp/ansible-local-454071ta56ev4/tmp5cebg91c TO /tmp/ansible-stack/ansible-tmp-1560779449.07787-35710612521861/AnsiballZ_switch_vlans.py
<localhost> EXEC /bin/sh -c 'chmod u+x /tmp/ansible-stack/ansible-tmp-1560779449.07787-35710612521861/ /tmp/ansible-stack/ansible-tmp-1560779449.07787-35710612521861/AnsiballZ_switch_vlans.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python3 /tmp/ansible-stack/ansible-tmp-1560779449.07787-35710612521861/AnsiballZ_switch_vlans.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /tmp/ansible-stack/ansible-tmp-1560779449.07787-35710612521861/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible-stack/ansible-tmp-1560779449.07787-35710612521861/AnsiballZ_switch_vlans.py", line 114, in <module>
    _ansiballz_main()
  File "/tmp/ansible-stack/ansible-tmp-1560779449.07787-35710612521861/AnsiballZ_switch_vlans.py", line 106, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/tmp/ansible-stack/ansible-tmp-1560779449.07787-35710612521861/AnsiballZ_switch_vlans.py", line 49, in invoke_module
    imp.load_module('__main__', mod, module, MOD_DESC)
  File "/usr/lib64/python3.6/imp.py", line 235, in load_module
    return load_source(name, filename, file)
  File "/usr/lib64/python3.6/imp.py", line 170, in load_source
    module = _exec(spec, sys.modules[name])
  File "<frozen importlib._bootstrap>", line 618, in _exec
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/tmp/ansible_switch_vlans_payload_kr3f9l85/__main__.py", line 225, in <module>
  File "/tmp/ansible_switch_vlans_payload_kr3f9l85/__main__.py", line 209, in main
  File "/tmp/ansible_switch_vlans_payload_kr3f9l85/__main__.py", line 208, in <dictcomp>
ValueError: too many values to unpack (expected 2)

fatal: [undercloud]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible-stack/ansible-tmp-1560779449.07787-35710612521861/AnsiballZ_switch_vlans.py\", line 114, in <module>\n    _ansiballz_main()\n  File \"/tmp/ansible-stack/ansible-tmp-1560779449.07787-35710612521861/AnsiballZ_switch_vlans.py\", line 106, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/tmp/ansible-stack/ansible-tmp-1560779449.07787-35710612521861/AnsiballZ_switch_vlans.py\", line 49, in invoke_module\n    imp.load_module('__main__', mod, module, MOD_DESC)\n  File \"/usr/lib64/python3.6/imp.py\", line 235, in load_module\n    return load_source(name, filename, file)\n  File \"/usr/lib64/python3.6/imp.py\", line 170, in load_source\n    module = _exec(spec, sys.modules[name])\n  File \"<frozen importlib._bootstrap>\", line 618, in _exec\n  File \"<frozen importlib._bootstrap_external>\", line 678, in exec_module\n  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n  File \"/tmp/ansible_switch_vlans_payload_kr3f9l85/__main__.py\", line 225, in <module>\n  File \"/tmp/ansible_switch_vlans_payload_kr3f9l85/__main__.py\", line 209, in main\n  File \"/tmp/ansible_switch_vlans_payload_kr3f9l85/__main__.py\", line 208, in <dictcomp>\nValueError: too many values to unpack (expected 2)\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

PLAY RECAP ********************************************************************************************************************************************************************************************************
undercloud                 : ok=3    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0   

(undercloud) [stack@undercloud temp1]$

Hi Alexander, 

Sorry I didn't mention it sooner that during the Stein cycle we refactored all the tripleo-validations architecture.

All the contents of the validations directory are now obsolete and will be removed soon!

All the validations have been migrated into Ansible roles (/usr/share/openstack-tripleo-validations/roles/) and
you can run the playbooks from /usr/share/openstack-tripleo-validations/playbooks/.

Please use the run-validations.sh script which should be present in /bin of the undercloud. Otherwise, you can grab it 
here https://github.com/openstack/tripleo-validations/blob/master/scripts/run-validations.sh and run:

```
$ ./run-validations.sh --plan my-templates --validation-name switch-vlans
```

Sorry for the confusion!

Gaël

Verified:

Environment: openstack-tripleo-validations-10.4.1-0.20190626140421.fae8a85.el8ost.noarch

The reported issue doesn't reproduce.
+ ansible-playbook -vvv -i /usr/bin/tripleo-ansible-inventory /usr/share/openstack-tripleo-validations/validations/switch-vlans.yaml
+ grep 'VLAN ID 193 not on attached switch'
    "msg": "VLAN ID 193 not on attached switch\nVLAN ID 193 not on attached switch\nVLAN ID 193 not on attached switch"
+ '[' 0 '!=' 0 ']'

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:2811