Bug 1378029 - OSP9/10 Init stage fails: Incorrect arguments to "list_join" should be: "list_join" [NEEDINFO]
Summary: OSP9/10 Init stage fails: Incorrect arguments to "list_join" should be: "list...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: ---
: 10.0 (Newton)
Assignee: mathieu bultel
QA Contact: Arik Chernetsky
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-09-21 11:02 UTC by Sofer Athlan-Guyot
Modified: 2020-06-11 13:02 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-10-13 10:58:27 UTC
Target Upstream Version:
ipetrova: needinfo? (mbultel)


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1620696 0 None None None 2016-09-21 11:02:37 UTC
OpenStack gerrit 370069 0 None None None 2016-09-21 11:08:15 UTC

Description Sofer Athlan-Guyot 2016-09-21 11:02:37 UTC
Description of problem:  Running the init stage of the upgrade fails with UPDATE_FAILED .enabled_services.list_join: Incorrect arguments to "list_join" should be: "list_join" : [ " ", [ "str1", "str2"]]

How reproducible: always

Comment 2 Sofer Athlan-Guyot 2016-09-21 11:08:15 UTC
Adding upstream review.

Comment 4 Jaromir Coufal 2016-10-10 03:40:03 UTC
Seemsnot to be reproducable regarding the upstrema comments, Matthie can you update the bug accordingly and close if not reproducable?

Comment 5 Marios Andreou 2016-10-13 10:58:27 UTC
I am going to close this as not a bug based on the upstream bug comments (TBD if what Graeme reports is due to old packages?)

Please reopen if you disagree

Comment 6 Marios Andreou 2016-10-13 11:00:17 UTC
adding @matbu since he worked on the fix when we thought we needed one - are we OK to close this now. I believe I also hit this at the time but have since not encountered it; could have been addressed by one of the client/commmon patches?

Comment 7 Sofer Athlan-Guyot 2016-10-14 05:33:24 UTC
I confirm that we are not using any upstream patch related to this bug.  Good to close on my side.

Comment 8 Graeme Gillies 2016-10-20 23:06:32 UTC
Hi,

Can we get this bug re-opened? I'm still experiencing it 100% of the time trying to upgrade from mitaka to newton (9-10) using the latest packages.

I can provide access to the environment if needed to see for yourself.

Regards,

Graeme

Comment 10 Sean Lee 2017-01-12 22:23:52 UTC
This appears to be caused by Join() vs JoinMultiple() in heat/engine/hot/functions.py.

If I understand the code correctly, per heat/engine/hot/template.py, template versions older than 20151015 would set list_join to use Join(). 20151015 and newer would use JoinMultiple().

Join() expects one list:

    self._delim, self._strings = self.args

JoinMultiple() supports more than one list:

    self._delim = args[0]
    self._joinlists = args[1:]

"enabled_services" (overcloud.j2.yaml) is trying to join multiple lists. So use of Join() would fail.

The 'Incorrect arguments to "list_join"' message is misleading. If you replace the line in the code with:

    raise ValueError(traceback.format_exc())

you would see the actual python error:

    ValueError: too many values to unpack

Comment 11 Irina Petrova 2017-04-10 10:04:25 UTC
Hi, we have a new customer hitting the bug:

~~~ heat-engine.log 
2017-04-07 11:56:29.792 5409 INFO heat.engine.stack [req-83fcf90b-6675-400e-b2d8-4dd6aea2e213 - - - - -] Stack UPDATE FAILED (overcloud): resources.allNodesConfig.properties.enabled_services.list_join: Incorrect arguments to "list_join" should be: "list_join" : [ " ", [ "str1", "str2"]]    
~~~

$ grep heat sosreport-20170406-153529/gxospdir41.isamgmt.local/installed-rpms 
heat-cfntools-1.3.0-2.el7ost.noarch                         Mon Apr  3 16:39:59 2017
openstack-heat-api-7.0.2-1.el7ost.noarch                    Mon Apr  3 16:40:36 2017
openstack-heat-api-cfn-7.0.2-1.el7ost.noarch                Mon Apr  3 16:40:36 2017
openstack-heat-api-cloudwatch-7.0.2-1.el7ost.noarch         Mon Apr  3 16:40:36 2017
openstack-heat-common-7.0.2-1.el7ost.noarch                 Mon Apr  3 16:40:16 2017
openstack-heat-engine-7.0.2-1.el7ost.noarch                 Mon Apr  3 16:40:36 2017
openstack-heat-templates-0-0.11.1e6015dgit.el7ost.noarch    Mon Apr  3 16:39:53 2017
openstack-tripleo-heat-templates-5.2.0-3.el7ost.noarch      Mon Apr  3 16:39:26 2017
openstack-tripleo-heat-templates-compat-2.0.0-41.el7ost.noarch Mon Apr  3 16:41:50 2017
puppet-heat-9.5.0-1.el7ost.noarch                           Mon Apr  3 16:39:27 2017
python-heat-agent-0-0.11.1e6015dgit.el7ost.noarch           Mon Apr  3 16:40:03 2017
python-heat-tests-7.0.2-1.el7ost.noarch                     Mon Apr  3 16:40:36 2017
python-heatclient-1.5.0-1.el7ost.noarch                     Mon Apr  3 16:39:55 2017

$ openstack stack template show overcloud | grep heat_template_version
heat_template_version: '2015-04-30'


Please advise/comment.

Comment 12 mathieu bultel 2017-04-10 13:59:30 UTC
Hi, thank you for additional information.
I checked the latest 10 package (from the undercloud) and I got this:
rpm -qa | grep heat
python-heat-agent-0-0.11.1e6015dgit.el7ost.noarch
heat-cfntools-1.3.0-2.el7ost.noarch
openstack-heat-templates-0-0.11.1e6015dgit.el7ost.noarch
openstack-tripleo-heat-templates-5.2.0-9.el7ost.noarch
openstack-heat-api-cfn-7.0.2-4.el7ost.noarch
python-heat-tests-7.0.2-4.el7ost.noarch
openstack-heat-api-7.0.2-4.el7ost.noarch
python-heatclient-1.5.0-2.el7ost.noarch
openstack-heat-common-7.0.2-4.el7ost.noarch
openstack-heat-engine-7.0.2-4.el7ost.noarch
puppet-heat-9.5.0-1.el7ost.noarch


So few packages is newer than what the customer has on this undercloud.
I suggest first, (and this is what we recommend too), to do a minor to the very latest 10 packages (on the undercloud).
Then it would be nice to see if the issue is still happening. I think it's the first very simple step to do before jumping in larger debugging.


Note You need to log in before you can comment on or make changes to this bug.