Created attachment 1114720 [details] templates dir Description of problem: RHEL 7.2 When I'm trying to deploy the overcloud HA 3 controllers and 1 compute and I'm getting the following error: 2016-01-13 16:01:50 [MysqlRootPassword]: CREATE_COMPLETE state changed 2016-01-13 16:01:50 [PcsdPassword]: CREATE_COMPLETE state changed 2016-01-13 16:01:50 [overcloud]: CREATE_FAILED Resource CREATE failed: resources.InternalNetwork: resources.Networks.Property error: resources.InternalApiSubnet.properties.allocation_pools[0].start: "fd00: fd00: fd00: 2000: 0000: 0000: 0000: 10" does not validate ip_addr (constraint not found) Stack overcloud CREATE_IN_PROGRESS Stack overcloud CREATE_FAILED Heat Stack create failed. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cliff/app.py", line 374, in run_subcommand result = cmd.run(parsed_args) File "/usr/lib/python2.7/site-packages/cliff/command.py", line 54, in run self.take_action(parsed_args) File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 977, in take_action self._deploy_tripleo_heat_templates(stack, parsed_args) File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 344, in _deploy_tripleo_heat_templates environments, parsed_args.timeout) File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 276, in _heat_deploy raise Exception("Heat Stack create failed.") Exception: Heat Stack create failed. clean_up DeployOvercloud: Heat Stack create failed. Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 112, in run ret_val = super(OpenStackShell, self).run(argv) File "/usr/lib/python2.7/site-packages/cliff/app.py", line 255, in run result = self.run_subcommand(remainder) File "/usr/lib/python2.7/site-packages/cliff/app.py", line 374, in run_subcommand result = cmd.run(parsed_args) File "/usr/lib/python2.7/site-packages/cliff/command.py", line 54, in run self.take_action(parsed_args) File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 977, in take_action self._deploy_tripleo_heat_templates(stack, parsed_args) File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 344, in _deploy_tripleo_heat_templates environments, parsed_args.timeout) File "/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 276, in _heat_deploy raise Exception("Heat Stack create failed.") Exception: Heat Stack create failed. END return value: 1 Version-Release number of selected component (if applicable): How reproducible: 100% Steps to Reproduce: 1. Try to deploy overcloud HA with 3 Controllers + 1 Compute 2. 3. Actual results: Deployment failed Expected results: Deployment passed Additional info: Deploy command is: openstack overcloud deploy --templates $THT -e $THT/environments/network-isolation-v6.yaml -e $THT/environments/net-single-nic-with-vlans.yaml -e /home/stack/templates/network-environment.yaml --control-scale 3 --compute-scale 1 --neutron-network-type vxlan --neutron-tunnel-types vxlan --libvirt-type qemu --timeout 180 --ntp-server clock.redhat.com Templates Dir is attached.
Steve, could you have a look at this, it seems like it might be a simple parameter validation thing.
Ok, so from looking at the heat code, it seems we validate this via oslo.netutils: https://github.com/openstack/oslo.utils/blob/master/oslo_utils/netutils.py#L168 So one thing to confirm is the version if oslo.utils, and check by inspection that netutils.py on the platform has the expected IPv6 functions (don't look like recent additions from the git history so it should). However, the log above looks suspicious: resources.InternalApiSubnet.properties.allocation_pools[0].start: "fd00: fd00: fd00: 2000: 0000: 0000: 0000: 10" does not validate ip_addr (constraint not found) The IPv6 address is split by spaces, which would cause this: >>> from oslo_utils import netutils >>> a = "fd00: fd00: fd00: 2000: 0000: 0000: 0000: 10" >>> netutils.is_valid_ip(a) False >>> b = "fd00:fd00:fd00:2000:0000:0000:0000:10" >>> netutils.is_valid_ip(b) True So, we need to figure out where those spaces are coming from, in the template the parameter looks like this, and from the attached tarball, the error appears to be from the defaults vs a user-provided value: InternalApiAllocationPools: default: [{'start': 'fd00:fd00:fd00:2000::10', 'end': 'fd00:fd00:fd00:2000:ffff:ffff:ffff:fffe'}] description: Ip allocation pool range for the internal API network. type: json
Hi Udi, I've tried reproducing this with the attached minimal reproducer (just the network/subnet), on upstream master it works fine and I cannot reproduce. Also, the report is against OSP7/Kilo, but the constraint failing does not exist in kilo heat, which is confusing: https://github.com/openstack/heat/blob/stable/kilo/heat/engine/resources/openstack/neutron/subnet.py#L132 vs https://github.com/openstack/heat/blob/stable/liberty/heat/engine/resources/openstack/neutron/subnet.py#L145 Please confirm, is the undercloud OSP7 (kilo) or OSP8 (liberty)? Also, please confirm the version of oslo.utils and the version of all heat packages so I can be sure what version you're testing, thanks!
Minimal reproducer: http://paste.fedoraproject.org/310613/66427145/raw/ Try downloading that, then do: heat stack-create test -f the_template.yaml Ensure the overcloud has been deleted before trying this - on master TripleO this works fine for me.
(In reply to Steven Hardy from comment #5) > Minimal reproducer: > > http://paste.fedoraproject.org/310613/66427145/raw/ > > Try downloading that, then do: > > heat stack-create test -f the_template.yaml > > Ensure the overcloud has been deleted before trying this - on master TripleO > this works fine for me. It worked ok on a 7.2 director environment: openstack-heat-engine-2015.1.2-4.el7ost.noarch heat-cfntools-1.2.8-2.el7.noarch openstack-tripleo-heat-templates-0.8.6-94.el7ost.noarch openstack-heat-api-2015.1.2-4.el7ost.noarch openstack-heat-templates-0-0.8.20150605git.el7ost.noarch openstack-heat-common-2015.1.2-4.el7ost.noarch openstack-heat-api-cfn-2015.1.2-4.el7ost.noarch python-heatclient-0.6.0-1.el7ost.noarch openstack-heat-api-cloudwatch-2015.1.2-4.el7ost.noarch stack@instack:~>>> heat stack-show test +-----------------------+--------------------------------------------------------------------------------------------------------------------------------------+ | Property | Value | +-----------------------+--------------------------------------------------------------------------------------------------------------------------------------+ | capabilities | [] | | creation_time | 2016-01-14T10:23:38Z | | description | Internal API network. Used for most APIs, Database, | | | RPC. | | disable_rollback | True | | id | 185ebc16-cf6b-4277-9303-6fd6eb60bfc7 | | links | http://192.0.2.1:8004/v1/bba90a9c3682484ea6acf64484f56334/stacks/test/185ebc16-cf6b-4277-9303-6fd6eb60bfc7 (self) | | notification_topics | [] | | outputs | [ | | | { | | | "output_value": "5f2151b1-0eef-49ff-84c6-96928eded09e", | | | "description": "Neutron internal network", | | | "output_key": "OS::stack_id" | | | } | | | ] | | parameters | { | | | "OS::project_id": "bba90a9c3682484ea6acf64484f56334", | | | "InternalApiSubnetName": "internal_api_subnet", | | | "OS::stack_id": "185ebc16-cf6b-4277-9303-6fd6eb60bfc7", | | | "OS::stack_name": "test", | | | "InternalApiNetCidr": "fd00:fd00:fd00:2000::/64", | | | "InternalApiAllocationPools": "[{\"start\": \"fd00:fd00:fd00:2000::10\", \"end\": \"fd00:fd00:fd00:2000:ffff:ffff:ffff:fffe\"}]", | | | "InternalApiNetValueSpecs": "{\"provider:physical_network\": \"internal_api\", \"provider:network_type\": \"flat\"}", | | | "InternalApiNetShared": "False", | | | "InternalApiNetAdminStateUp": "False", | | | "InternalApiNetName": "internal_api" | | | } | | parent | None | | stack_name | test | | stack_owner | admin | | stack_status | CREATE_COMPLETE | | stack_status_reason | Stack CREATE completed successfully | | stack_user_project_id | bfcf4eba3981475da17d241a062f448c | | template_description | Internal API network. Used for most APIs, Database, | | | RPC. | | timeout_mins | None | | updated_time | None | +-----------------------+--------------------------------------------------------------------------------------------------------------------------------------+
Hi, The version is OSP 7. I'll update the bug with the required information once we will start testing on OSP8 Thanks, Udi
Please re-open if this appears to be an issue. Based on tested version and Marius's comment, I am closing for now.
Hi, I'm getting same error in open stack liberty. [root@controller prerequisites]# heat stack-create -f abc.hot.orc.yaml -Pf orc=abc.hot.orc.json -P prefix=abcPreReqStack prerequisiteStack ERROR: Property error: : resources.internal_network_for_ipv6_subnet.properties.allocation_pools[0].start: : "192.168.4.100" does not validate ip_addr (constraint not found) on heat-engine.log 2016-09-16 19:35:13.141 23433 INFO heat.engine.resource [req-6dd072b1-560d-41af-a4eb-6045fc164c82 7d2a7dcaa26040e189cce72b412e35d8 b6bafa4dc1c948e99dc2afadfa4c68bb] Validating Subnet "internal_network_for_ipv6_subnet" error is from the following abc.hot.orc.yaml content internal_network_for_ipv6_subnet: type: OS::Neutron::Subnet depends_on: [ internal_network_for_ipv6 ] properties: name: { list_join: [ "-", [ { get_param: prefix }, ipv6 ] ] } network: { get_resource: internal_network_for_ipv6 } cidr: { get_param: [orc, externalVirtualLinks, ipv6, cidr ] } allocation_pools: [ {"start": { get_param: [orc, externalVirtualLinks, ipv6, pool_start ] }, "end": { get_param: [orc, externalVirtualLinks, ipv6, pool_end ] } }] gateway_ip: null Let me know if more information is needed.
contents of json file { "externalVirtualLinks": { "ipv6": { "cidr": "192.168.4.0/24", "pool_start" : "192.168.4.100", "pool_end" : "192.168.4.253" } } }
heat-engine logs show following ======================================= 2016-09-17 14:57:44.789 8068 ERROR stevedore.extension [-] Could not load 'zaqar': No module named zaqarclient.queues.v1 2016-09-17 14:57:44.805 8068 ERROR stevedore.extension [-] Could not load 'mistral': No module named mistralclient.api 2016-09-17 14:57:44.812 8068 ERROR stevedore.extension [-] Could not load 'manila': No module named manilaclient 2016-09-17 14:57:44.813 8068 ERROR stevedore.extension [-] Could not load 'magnum': No module named magnumclient.openstack.common.apiclient 2016-09-17 14:57:44.815 8068 ERROR stevedore.extension [-] Could not load 'barbican': No module named barbicanclient 2016-09-17 14:57:44.845 8068 ERROR stevedore.extension [-] Could not load 'designate': No module named designateclient 2016-09-17 14:57:44.846 8068 ERROR stevedore.extension [-] Could not load 'designate.domain': No module named designateclient 2016-09-17 14:57:44.847 8068 ERROR stevedore.extension [-] Could not load 'ip_addr': No module named croniter 2016-09-17 14:57:44.848 8068 ERROR stevedore.extension [-] Could not load 'manila.share_type': No module named manilaclient 2016-09-17 14:57:44.848 8068 ERROR stevedore.extension [-] Could not load 'timezone': No module named croniter 2016-09-17 14:57:44.849 8068 ERROR stevedore.extension [-] Could not load 'net_cidr': No module named croniter 2016-09-17 14:57:44.850 8068 ERROR stevedore.extension [-] Could not load 'iso_8601': No module named croniter 2016-09-17 14:57:44.850 8068 ERROR stevedore.extension [-] Could not load 'manila.share_network': No module named manilaclient 2016-09-17 14:57:44.851 8068 ERROR stevedore.extension [-] Could not load 'manila.share_snapshot': No module named manilaclient 2016-09-17 14:57:44.852 8068 ERROR stevedore.extension [-] Could not load 'mac_addr': No module named croniter ip_addr plugin not present.. installing python-crontier and restarting heat engine solves the issue