Bug 1514949 - /etc/os-net-config/config.json is empty [NEEDINFO]
Summary: /etc/os-net-config/config.json is empty
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-common
Version: 11.0 (Ocata)
Hardware: Unspecified
OS: Unspecified
Target Milestone: beta
: 13.0 (Queens)
Assignee: Bob Fournier
QA Contact: Alexander Chuzhoy
Depends On:
Blocks: 1473267
TreeView+ depends on / blocked
Reported: 2017-11-19 16:37 UTC by David Hill
Modified: 2021-09-09 12:50 UTC (History)
13 users (show)

Fixed In Version: openstack-tripleo-heat-templates-8.0.2-0.20180414062830.5f869f2.el7ost openstack-tripleo-common-8.6.1-0.20180410165748.4d8ca16.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2018-06-27 13:39:31 UTC
Target Upstream Version:
bfournie: needinfo? (ebarrera)

Attachments (Terms of Use)
screenshot of the config.json (74.43 KB, image/png)
2017-11-19 18:23 UTC, David Hill
no flags Details

System ID Private Priority Status Summary Last Updated
Launchpad 1758161 0 None None None 2018-03-23 00:12:25 UTC
OpenStack gerrit 470017 0 'None' MERGED Install os-net-config as a package, not an element 2021-01-22 00:06:32 UTC
OpenStack gerrit 557739 0 'None' MERGED Remove the os-net-config element config.json template 2021-01-22 00:06:32 UTC
OpenStack gerrit 560022 0 'None' MERGED Remove config.json for old style nic config files when using new style 2021-01-22 00:06:32 UTC
Red Hat Product Errata RHEA-2018:2086 0 None None None 2018-06-27 13:40:23 UTC

Description David Hill 2017-11-19 16:37:14 UTC
Description of problem:
/etc/os-net-config/config.json is empty but network is properly configured in RHOSP 11.   I'm able to reproduce this on my side.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Deploy overcloud
2. cat /etc/os-net-config/config.json after the network is configured

Actual results:

Expected results:

Additional info:
Big Switch plugin relies on this to properly configure itself

Comment 1 David Hill 2017-11-19 16:57:26 UTC
Shouldn't it be something like this instead? 

--- run-os-net-config.sh.orig	2017-11-19 11:48:40.295720753 -0500
+++ run-os-net-config.sh	2017-11-19 11:51:17.024318363 -0500
@@ -109,7 +109,7 @@
-if [ -n '$network_config' ]; then
+if [ -n "$network_config" ]; then
     if [ -z "${disable_configure_safe_defaults:-''}" ]; then
         trap configure_safe_defaults EXIT

Comment 2 David Hill 2017-11-19 18:19:36 UTC
Something runs and remove the content of /etc/os-net-config/config.json after the network deployment is completed.   I connected to one of the overcloud node and we can see /etc/os-net-config/config.json is actually properly populated with the network configuration json data and after a while, the file is emptied ...  The above patch didn't solve this.

Comment 3 David Hill 2017-11-19 18:23:04 UTC
Created attachment 1355332 [details]
screenshot of the config.json

Comment 4 David Hill 2017-11-19 19:54:38 UTC
It looks like os-apply-config removes the content of /etc/os-net-config/config.json ...

Comment 5 David Hill 2017-11-19 20:00:06 UTC
And it looks like if we delete the template in /usr/libexec-os-apply-config/templates/etc/os-net-config/config.json, it no longer overwrites our file.   I'm wondering if this is a tripleo issue, os-net-config or os-apply-config issue ... since the behavior changed in RHOSP 11, I guess we can't easily fix this by remove the file from the RPM if the same RPM is needed in RHOSP 10 and that we're using that template.

Comment 8 David Hill 2017-11-20 16:58:29 UTC
One confirmed workaround here is to delete the file using virt-customize:

virt-customize -a overcloud-full.qcow2 --delete /usr/libexec/os-apply-config/templates/etc/os-net-config/config.json

We tested this and it solved the problem we experienced before.

Comment 9 Bob Fournier 2017-11-20 18:40:55 UTC
I assume this is the src location of this file - ./instack-undercloud/elements/undercloud-install/os-apply-config/etc/os-net-config/config.json.

Its not clear what affect there is, if any, of removing this from the undercloud install. Investigating...

Comment 11 Bob Fournier 2017-11-20 19:18:16 UTC
Note also the documentation changes here to describe changes to the os-net-config element - https://review.openstack.org/#/c/521197/.

Changing the target release for this bug as there is a workaround for OSP-11.

Comment 13 Bob Fournier 2018-03-22 20:00:47 UTC
Problem appears to not be fixed, so reopening.

We are seeing /etc/os-net-config/config.json empty and os-net-config being invoked by os-apply-config after the network has been configured:
messages:Mar 22 13:59:09 localhost os-collect-config: [2018/03/22 01:58:36 PM] [INFO] Using config file at: /etc/os-net-config/config.json
messages:Mar 22 13:59:20 localhost os-collect-config: [2018/03/22 01:59:20 PM] [INFO] writing /etc/os-net-config/config.json
messages:Mar 22 13:59:30 localhost os-collect-config: [2018/03/22 01:59:30 PM] [INFO] writing /etc/os-net-config/config.json
messages:Mar 22 13:59:43 localhost os-collect-config: [2018/03/22 01:59:43 PM] [INFO] writing /etc/os-net-config/config.json
messages:Mar 22 13:59:55 localhost os-collect-config: [2018/03/22 01:59:55 PM] [INFO] writing /etc/os-net-config/config.json
messages:Mar 22 14:00:18 localhost os-collect-config: [2018/03/22 02:00:18 PM] [INFO] writing /etc/os-net-config/config.json
messages:Mar 22 14:00:33 localhost os-collect-config: [2018/03/22 02:00:33 PM] [INFO] writing /etc/os-net-config/config.json
messages:Mar 22 14:00:46 localhost os-collect-config: [2018/03/22 02:00:46 PM] [INFO] writing /etc/os-net-config/config.json
messages:Mar 22 14:01:03 localhost os-collect-config: [2018/03/22 02:01:03 PM] [INFO] writing /etc/os-net-config/config.json
messages:Mar 22 14:01:23 localhost os-collect-config: [2018/03/22 02:01:23 PM] [INFO] writing /etc/os-net-config/config.json

Its likely that we need to remove the creation of config.json under libexec here:

Comment 14 Bob Fournier 2018-03-22 23:53:59 UTC
Fix upstream will be backported to stab;e/queens - https://review.openstack.org/#/c/555452/

Comment 19 Alexander Chuzhoy 2018-05-07 20:14:10 UTC


The mentioned file is not empty on OC node:
[root@overcloud-cmp-leaf0-0 ~]# cat /etc/os-net-config/config.json 
{"network_config": [{"addresses": [{"ip_netmask": ""}], "dns_servers": [""], "name": "nic1", "routes": [{"ip_netmask": "", "next_hop": ""}, {"default": true, "next_hop": ""}], "type": "interface", "use_dhcp": false}, {"members": [{"name": "nic2", "primary": true, "type": "interface"}, {"addresses": [{"ip_netmask": ""}], "routes": [{"ip_netmask": "", "next_hop": ""}], "type": "vlan", "vlan_id": 1183}, {"addresses": [{"ip_netmask": ""}], "routes": [{"ip_netmask": "", "next_hop": ""}], "type": "vlan", "vlan_id": 1185}, {"addresses": [{"ip_netmask": ""}], "routes": [{"ip_netmask": "", "next_hop": ""}], "type": "vlan", "vlan_id": 1189}], "name": "br-ex", "type": "ovs_bridge", "use_dhcp": false}]}

Comment 20 Eduard Barrera 2018-05-13 15:51:04 UTC
I'm experiencing the same issue on 11, but the workaround deleting a file from the overcloud-full image didn't work. Now there is no file at all instead of one empty file.

I can see this on logs:

May 13 11:44:46 overcloud-ceph-env1-0 os-collect-config[2936]: dib-run-parts Sun May 13 11:44:46 EDT 2018 Running /usr/libexec/os-refresh-config/configure.d/20-os-apply-config
May 13 11:44:46 overcloud-ceph-env1-0 os-collect-config[2936]: [2018/05/13 11:44:46 AM] [INFO] writing /var/run/heat-config/heat-config
May 13 11:44:46 overcloud-ceph-env1-0 os-collect-config[2936]: [2018/05/13 11:44:46 AM] [INFO] writing /etc/os-collect-config.conf
May 13 11:44:46 overcloud-ceph-env1-0 os-collect-config[2936]: [2018/05/13 11:44:46 AM] [INFO] success
May 13 11:44:46 overcloud-ceph-env1-0 os-collect-config[2936]: dib-run-parts Sun May 13 11:44:46 EDT 2018 20-os-apply-config completed
May 13 11:44:46 overcloud-ceph-env1-0 os-collect-config[2936]: dib-run-parts Sun May 13 11:44:46 EDT 2018 Running /usr/libexec/os-refresh-config/configure.d/20-os-net-config
May 13 11:44:46 overcloud-ceph-env1-0 os-collect-config[2936]: ++ os-apply-config --key os_net_config --type raw --key-default ''
May 13 11:44:47 overcloud-ceph-env1-0 os-collect-config[2936]: + NET_CONFIG=
May 13 11:44:47 overcloud-ceph-env1-0 os-collect-config[2936]: + '[' -n '' ']'
May 13 11:44:47 overcloud-ceph-env1-0 os-collect-config[2936]: dib-run-parts Sun May 13 11:44:47 EDT 2018 20-os-net-config completed

Comment 22 Bob Fournier 2018-05-13 18:04:49 UTC
Hi Eduard - there is a separate issue reported in OSP-11 - https://bugzilla.redhat.com/show_bug.cgi?id=1574473.  Is this what you are hitting?
Also are you using old-style nic config files or the new style, i.e. with the run-os-net-config.sh script?

As this bug is targeted for osp-13 we should either track what you are seeing in 1574473 or create a new bug targeted to OSP-11. Thanks.

Comment 24 errata-xmlrpc 2018-06-27 13:39:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.