Bug 1227955 - [docs] [networking] [overcloud] os-net-config is registering only NICs that are UP which is wrong.
Summary: [docs] [networking] [overcloud] os-net-config is registering only NICs that a...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: documentation
Version: Director
Hardware: Unspecified
OS: Unspecified
high
urgent
Target Milestone: ---
: 12.0 (Pike)
Assignee: Martin Lopes
QA Contact: Dan Macpherson
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-06-03 23:09 UTC by Leonid Natapov
Modified: 2023-02-22 23:02 UTC (History)
19 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
Only NICs with an active connection to a switch port will be counted in the numbered NIC abstractions (such as `nic1` or `nic2`). As a workaround, the director includes a script that pings the first controller on all interfaces from each node; if a node has a disconnected link at deployment time, this is detected so that it can be corrected. Another possible workaround is to use a mapping file for each host; which has a mapping of NIC number to physical NIC. If one or more overcloud nodes are not configured correctly due to a down link, this is now detected and the overcloud process can re-deploy.
Clone Of:
Environment:
Last Closed: 2019-03-19 10:09:00 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 265014 0 None MERGED Add example of creating per-node os-net-config mappings 2021-01-07 20:20:40 UTC
Red Hat Bugzilla 1437320 0 medium CLOSED os-net-config mapping.yaml will only map active interfaces, but no-carrier interfaces need to be mapped as well 2024-01-06 04:26:58 UTC

Description Leonid Natapov 2015-06-03 23:09:19 UTC
Description of problem:
overcloud: os-net-config is registering only NICs that are UP which is wrong.

I have a box that has has em1, em2, p1p1, and p1p2. 
os-net-config register only 3 NICs instead of all 4 NICs.

Jun  3 18:00:19 localhost os-collect-config: [2015/06/03 06:00:19 PM] [INFO] Using config file at: /etc/os-net-config/config.json
Jun  3 18:00:19 localhost os-collect-config: [2015/06/03 06:00:19 PM] [INFO] Using mapping file at: /etc/os-net-config/mapping.yaml
Jun  3 18:00:19 localhost os-collect-config: [2015/06/03 06:00:19 PM] [INFO] Ifcfg net config provider created.
Jun  3 18:00:19 localhost os-collect-config: [2015/06/03 06:00:19 PM] [INFO] nic1 mapped to: em1
Jun  3 18:00:19 localhost os-collect-config: [2015/06/03 06:00:19 PM] [INFO] nic2 mapped to: p1p1
Jun  3 18:00:19 localhost os-collect-config: [2015/06/03 06:00:19 PM] [INFO] nic3 mapped to: p1p2


nic3 should be p1p1, but os-net-config is not registering em2.

When I checked all NICs I saw that em2 is DOWN and all other NICs are UP.

4: em2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000

Comment 1 Leonid Natapov 2015-06-03 23:10:21 UTC
Happens on bare metal environment.

Comment 2 Dan Prince 2015-06-05 02:07:45 UTC
This sounds like the intended behavior for os-net-config because by default we are only looking for active NICs on the system:

http://git.openstack.org/cgit/openstack/os-net-config/tree/os_net_config/utils.py#n52

This sort of assumes that any desired cabling changes are done ahead of time.

That said, we do support a strict mapping mechanism such that the NIC naming can be passed in explicitly. See this file format:

http://git.openstack.org/cgit/openstack/os-net-config/tree/etc/os-net-config/samples/mapping.yaml

Using os-net-config on the CLI we can pass this in with -m.

----

The end goal is for the mapping file to be generated by something like an Ironic plugin (edeploy discovery plugin)... and then we should know for sure which nic is mapped to nic1, nic2, ....

Comment 3 chris alfonso 2015-06-16 17:50:24 UTC
Leonid, can you look at this one again after reviewing comment #2

Comment 5 Leonid Natapov 2015-06-22 13:23:39 UTC
Chris,
How exactly do I need to check this issue ? I understand that os-net-config  is meant to only configure the active NICs on the system. I have tried to check the -m switch but couldn't find yaml file in the /etc/os-net-config directory.

Comment 6 chris alfonso 2015-06-22 14:51:26 UTC
Dan, can you provide guidance or a link to docs that describe how the mapping.yaml file should be used/created?

Comment 7 Dan Prince 2015-12-06 21:52:47 UTC
I'm not sure we document the usage for this feature in tripleo-docs. But we do have a sample mapping file in the os-net-config source code:

http://git.openstack.org/cgit/openstack/os-net-config/tree/etc/os-net-config/samples/mapping.yaml

Comment 8 Jaromir Coufal 2016-01-07 09:28:40 UTC
Moving to documentation team to cover this usage. Dan, would you be able to help docs team if they have questions?

Comment 9 Cyril Lopez 2016-02-08 12:47:54 UTC
Hello

I thinks is not a documentation part, is related to this commit https://review.openstack.org/#/c/265014

Could be better to use that regarding upgrade processing.

Regards

Comment 12 Ben Nemec 2017-03-08 18:11:07 UTC
Seems like a better fit for Networking DFG.

Comment 13 Assaf Muller 2017-03-15 19:18:40 UTC
As far as I know os-net-config is HardProv's baby.

Comment 14 Dan Prince 2017-03-16 11:03:49 UTC
(In reply to Jaromir Coufal from comment #8)
> Moving to documentation team to cover this usage. Dan, would you be able to
> help docs team if they have questions?

Sure, I'm up for helping with the documentation on this issue.

Dan

Comment 15 Dmitry Tantsur 2017-03-17 09:25:25 UTC
Bob, mind taking a look (feel free to assign further).

Comment 16 Bob Fournier 2017-03-17 14:16:32 UTC
Moving to Documentation. The doc text has already been written. I can help with any questions or changes to this text.

Comment 17 Bob Fournier 2017-04-10 13:08:46 UTC
See also https://bugzilla.redhat.com/show_bug.cgi?id=1437320, in which a fix is proposed to map the nics with no-carrier if they are defined in a mapping file.

Comment 18 Lucy Bopf 2017-04-24 02:10:42 UTC
Moving to 'NEW' to be triaged as resources allow.

Comment 19 Martin Lopes 2017-07-06 03:32:26 UTC
Looks like the best place for this is under `Isolating Networks` - 7.1. Creating Custom Interface Templates:

https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/11/html-single/advanced_overcloud_customization/#sect-Creating_Custom_Interface_Templates


This section includes coverage of os-net-config.

Comment 23 Bob Fournier 2017-07-06 14:34:01 UTC
I agree with Dan, the section under Configuring Interfaces in this doc explains that only interfaces which are UP will be used so putting it in the Creating Custom Interface Templates section is not necessary.

The description may need to be expanded to describe that a custom mapping file can be provided.  In that case the actual interface names will be mapped to the names in the mapping file regardless of the state of the link, i.e. even down links will be mapped.

Comment 24 Dan Macpherson 2017-07-06 14:47:48 UTC
(In reply to Bob Fournier from comment #23)
> The description may need to be expanded to describe that a custom mapping
> file can be provided.  In that case the actual interface names will be
> mapped to the names in the mapping file regardless of the state of the link,
> i.e. even down links will be mapped.

Hi Bob,

Is the mapping necessary? My understanding is that you can use the interface name ("eth0", "eth1", "enp0s25") instead of the standard "nic1", "nic2", etc, and that  the "nicX" convention is mainly used to standardize interfaces across nodes using different hardware (e.g. you have two compute nodes -- one using "ethX" and the other using "emX").

Comment 25 Bob Fournier 2017-07-06 14:55:48 UTC
Dan,

Yes exactly, the mapping convention done by os-net-config is used so that the actual interface names need not be known and can be different across nodes.  The mapping file is not necessary - os-net-config checks first to see if one exists and will use it if it does, otherwise it will do the standard, ordered mapping as you described.  The mapping file can help in situations, for example, in which you have more live interfaces on one node and you want to ensure that the mapping is done consistently.

Comment 26 Dan Macpherson 2017-07-06 15:26:27 UTC
Okay, I understand now. Is there a syntax for the mapping file? Would this just be a standard environment file that you include upon deployment?

Sorry for all these questions. :S

Comment 27 Bob Fournier 2017-07-06 15:42:04 UTC
Dan,

The syntax is just standard yaml file with mappings under "interface_mapping", for example:

cat mapping.yaml 
interface_mapping:
  nic1: em2 
  nic2: em1 
  nic3: a0:36:9f:bc:22:82
  nic4: a0:36:9f:bc:22:80
  nic5: 90:1b:0e:9a:cb:1c
  nic6: 90:1b:0e:9a:cb:1d

By default, os-net-config looks for the file at /etc/os-net-config/mapping.yaml
but this location can be overridden if you run os-net-config manually by the -m option.

You would have to generate the file, for example by a first boot script:https://github.com/openstack/tripleo-heat-templates/blob/master/firstboot/os-net-config-mappings.yaml

Note also, in example above, that the mapping file can be used in situations where you want to map the NICs mac address to a specific alias.

Comment 28 Dan Macpherson 2017-07-06 15:49:26 UTC
Ah, thanks Bob!

Martin, we should document how to implement these mappings in that appendix. I'll catch up with you in the office tomorrow in case you need a hand with documenting this.

Comment 31 Martin Lopes 2018-02-21 04:19:14 UTC
Bob has approved draft via email.

Comment 35 Bob Fournier 2018-07-11 20:24:01 UTC
Hi Dan - can this be closed?

Comment 36 Dan Macpherson 2018-07-16 06:38:30 UTC
Hi Bob,

I think this got lost in my bug filter. I'm currently revamping the networking section in that guide (there's a lot of out-of-date info) and will factor in this content. I'll get this resolved before the end of the week.

- Dan

Comment 37 Dan Macpherson 2018-12-17 01:47:52 UTC
Merged with original osp12 branch. For the osp13 and master, I added the content to an existing module on network interface mapping.

Comment 38 Martin Lopes 2019-03-19 10:09:00 UTC
Closing as per comment 37.


Note You need to log in before you can comment on or make changes to this bug.