RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1099951 - document configuring predictable ip address assignment for libvirt network
Summary: document configuring predictable ip address assignment for libvirt network
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: doc-Virtualization_Deployment_and_Administration_Guide
Version: 7.0
Hardware: Unspecified
OS: Unspecified
high
unspecified
Target Milestone: rc
: ---
Assignee: Platform Virt Docs
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 1286552
TreeView+ depends on / blocked
 
Reported: 2014-05-21 15:09 UTC by David Vossel
Modified: 2019-03-06 00:41 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-12-20 08:40:41 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description David Vossel 2014-05-21 15:09:16 UTC
Description of problem:

The HA docs team is writing a deployment guide for a HA product that is deployed within a virtual machine. As part of that guide, we mention that the deployment requires the virtual machines to be assigned static (predictable) ip addresses.  By default, the libvirt network assigns random ip addresses via dhcp to the vm guests. 

We need to give the user some sort of resource that describes how to configure the libvirt virtual network to assign static ip addresses to the guest vms.  We aren't confused on how this is done technically. We want some officially Red Hat libvirt documentation we can reference in our HA document to outline the process of configuring static ips.

If this has already been addressed please point us to the relevant docs. This issue was investigated a few months ago and we could not find any suitable documentation to reference related to this.

Thanks,
-- vossel

Comment 5 Charlie 2015-02-04 06:04:15 UTC
Hiya David,
I've looked through all our documents and can't find any suitable documentation there. I think the idea is to include the process in the Virtualization_Deployment_and_Administration_Guide. I did find some stuff on the libvirt wicki so I could make a draft section and then you could verify or edit the section. Of if you have some notes on the process that would be better.

Regards, Charles.

Comment 6 David Vossel 2015-02-11 16:11:10 UTC
What documentation did you find. The techniques I've been using is to modify the default libvirt network's dhcp assignments so certain IPs/hostnames are mapped to a virtual machine with a unique MAC address.

Example:
# virsh net-dumpxml default
<network connections='3'>
  <name>default</name>
  <uuid>41ebdb84-7134-1111-a136-91f0f1119225</uuid>
  <forward mode='nat'/>
  <bridge name='virbr0' stp='on' delay='0' />
  <mac address='52:54:00:A8:12:35'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254' />
      <host mac='52:54:31:21:33:35' name='rhel7-auto1' ip='192.168.122.171' />
      <host mac='52:54:60:75:45:60' name='rhel7-auto2' ip='192.168.122.172' />
      <host mac='52:54:25:57:46:74' name='rhel7-auto3' ip='192.168.122.173' />
      <host mac='52:54:74:46:78:14' name='rhel7-auto4' ip='192.168.122.174' />
      <host mac='52:54:23:47:82:51' name='rhel6-auto1' ip='192.168.122.161' />
      <host mac='52:54:07:05:83:52' name='rhel6-auto2' ip='192.168.122.162' />
      <host mac='52:54:63:84:77:15' name='rhel6-auto3' ip='192.168.122.163' />
      <host mac='52:54:78:70:35:82' name='rhel6-auto4' ip='192.168.122.164' />
      <host mac='52:54:65:11:45:07' name='rhel6-auto5' ip='192.168.122.165' />
      <host mac='52:54:61:32:57:24' name='rhel7-alt1' ip='192.168.122.181' />
      <host mac='52:54:30:71:30:55' name='rhel7-alt2' ip='192.168.122.182' />
      <host mac='52:54:13:55:75:84' name='rhel7-alt3' ip='192.168.122.183' />
      <host mac='52:54:64:31:65:07' name='rhel7-alt4' ip='192.168.122.184' />
      <host mac='52:54:35:76:66:37' name='rhel7-alt5' ip='192.168.122.185' />
    </dhcp>
  </ip>
</network>

Comment 7 Charlie 2015-03-12 01:12:25 UTC
Hiya David.
Sorry about the delay, it was actually this link I found doesn't say much.
https://www.redhat.com/archives/libvirt-users/2014-March/msg00110.html
I'd say the best thing is to create a new section in the Virtualization_Deployment_and_Administration_Guide using the information above. That way you can refer to it in an official Red Hat document. Do you agree?

Thanks.

Comment 8 David Vossel 2015-03-12 14:16:51 UTC
(In reply to Charlie from comment #7)
> Hiya David.
> Sorry about the delay, it was actually this link I found doesn't say much.
> https://www.redhat.com/archives/libvirt-users/2014-March/msg00110.html
> I'd say the best thing is to create a new section in the
> Virtualization_Deployment_and_Administration_Guide using the information
> above. That way you can refer to it in an official Red Hat document. Do you
> agree?

yes, that looks reasonable to me.

> Thanks.

Comment 16 Laine Stump 2015-12-08 16:09:01 UTC
1) "In cases where you need a predictable static IP address for your guest, the following procedure should be used. This procedure needs to be done for a guest that is already created."

Maybe instead say something like this:

"In cases where a guest is configured to acquire its IP address from DHCP, but you need it to have a predictable static IP address, you can use the following procedure to modify the DHCP server configuration used by libvirt (you must know the MAC address of the guest interface to make this change, so either perform the operation after the guest has been created, or decide on a MAC address for the guest yourself, then set this same address manually when creating the guest machine configuration)"

(the important differences are a) this points out that the guest must be using DHCP to acquire its IP, and 2) doesn't insist that the guest already be created - it's actually probably quite common that people will want to have the entry already setup when they create the guest for the first time, so that the static IP address is used even when installing the guest OS.)

2) you should note that this only works for guest interfaces that are connected to a libvirt virtual network with a forwarding mode of "nat", "route", or no <forward> at all - it will not work if the network has forward mode="bridge" or "hostdev". In those cases, the dhcp server will be elsewhere on the network, not under control of libvirt, and the static IP entry would need to be made on that server, so they will need to consult the dhcp server's documentation (could be anything from a consumer wireless router to a mucho-bucks Cisco something-or-other)

3) I think section 23.6 can/should be removed. It is very disruptive to bring down the network, as any guest currently connected to the network will then need to be restarted. The instructions in 23.7 can do everything that's done by 23.6, but without needing to bring down the network (if this text originated on the libvirt wiki, we should also probably remove it from there)

4) You only need to "restart the interface" if the guest is already running. To force an already-running guest to re-request its IP address, you can do this from the *host*:

  virsh domif-setlink guest1 52:54:00:6c:a7:6d down
  sleep 10
  virsh domif-setlink guest1 52:54:00:6c:a7:6d up

This makes the guest OS think that the ethernet cable has been unplugged, then re-plugged after 10 seconds. The "sleep 10" is important, because many/most dhcp clients allow for a short disconnection of the cable without re-requesting an IP address; 10 seconds is long enough that most dhcp clients should forget their old IP address and request a new one when domif-setlink up is executed.

If that doesn't work for a particular guest, they can use "whatever is the appropriate procedure on the guest's operating system" to bring the interface down and back up. For example:

   service NetworkManager restart

or
   systemctl restart network.service

or
  ifdown eth0; ifup eth0

are just a few. I don't know if you want to bother spelling any of these out - using virsh domif-setlink has the advantage that it should work for any guest OS.

Comment 18 Laine Stump 2015-12-08 18:38:43 UTC
*  replace "but instead you need it" with "but you still need it"

* All MAC addresses in libvirt use : as the separating character, not -. You will need to replace - with : in *all* MAC addresses, both in what is output by the commands as well as what you type.

* Step 1 - The MAC address from this step will also be used later. You may want to mention that if the guest hasn't yet been created, they should use the name they *plan* to give to the guest, and create a valid unicast MAC address (suggest that it start with 52:54:00: since that's what libvirt uses when creating random MAC addresses, and that the last three numbers can each be any random hexadecimal number as long as the final MAC address doesn't match any MAC address already in use on the network).

Step 2: the example command is in boldface, but all other examples are not boldface. (Actually I looked back through the document and this seems to be inconsistent. Maybe all typed commands should be in bold and all output from programs should be un-bold?)

Step 3: "fogets" 

At the end maybe "reset the guest's interface from the guest operating system's management interface".

Off-topic, but in Section 23.8.9. "Manipulating interface snapshots", this sentence:

  Note that rebooting the host physical machine also serves as an
  implicit rollback point. 

should instead be:

  Note that if the host is rebooted before virsh iface-commit is run
  an automatic rollback will be performed, restoring the configuration
  to its state at the time of virsh iface-begin. This is useful in cases
  where an improper change to the network configuration may render the
  host unreachable for purposes of undoing the change, but the host may
  still be power-cycled or otherwise forced to reboot.

Comment 20 Laine Stump 2015-12-09 21:31:12 UTC
* You may want to point out that, along with the network name in the output of virsh domiflist, the MAC address listed there is also the one that will be used for the rest of the steps.

* "The MAC address should start with 52:54:00 for unicast addresses and 51:54:00 for multicast addresses."

 * The MAC address of an interface must *always* be a unicast address.
 * It seems odd that the examples use the 00:00:5E range, but the test
   recommends using 52:54:00. I'm not sure what to do about this though;
   I think it's important that people know 52:54:00 is the range commonly
   used by libvirt and qemu, but understand that you're supposed to use
   addresses from an official "test range" in all examples.

* More importantly, I also just realized that there has been no mention of the requirement that the IP address be within the dhcp "range" specified for the network, and that it must also not conflict with any existing static IP address on the network. You can see both of these with the following command:

   virsh net-dumpxml default | egrep 'range|host\ mac'

The output will look something like this:

      <range start='192.168.122.2' end='192.168.122.254'/>
      <host mac='52:54:00:48:27:1D' ip='192.168.122.170'/>

(there may be multiple "host mac" lines). I suppose you could mention this somewhere around where you talk about valid IP address?

Comment 24 Laine Stump 2015-12-14 18:20:38 UTC
(Sorry, I typed this reply last week then got distracted before I hit Save)

> I have a test setup with <forward mode='nat'> and <bridge name='virbr0'
> stp='on' delay='0'/>. When I do virsh domiflist, "Source" is virbr0, but the
> net-update must be set for "default", not "virbr0". Suggestions for how to
> determine that?


My guess is that your guest is configured with

     <interface type='bridge'> ... <source bridge='virbr0'/> ...

instead of

     <interface type='network'> ... <source network='default'/>

There is no simple / single line way to get from the former (the name of the bridge) to the network name.

How did you create your guest? Is there a specific reason why it is configured this way?

You can certainly configure your guest in this way (pointing directly at the bridge created by a network rather than at the network itself) and it will work, but I don't think we should encourage/enable or support it in our documentation.

Comment 25 Laine Stump 2015-12-14 18:32:40 UTC
Laura - I notice that the starting IP address in the range is .......0. Depending on the netmask/prefix used ("<ip address='blah' prefix='n'>..."), this could lead to a non-working configuration. If you're going to use 198.51.100.n for your examples, you should probably assume this:


1) The IP address of the bridge itself is 198.51.100.1 and the prefix is 24

2) The DHCP range is 198.51.100.2 - 198.51.100.254

3) The static IP addresses you assign will start at .......2. So, the network's ip element would look like this:


    <ip address='198.51.100.1 prefix='24'>
       <dhcp>
         <range start='198.51.100.2' end='198.51.100.254'/>
         <host mac='52:54:00:48:27:1C' ip='198.51.100.2'/>
       </dhcp>
    </ip>

and the new entry would be something like:

         <host mac='52:54:00:48:27:1D' ip='198.51.100.3'/>

Comment 26 Ken Gaillot 2015-12-14 18:36:43 UTC
(In reply to Laine Stump from comment #24)
> (Sorry, I typed this reply last week then got distracted before I hit Save)
> 
> > I have a test setup with <forward mode='nat'> and <bridge name='virbr0'
> > stp='on' delay='0'/>. When I do virsh domiflist, "Source" is virbr0, but the
> > net-update must be set for "default", not "virbr0". Suggestions for how to
> > determine that?
> 
> 
> My guess is that your guest is configured with
> 
>      <interface type='bridge'> ... <source bridge='virbr0'/> ...
> 
> instead of
> 
>      <interface type='network'> ... <source network='default'/>

Yes

> There is no simple / single line way to get from the former (the name of the
> bridge) to the network name.
> 
> How did you create your guest? Is there a specific reason why it is
> configured this way?

I used virt-install with --network "bridge:virbr0". I don't recall why; it was probably in some tutorial I followed.

> You can certainly configure your guest in this way (pointing directly at the
> bridge created by a network rather than at the network itself) and it will
> work, but I don't think we should encourage/enable or support it in our
> documentation.

OK, then I'll stop doing that :) and I'm fine with the suggested command in the documentation.

Comment 28 Laine Stump 2015-12-15 15:49:43 UTC
* Step 1: You haven't changed the bad wording pointed out by Ken:

   "Display the guest's configuration file by ..."

should be replaced with:

   "Display the guest's network configuration by ..."

* Step 2: replace:

   "run the following command as root in a terminal on the guest"

with:

   "run the following command as root in a terminal on the host"

Replace:

   "The output you see will differ than that of the example"

with:

   "The output you see will differ from the example"

* Step 3 - Replace:

   "The MAC address should start with 52:54:00 for unicast addresses
    and 51:54:00 for multicast addresses"

with:

   "The MAC address should be a valid unicast MAC address (6 hexadecimal digit pairs separated by ":", with the first digit pair being an even number); when libvirt creates a new random MAC address, it uses "52:54:00" for the first three digit pairs, and it's recommended (but not required) to follow that convention."

(There is no point in mentioning multicast MAC addresses, because this address should *never* be multicast. I pointed this out in Comment 20, but I guess you missed it).

* Step 4: Replace:

   "run the following commands in as root"

with:

   "run the following commands as root"

Comment 30 Laura Novich 2015-12-16 16:21:15 UTC
following confirmation with Laine, changing status to VERIFIED
Thank you Laine!
Best Regards,
Laura

Comment 31 Laura Novich 2015-12-20 08:40:41 UTC
Published on December 17 version 2-24

Comment 32 Laura Novich 2015-12-20 08:43:19 UTC
Published on December 17 version 2-24


Note You need to log in before you can comment on or make changes to this bug.