Re: Ensuring predictable MAC address for bond interface

[Dan Williams <dcbw redhat com>]

On Wed, 2014-10-08 at 07:54 +0200, Thomas Haller wrote:
> On Tue, 2014-10-07 at 20:25 -0700, Ed Swierk wrote:
> > For these experiments I upgraded to NetworkManager 0.9.10.0.
> >
> >
> > I tried setting primary=eth0 in the bond options. This option has an
> > effect in only one case: if I start with only eth1 plugged in (i.e.
> > carrier up), and then plug in eth0, the bond driver will switch the
> > active slave from eth1 to eth0 (whereas normally it leaves eth1
> > active, switching to eth0 only when link drops on eth1). But this has
> > no effect on the MAC address of the bond device--either way, the bond
> > device gets eth1's MAC address and keeps it, since eth1 was the first
> > enslaved device.
> >
> > I also experimented with setting ignore-carrier=eth0,eth1 in the main
> > section of NetworkManager.conf, in the hope of tricking the
> > nm_device_state_changed() logic into transitioning the slave devices
> > to DISCONNECTED and enslaving them regardless of their carrier state.
> > This works, sort of: now NetworkManager starts up, creates the bond
> > device, and enslaves eth0 and eth1, whether or not they are plugged
> > in. The bond device gets eth0's MAC address. It's still pretty easy to
> > mess things up, for example by bringing down the bond device (via
> > nmcli conn down) and then bringing up eth1.
> >
> >
> > I'm getting the sense that I'm fighting a losing battle against a
> > fundamental aspect of NetworkManager, which is to react dynamically to
> > link events, and only secondarily try to guarantee a certain
> > configuration. I'm going to try systemd-networkd next, and see if it's
> > a better fit for my application.
>
>
> I think that the concept "order of how slaves are attached to bond
> master" does not mean anything for NetworkManager, hence there is no
> ~first~ slave.
>
> (of course, whenever you attach slaves to a bond, one of those will be
> the first. But that is random in case of autoconnect and nothing that
> you can configure or that NM anticipates).
>
>
> Is it possible to set the mac address when creating the bond? If yes, we
> could add a configuration option similar to
> NM_SETTING_BRIDGE_MAC_ADDRESS.

Yeah, the kernel allows that, and we should add this option to bonds
like we did for bridges, as you suggest.

> We could make NM_SETTING_BOND_MAC_ADDRESS be a hex string, or
> alternatively, an interface name -- so we would look at the interfaces
> and take it's MAC address.

That gets complicated though, since if the slave that you specify
doesn't exist, you cannot set up the bond.  I'd rather handle that in
editors, like we do for the MAC Address boxes of other interface types
with a dropdown suggesting interfaces you can choose.  But we should do
this either way.

Dan

> Thomas
> >
>
> > On Tue, Oct 7, 2014 at 7:34 AM, Dan Williams <dcbw redhat com> wrote:
> >         On Mon, 2014-10-06 at 12:51 -0700, Ed Swierk wrote:
> >         > I'm using NetworkManager on a server with two wired Ethernet
> >         interfaces
> >         > (eth0 and eth1) configured as slaves of a bond in
> >         active-backup mode. I'd
> >         > like the bond interface to always be assigned eth0's MAC
> >         address.
> >         >
> >         > This is easy with old-school static network configuration
> >         like ifupdown:
> >         > just make eth0 the first slave of the bond interface, and
> >         Linux will copy
> >         > eth0's MAC address to the bond.
> >         >
> >         > When NetworkManager starts up and reads system-interface
> >         config files, it
> >         > creates the bond interface right away, but doesn't add a
> >         slave until it
> >         > notices that the slave's link is up (i.e. carrier is 1). And
> >         of course the
> >         > link state depends on lots of things, like whether a cable
> >         is plugged in
> >         > and the state of the switch or host at the other end. Thus
> >         whether eth0 or
> >         > eth1 gets enslaved to the bond first is unpredictable,
> >         meaning the bond
> >         > interface's MAC address is unpredictable. This is
> >         troublesome in some
> >         > environments, such as ones where a DHCP server assigns IP
> >         addresses based
> >         > on the MAC address of the client.
> >         
> >         Does setting the "primary" bond option when in active-backup
> >         mode to
> >         "eth0" make things better?  In nm-connection-editor it'll be
> >         in the page
> >         for Bond options, in nmtui it'll be in about the same place,
> >         and in
> >         nmcli you'd do:
> >         
> >         nmcli con mod Bond1 +bond.options "primary=eth0"
> >         
> >         and then re-activate the connection.  Let me know if that
> >         does/doesn't
> >         help.  (note; this may only work with 0.9.10+)
> >         
> >         Dan
> >         
> >         > NetworkManager already treats a bond slave interface
> >         differently when its
> >         > link goes down, leaving it in DISCONNECTED state rather than
> >         switching it
> >         > to UNAVAILABLE. So there's precedent for having a bond
> >         interface with one
> >         > or more link-down slave interfaces. I think the easiest way
> >         to achieve a
> >         > stable MAC address is to extend that behavior to the startup
> >         case: as soon
> >         > as NetworkManager sees an interface that's configured as a
> >         bond slave, it
> >         > should move it from UNAVAILABLE to DISCONNECTED. That way
> >         the first
> >         > configured slave interface, rather than the first one with
> >         link up, is
> >         > enslaved to the bond.
> >         >
> >         > I couldn't figure out a way to configure this in
> >         NetworkManager 0.9.8.10,
> >         > nor in mainline code. As a proof of concept, I hacked
> >         > nm_device_state_changed() in src/nm-device.c: in the second
> >         switch(state),
> >         > UNAVAILABLE case, I force the transition to DISCONNECTED for
> >         eth0 and eth1.
> >         > This works as I'd hoped: both interfaces are enslaved right
> >         away, with eth0
> >         > always first.
> >         >
> >         > I didn't see any easy way to implement this behavior
> >         cleanly, though. This
> >         > new behavior should apply only to bond slave interfaces. At
> >         the point where
> >         > nm_device_state_changed() is called, there's not yet a
> >         corresponding
> >         > connection for eth0 or eth1, so I can't check whether the
> >         interface is
> >         > configured as a bond slave. I thought I'd ask for advice
> >         before spending
> >         > more time on this.
> >         >
> >         > Any help would be appreciated!
> >         >
> >         > --Ed
> >         > _______________________________________________
> >         > networkmanager-list mailing list
> >         > networkmanager-list gnome org
> >         > https://mail.gnome.org/mailman/listinfo/networkmanager-list
> >         
> >         
> >
> >
> > _______________________________________________
> > networkmanager-list mailing list
> > networkmanager-list gnome org
> > https://mail.gnome.org/mailman/listinfo/networkmanager-list