Bugreport: macvlan device choosing wrong parent device

[Ulrich Ölmann <u oelmann pengutronix de>]

Hi there,

trying to work with a MAC VLAN device on top of a standard ethernet device I
stumbled across misbehaviour of NetworkManager <= 1.8.4 in its choice of the
underlying ethernet device when one of them is declared as unmanaged (needed as
the system is running from NFS-root). Here is my setup:

    root@host:~ ip link
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
        link/ether 00:13:95:0a:4b:30 brd ff:ff:ff:ff:ff:ff
    3: sit0@NONE: <NOARP> mtu 1480 qdisc noop qlen 1
        link/sit 0.0.0.0 brd 0.0.0.0
    4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel qlen 1000
        link/ether 00:09:5b:be:62:71 brd ff:ff:ff:ff:ff:ff

    root@host:~ cat /etc/NetworkManager.conf
    [main]
    plugins=keyfile

    [keyfile]
    unmanaged-devices=00:13:95:0a:4b:30

    root@host:~ ls -l /etc/NetworkManager/system-connections
    -rw-------    1 root     root           161 Jan  1 00:12 routable

    root@host:~ cat /etc/NetworkManager/system-connections/routable
    [connection]
    id=routable
    uuid=7ac61f21-bf59-4c4c-ae38-51ce131b2afc
    type=802-3-ethernet

    [802-3-ethernet]
    duplex=full

    [ipv6]
    method=auto

    [ipv4]
    method=auto

When I start NetworkManager everything works as expected:

    root@host:~ systemctl start NetworkManager
    root@host:~ nmcli
    eth1: connected to routable
            "NETGEAR FA120 Adapter"
            ethernet (asix), 00:09:5B:BE:62:71, hw, mtu 1500
            ip4 default
            inet4 192.168.24.156/16
            inet6 fe80::209:5bff:febe:6271/64
            route6 ff00::/8

    eth0: unmanaged
            "eth0"
            ethernet (pch_gbe), 00:13:95:0A:4B:30, hw, mtu 1500
    [...]

Now I add a MAC VLAN connection that shall implicitely create a new virtual
MAC VLAN device on top of eth1:

    root@host:~ ls -l /etc/NetworkManager/system-connections/*
    -rw-------    1 root     root           226 Jan  1 00:14 /etc/NetworkManager/system-connections/link-local
    -rw-------    1 root     root           161 Jan  1 00:12 /etc/NetworkManager/system-connections/routable

    root@host:~ cat /etc/NetworkManager/system-connections/link-local
    [connection]
    id=link-local
    uuid=51d4b623-d0de-9c56-e762-ccd159bfcfe0
    interface-name=link-local
    type=macvlan

    [macvlan]
    parent=7ac61f21-bf59-4c4c-ae38-51ce131b2afc
    mode=3

    [ipv6]
    method=link-local

    [ipv4]
    method=link-local

Asking NetworkManager to reload its connections the system does what is expected

    root@host:~ nmcli c reload
    root@host:~ nmcli
    eth1: connected to routable
            "NETGEAR FA120 Adapter"
            ethernet (asix), 00:09:5B:BE:62:71, hw, mtu 1500
            ip4 default
            inet4 192.168.24.156/16
            inet6 fe80::209:5bff:febe:6271/64
            route6 ff00::/8

    link-local: connected to link-local
            "link-local"
            macvlan, sw, mtu 1500
            inet4 169.254.170.34/16
            route4 224.0.0.0/4
            inet6 fe80::6091:15ff:fe4b:5aa0/64
            route6 ff00::/8

    eth0: unmanaged
            "eth0"
            ethernet (pch_gbe), 00:13:95:0A:4B:30, hw, mtu 1500
    [...]

with the MAC VLAN device sitting on top of eth1

    root@host:~ ip link
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
        link/ether 00:13:95:0a:4b:30 brd ff:ff:ff:ff:ff:ff
    3: sit0@NONE: <NOARP> mtu 1480 qdisc noop qlen 1
        link/sit 0.0.0.0 brd 0.0.0.0
    4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel qlen 1000
        link/ether 00:09:5b:be:62:71 brd ff:ff:ff:ff:ff:ff
    6: link-local@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue qlen 1000
        link/ether 62:91:15:4b:5a:a0 brd ff:ff:ff:ff:ff:ff

So far so good. The problem is met when the connections are teared down and
NetworkManager is finally restarted:

    root@host:~ nmcli d disconnect link-local eth1
    Device 'link-local' successfully disconnected.
    Device 'eth1' successfully disconnected.

    root@host:~ systemctl restart NetworkManager
    root@host:~ nmcli
    eth1: connected to routable
            "NETGEAR FA120 Adapter"
            ethernet (asix), 00:09:5B:BE:62:71, hw, mtu 1500
            ip4 default
            inet4 192.168.24.156/16
            inet6 fe80::209:5bff:febe:6271/64
            route6 ff00::/8

    eth0: unmanaged
            "eth0"
            ethernet (pch_gbe), 00:13:95:0A:4B:30, hw, mtu 1500

    [...]

    link-local: unmanaged
            "link-local"
            macvlan, sw, mtu 1500

    root@host:~ ip link
    1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
        link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
        link/ether 00:13:95:0a:4b:30 brd ff:ff:ff:ff:ff:ff
    3: sit0@NONE: <NOARP> mtu 1480 qdisc noop qlen 1
        link/sit 0.0.0.0 brd 0.0.0.0
    4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel qlen 1000
        link/ether 00:09:5b:be:62:71 brd ff:ff:ff:ff:ff:ff
    7: link-local@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
        link/ether 42:2c:23:3f:4c:b1 brd ff:ff:ff:ff:ff:ff

NetworkManager obviously created the MAC VLAN device on top of eth0 now which
seems to be a bug (especially as eth0 was declared to be unmanaged). As a result
NetworkManager is not able to create the needed internal resources which is
logged in the journal:

    [...]
    device (eth0): link connected
    manager: (eth0): new Ethernet device (/org/freedesktop/NetworkManager/Devices/2)
    manager: (link-local): new Macvlan device (/org/freedesktop/NetworkManager/Devices/3)
    device (eth1): link connected
    manager: (eth1): new Ethernet device (/org/freedesktop/NetworkManager/Devices/4)
    [...]
    manager: (link-local) can't register the device with manager: A device with ifindex 7 already exists
    [...]

It looks like NetworkManager does not create the devices in the correct order in
contrast to what has been done manually before restarting NetworkManager. This
behaviour is only seen if the parent is referenced via a UUID. Using an
interface name instead ("parent=eth1"), everything is fine.

Best regards
Ulrich
--
Pengutronix e.K.                           |                             |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |