NetworkManager behavior answers not found in docs

[Thomas HUMMEL <thomas hummel pasteur fr>]

Hello,

I'm considering migrating from a CentOS 6 HPC cluster to CentOS 7 one. 
For this purpose I've read quite a lot of doc (man, guides, ...) about 
systemctl and NetworkManager to be fluent in their use.

Regarding NetworkManager, I've been at first confused by the induced 
complexity or number of possible mixed old way/native use cases brought 
by the "network-scripts" compatibility layer provided by the ifcfg-rh 
plugin.

As a matter of fact, I experimented a lot to be sure of who did what and 
how or when between network-scripts and NetworkManager.

Digging deeper, things got clearer. Still I'm not really sure about the 
points described below.

Notes :

- I'm running NetworkManager-1.10.2-16.el7_5.x86_64 on a CentOS Linux 
release 7.5.1804 (Core)
- I'm using default settings, so this must be ifcfg-rh plugin first, 
then keyfile otherwise
- I'm talking only about ethernet connections/devices here


1. /run/NetworkManager/devices directory


I know NetworkManager exports connections as 
/org/freedesktop/NetworkManager/Settings/<num> D-Bus objets

What exactly is under /run/NetworkManager/devices/<num> path ? UUID 
inside those files don't always match what nmcli shows me ?


2. device vs connection


My understanding is that there's a clear distinction between a device 
and a connection. To be more precise, that an active connection is a set 
of settings (potentially persisted on disk on a connection file via some 
plugin) "applied" to a device. The applied part is not very clear to me :

What's the difference between connecting a device (nmcli device connect) 
and activating a connection (nmcli connection up) ? Maybe my confusion 
comes from the fact that adding a connection automatically connects it 
to a device ?


3. connection autocreation

in what circumstances exactly does NetworkManager autocreate connections ?

a) What I found as well is that nmcli connection add...ifname=xxx seems 
to auto-connect the xxx device to the named interface thus voiding the 
need to connect the device.

Is device connect needed only for special cases, like when we set the 
NM_UNMANAGED udev property to 1 or do I miss something (refer to 
question 2 above)

Is there a distinction between autocreation vs autoactivation of a 
connection (if yes in what cases) ?

b) auto create and connect when restarted

I'm confused about a case where a connection is autocreated and 
activated at NetworkManager restart while not at boot (twisted case I 
must admit) :

Let's take the following use case as an example :

- a host with 2 ethernet devices : eth0 and eth1

- eth0 has a network-script like connection file 
(/etc/sysconfig/network-scripts/ifcfg-eth0) and is NetworkManager 
managed (either by default because no "NM_CONTROLLED=no" is stated or 
explicitly with "NM_CONTROLLED=yes")
- eth1 has no connection file (neither ifcfg-rh style nor keyfile style, 
that is nothing for eth1 neither in /etc/sysconfig/network-scripts nor 
in /etc/NetworkManager/system-connections)

- no NM_UNMANAGED=1 in udev

test 1 :

-> after boot, as expected eth0 is configured and up, nmcli shows a 
connection for it but eth1 has no connection. It has to be manually 
connected for that matter. If done, NetworkManager seems to auto 
magically CREATE the connection

-> where is that magic described ? How does a simple connect CREATES a 
connection. Man states that connect looks for a matching EXISTING 
connection ?

test 2 under the same conditions :

- reboot of the host
- only one connection (eth0) as expected
- create the foobar connection like this

    nmcli connection add type ethernet con-name foobar ifname eth1

-> as expected a ifcgh-rh format connection file has been created in 
/etc/sysconfig/network-scripts/ifcfg-foobar and nmcli shows 2 
connections : eth0 and foobar

- manually delete foobar connection file : rm 
/etc/sysconfig/network-scripts/ifcfg-foobar

-> as expected nmcli still shows foobar connection

- reload connection files via nmcli connection reload

-> as expected nmcli only shows eth0 connection

So far, everything seems to work as expected BUT :

if instead of nmcli connection reload we RESTART (not reload, but 
restart) NetworkManager via systemctl restart NetworkManager, ANOTHER 
(different UUID and eth1 instead of foobar for the name) connection is 
auto created and auto activated

I cannot explain why and how this connection is created on restart since 
no connection is auto created at boot.
Granted manually removing the connection file is tricky but even then...

Can you help me figure out those points which would complete a bit 
further my understanding of NetworkManager ?

Thanks

--
Thomas H.