Hi,
Am Do., 21. März 2019 um 09:16 Uhr schrieb Edward Haas <edwardh redhat com>:
>
>
>
> On Wed, Mar 20, 2019 at 8:35 PM Till Maas <till redhat com> wrote:
>>
>> Hi,
>>
>> Am Mi., 20. März 2019 um 18:13 Uhr schrieb Edward Haas via
>> networkmanager-list <networkmanager-list gnome org>:
>> >
>> >
>> >
>> > On Wed, Mar 13, 2019 at 3:48 PM Gris Ge via networkmanager-list <networkmanager-list gnome org> wrote:
>> >>
>> >> Hi Guys,
>> >>
>> >> For the IP route editing in nmstate, the parietal editing is
>> >> problematic:
>> >>
>> >> User case A:
>> >> Current routes: 1.1.1.0/24 via 2.2.2.2 dev eth1
>> >> Desired routes: changed next hope to 3.3.3.3
>> >>
>> >> User case B:
>> >> Current routes: 1.1.1.0/24 via 2.2.2.2 dev eth1
>> >> Desired routes: add another route entry,
>> >> 1.1.1.0/24 via 3.3.3.3 dev eth1
>> >> for load balance.
>> >>
>> >> With partial editing:
>> >> For A), nmstate API user need to mark exiting routes as 'absent',
>> >> then add new one.
>> >> For B), nmstate API user just add new one.
>> >>
>> >> Pro:
>> >> * It's easy for user to add new route entry with
>> >> minimum/incremental data passing to nmstate API.
>> >> Cons:
>> >> * When editing certain entry, user need to search it from
>> >> current state and mark it as absent. This adds a lot more
>> >> works in API user.
>> >>
>> >> * The API is complexing the common user case -- editing route.
>> >
>> >
>> > A user will either persist the state he wants or read it when needed from nmstate.
>>
>> users cannot persist the state they want in the current form because
>> they cannot express that everything that is not specified should be
>> removed.
>
>
> Applications like oVirt does it by persisting the state they are interested in. The oVirt manager
> is only concern about the networks it manages, not the one it does not.
> It is not interested to manage or control the others and does not persist non interested entries.
> (although it will show available resources to allow acquiring them into the pool if needed)
The fact that oVirt does not care about this does not mean that it
makes sense to limit the Nmstate API to only partial states.
I do not think it is limiting, if you want to develop a full state command (`apply_full`), you
are welcome to add it to the requirements and contribute an implementation.
I just do not want oVirt to care or somehow touch other resources on the system.
>> > At that moment, he can choose to remove and add or just add something new.
>> > I do not understand what complexity is there. We can easily provide SDK helpers to manipulate
>> > the state if that is what they need, after all, it is a dict/map.
>> > At the moment we do not focus on the SDK, but only on the raw API.
>> > With consumption, I am pretty sure we will be able to build a state sdk lib to ease user usage.
>> >
>> >>
>> >> With full editing: API user just define what he/she wants.
>> >>
>> >> Pro:
>> >> * It simple and clear to say in API document about route:
>> >> You get what you asked, nothing else.
>> >>
>> >> Cons:
>> >> * API user need to define the whole state, which might be
>> >> a headache with 10k+ routes.
>> >
>> >
>> > Fetching and Setting back on each change from a remote client is very costly,
>> > consider a management system (the user/client) and hundreds of hosts, each with 10k entries.
>> > That does not scale well.
>>
>> Nmstate does not provide a remote API only a local one. Whatever is
>> going to provide the remote API could also support partial states. Or
>> it can be an SDK as you call it.
>
>
> It currently does not, but it must arrive in one form or the other if we want the api to be useful.
> Adding another layer of abstraction is not something that makes sense to me,
> I consider the nmstate API the one that applications should use, remotely or locally.
>
>> Nevertheless, this would keep the
>> core (hopefully) simpler.
>
>
> We will still need to read the whole existing state, check if the request is possible, and
> we will need to recreate the whole thing anyway.
Why?
Because you will not want to change 1000 entries which have not changed. So you will
probably compare them and not touch those (not touch their NM profiles).
You will also take defaults and convert them to explicit things, which fixes them through upgrades.
There is also the race issue: If the client does not care about eth0, but he sends it because he must,
and in the middle eth0 changed, we will now overwrite it unintentionally.
We will need to handle these cases in some way which will require .
> There are even cases where the client is of an older version than the server, then the complementary config
> action is needed anyway. In addition, parameters existing in the provider (NM) and not
> in the state config we will have to treat like we do today (complement it, or not touch it).
> I am not convinced it is simpler, it is just a new bunch or new cases to handle.
> Finally, making it simpler for the core is a secondary objective. It needs to be simpler for the clients,
> and taking oVirt as the first client example, it will not be simpler to use the full state.
The layer to make it simpler for clients can still be added after the
core is working great.
I have not notice that these are the problems we are facing at the moment.
Complexity exists all over, all have their pros and cons.
>> Also it seems to me a lot easier to build
>> partial state handling on top of something that handles full states
>> than the other way around. If a program supports partial states, it
>> cannot distinguish them from full states.
>
>
> My claim is that management systems are not all-or-nothing, they usually are interested
> in managing only a partial config of all the available resources.
> I am not exactly clear on why moving from partial to full is more trouble then from full to partial.
> We do a version of it today:
> new-full-state = show-full-state.update(partial-state)
> (update like in a python dict, just recursive)
Nmstate does not support setting full states currently. Your example
shows the case that is easy today, the user providing a partial state.
To actually apply a state as a full state, the client needs to fetch
the current state and change the state to absent for every entity that
is not present in the desired state.
It is supporting it internally for an iface state as a whole.
Anyway, I was just pointing out that this is as easy as the other direction.
>> >> I personally favor the full editing staff and add below functions when
>> >> full editing is a real concern in the future:
>> >>
>> >> * nmstate.netapplier.add() # Add new stuff
>> >> * nmstate.netapplier.edit() # Override existing stuff
>> >> * nmstate.netapplier.remove() # Remove stuff
>> >>
>> >>
>> >> Any comments and suggestions?
>> >
>> >
>> > Nmstate is intended to represent a declarative API, not a functional one.
>> > You could suggest that we need to have two versions of apply/set: Partial and Full.
>> > The same as we will probably need to support to versions of show (like netconf does): Config only, and Config + Read-Only(Oper State).
>> >
>> > We could also propose special magic entries, which represent a collection:
>> > - all-other <interfaces, routes, ip-addresses>
>> > - all-other interfaces of type ethernet
>> > But these are almost identical to the configuration policies which have been proposed through the kubernetes-nmstate project.
>> >
>> > I would prefer to have the partial state applied whenever it makes sense.
>> > I also think it is safer: As an example, VDSM controls only a portion of the host interfaces, it does not touch parts
>> > which have not been explicitly configured in the management system. This provides a flexibility to the host owner
>> > to manage it through two management systems: The oVirt one and the Host deployment one, each having their dedicated
>> > interfaces. When oVirt wants to define something on its owFor pned interfaces, it should not touch and risk the others.
>>
>> How does this work with routes since they are not tied to an interface in oVirt?
>
>
> The non interested routes will not get touched because they are of no interest to the management system.
> The interface in my description was an example.
>
>>
>> > The user should mention only the parts it is interested to change, anything else should not change.
>>
>> It is a valid scenario. But the question is whether this needs to be
>> handled in the Nmstate core.
>
>
> Yes, because it is something that has been requested and needed by the potential clients.
> The ifcfg files and NM are an example of this: If it is not mentioned, it is not touched..
The need from clients does not mean that it is a good idea to do this
in the core. It only needs to be available for the clients.
nmstate is the API the clients will use, there is no other API here.
Internally, inside nmstate you can propose to complement to a full state and implement
you proposal. If I understand correctly, you are arguing that nmstate should offload this
task to the clients, and I'm arguing that nmstate should offload the clients work.
Again, if you think it will fit some other clients, you can propose to create an `apply-full` state command.
> You should also consider this scenario which exist too frequent: Provisioning systems have an one direction flow,
> they only set things and not query them. Without a query, the full state option will not work.
Given that Nmstate does not support full states, the provisioning
system needs to query the current state to apply a full authoritative
state. Also I am not saying that there should be now support for
partial states at all in Nmstate, just not at the core.
I do not understand this point with the provisioning.
The provisioning system only knows that it wants to set eth88 with a specific configuration,
it does not care what it has on the system at the moment.
So, if it cares only to set an mtu, it will specify only it and nothing else is assured not to be changed
(for the specific interfaces and the others, including routes, dns and other stuff).
If the system wants to change more things, it will specify them all, making sure they overwrite anything
that currently exists. It does not need to know what there is on the target, only what it wants to change.
Kind regards
Till
--
Till Maas
Ansible RHEL Networking System Role Maintainer
Red Hat GmbH
