On Thu, 2017-04-20 at 17:40 +0300, Dan Fruehauf wrote:
> Thanks Thomas for the reply.
>
> I'm en route to fix the problems with NetworkManager-ssh, yet I'm not
> quite sure what it doesn't do right. Or in other words - what changed
> in nm that makes the whole thing not work properly.
>
> Previously it would use ifconfig to configure the interface after the
> VPN is up. Now NetworkManager does that. I've been looking also
> through the openvpn plugin to see what has changed recently.
> Generally, the routes and interfaces are setup correctly, yet, as I
> said, there's no connectivity between the 2 VPN interfaces (client
> and server).
If NM changed in a way that breaks old plugins, it's a bug. We might
introduce new behaviors, but old plugins should continue to work
without modification. -- unless the plugin did something unsupported.
In general, plugins don't directly configure IP addressing on the
device. Instead they notify NM about the IP configuration and NM
applies it. The reason is, that the plugin is not aware of the system
wide configuration, only NetworkManager can get this right. Also,
properties like ipv4.never-default and ipv4.route-metric add additional
constrains to the IP configuration for the VPN, so, the plugin is
possible not aware of all the details.
I guess, if you have a plugin that really wants to configure IP
addresses on the interface, it should work as long as the plugin also
notifies NM about the very same configuration. Basically, NM would just
try to re-apply the same IP configure that are already present (and do
nothing).
> If you can come up with any useful advice, it'd be more than welcome.
> I can also find you on IRC if you think that might be easier.
I wonder what NM is doing to break your setup...
Thomas
>
> Cheers,
> Dan.
>
>
> On Thu, Apr 13, 2017 at 4:32 PM, Thomas Haller <thaller redhat com>
> wrote:
> > On Tue, 2017-04-11 at 09:17 +1000, Dan Fruehauf wrote:
> > > G'day,
> > >
> > > I hope that post will not be long, but I've spent a few hours
> > trying
> > > to narrow down the problem so I can provide as much information
> > > without wasting anyone's time.
> > >
> > > I started to debug why NetworkManager-ssh (which I maintain) does
> > not
> > > allow traffic through interfaces (tun interfaces usually) and
> > went
> > > down a deep rabbit hole, which I'm not entirely sure has got to
> > do
> > > much with NetworkManager-ssh at the moment.
> > >
> > > What I'm trying to do:
> > > * Setup a "poor man's VPN" aka SSH VPN to a remote host (on AWS)
> > > * On my machine I should eventually have a tun device with
> > > 172.16.40.2
> > > * On the server machine a tun device with 172.16.40.1
> > > * Those two internal addresses should be reachable from one
> > another
> > >
> > > The steps that are usually necessary are:
> > > 1. SSH to remote machine (with tunnel creation parameters) + run
> > a
> > > ifconfig command to configure the tun device
> > > 2. Configure the tun device on the client host with ifconfig
> > > (ifconfig tun0 ...)
> > > 3. Replacing default routes etc
> > >
> > > So when NetworkManager-ssh does what it does, the end result is
> > what
> > > I expect, except that things don't work. Traffic can reach
> > > 172.16.40.1, but nothing can reach 172.16.40.2. Before you point
> > a
> > > problem with the server routing tables, please read along.
> > >
> > > I decided to just run the SSH command from the command line,
> > > eliminating everything that NetworkManager does to the VPN
> > > connection. What happens here is very unclear to me, but someone
> > > might be able to explain it quickly. When running the steps 1 & 2
> > > quickly one after the other - the VPN is setup properly and
> > > 172.16.40.2 is reachable from the server side. When I see
> > quickly, I
> > > mean no delay in between, see log for that here (https://paste.fe
> > dora
> > > project.org/paste/QgnyCz7hEuvuLiAiOhtTfl5M1UNdIG ).YhyRLivL9gydE=
> > Then
> > > I can obviously proceed to replacing the default route and so on
> > and
> > > so forth. VPN works.
> > >
> > > On the other hand, if i introduce a delay of say 5 seconds, it
> > allows
> > > NetworkManager to have enough time to do something I don't
> > understand
> > > to the tun0 device which then renders it as unreachable, see log
> > for
> > > that here (https://paste.fedoraproject.org/paste/0~lxbpwNuIxGUvDw
> > jokW
> > > 6F5M1UNdIGYhyRLivL9gydE=). I spaced it out a bit where the 5
> > seconds
> > > gap appears.
> > >
> > > A few other things to note:
> > > * I'm running fedora 24 (with kernel 4.9.9-100.fc24.x86_64 and
> > > NetworkManager 1.2.6-1)
> > > * Selinux is disabled (both on server and client)
> > > * firewalld is disabled
> > > * When sniffing traffic, all traffic reaches the server
> > > (172.16.40.1)
> > > * Traffic comes back on on the SSH tunnel, but never reaches
> > > 172.16.40.2 (verified it both with strace and by sniffing)
> > > * Routing tables after the VPN is up are identical in both cases
> > (ip
> > > route show table main), except for metric, but I also tried to
> > modify
> > > it to be the same and it didn't help either
> > >
> > > So what I'm looking for here is to understand *what*
> > NetworkManager
> > > does to the tun0 interface that renders it unreachable? And why
> > if I
> > > don't allow NetworkManager do its thing and configure the
> > interface
> > > quickly - things actually work?
> > >
> > > My end result is to fix NetworkManager-ssh to do what it should
> > do,
> > > however at the moment I'm puzzled as to why this simple scenario
> > > doesn't even work out of NetworkManager-ssh.Any help more than
> > > appreciated.
> >
> >
> > Hi Dan,
> >
> >
> > NM notices that a device tun0 appears and "assumes" the connection.
> > That process should not modify the interface, in order not to
> > interfere
> > with whoever created and manages the device. It seems that doesn't
> > work
> > well here.
> >
> >
> > It's not clear to me what NM does to interfere with the tun device.
> > But
> > it would be interesting to see that setting tun0 as unmanaged
> > avoids the problem.
> > like:
> > [keyfile]
> > unmanaged-device=interface-name:tun0
> > and `killall -SIGHUP NetworkManager`, and reactivate the SSH VPN,
> > and
> > notice tun0 as unmanaged in `nmcli device`.
> >
> >
> > With upcoming 1.8, NM was changed to improve the situation here
> > (https://bugzilla.gnome.org/show_bug.cgi?id=746440 ).
> > It would also be interesting to see how 1.8 works there.
> > It's actually very simple to build a RPM for Fedora of upstream NM,
> > see
> > https://wiki.gnome.org/Projects/NetworkManager/ Hacking
> >
> >
> > best,
> > Thomas
>
>