combine to WAN's via two Wiregate Connections

[celevra]

Hi,

i try to combine two DSL Connections to one high speed connection.
I have one ISP with 200Mbit and a second one with 200Mbit
my Linux VM has one IP and connects to on vps with 1Gbit.
It connects two wireguard instances. In my local firewall is rule that one port connection is routet through isp1 and the second port connection is routed through isp2

in my linux system i got these interfaces:

speed ~ # ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 56:6f:41:99:00:04 brd ff:ff:ff:ff:ff:ff
    inet 10.99.0.205/24 brd 10.99.0.255 scope global enp1s0
       valid_lft forever preferred_lft forever
    inet 10.99.0.206/24 brd 10.99.0.255 scope global secondary enp1s0
       valid_lft forever preferred_lft forever
5: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none
    inet 192.168.6.2/24 scope global wg0
       valid_lft forever preferred_lft forever
6: wg1: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none
    inet 192.168.7.2/24 scope global wg1
       valid_lft forever preferred_lft forever

on the VPS side it looks like this:

vps~ # ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 96:00:01:5a:72:64 brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    inet 188.34.199.xx/32 metric 100 scope global dynamic eth0
       valid_lft 76933sec preferred_lft 76933sec
    inet6 2a01:4f8:c0c:xx::1/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::9400:1ff:fe5a:7264/64 scope link
       valid_lft forever preferred_lft forever
5: wg1: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none
    inet 192.168.7.1/24 scope global wg1
       valid_lft forever preferred_lft forever
6: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none
    inet 192.168.6.1/24 scope global wg0
       valid_lft forever preferred_lft forever

The systems are both on ubuntu 22.04 with kernel 5.15.0-37

connection between the systems is possible on both adresses (Like 192.168.6.1 and 192.168.7.1)
If i start a filetransfer via scp from 192.168.6.1 to 192.168.6.2 i can see that only one interface is used.
I now want to use both interfaces for one connection.

here is the configured routing:

vps~ # ip rule
0:      from all lookup local
32764:  from 192.168.7.1 lookup 2
32765:  from 192.168.6.1 lookup 1
32766:  from all lookup main
32767:  from all lookup default
vps~ # ip route
default via 172.31.1.1 dev eth0 proto dhcp src 188.34.199.xx metric 100
172.31.1.1 dev eth0 proto dhcp scope link src 188.34.199.xx metric 100
185.12.64.1 via 172.31.1.1 dev eth0 proto dhcp src 188.34.199.xx metric 100
185.12.64.2 via 172.31.1.1 dev eth0 proto dhcp src 188.34.199.xx metric 100
192.168.6.0/24 dev wg0 proto kernel scope link src 192.168.6.1
192.168.7.0/24 dev wg1 proto kernel scope link src 192.168.7.1

speed ~ # ip rule
0:      from all lookup local
32764:  from 192.168.7.2 lookup 2
32765:  from 192.168.6.2 lookup 1
32766:  from all lookup main
32767:  from all lookup default
speed ~ # ip route
default via 10.99.0.1 dev enp1s0
10.99.0.0/24 dev enp1s0 proto kernel scope link src 10.99.0.205
188.34.199.78 via 10.99.0.1 dev enp1s0
192.168.6.0/24 dev wg0 proto kernel scope link src 192.168.6.2
192.168.7.0/24 dev wg1 proto kernel scope link src 192.168.7.2

and here is the mptcp configuration:

vps~ # ip mptcp endpoint
192.168.6.2 id 1 signal dev wg0
192.168.7.2 id 2 subflow dev wg1
vps~ # ip mptcp limits show
add_addr_accepted 2 subflows 2

speed ~ # ip mptcp limits show
add_addr_accepted 2 subflows 2
speed ~ # ip mptcp endpoint
192.168.6.1 id 1 signal dev wg0
192.168.7.1 id 2 subflow dev wg1

could you point me in the right direction to configure mptcp correct so that i get 400Mbit instead of 200Mbit for a single connection?

regards

Celevra

[matttbe]

Hi @celevra

The configuration looks OK, just one question and one comment:

If i start a filetransfer via scp from 192.168.6.1 to 192.168.6.2 i can see that only one interface is used.

Applications have to specify they want to use MPTCP. Did you modify scp and sshd to create MPTCP sockets instead of "plain" TCP ones? Or did you use mptcpize to do that?

(For mptcpize you can find an example there: https://www.tessares.net/mptcp-proxy-setup-at-home-easier-now/ )

and here is the mptcp configuration:

For the ip mptcp endpoint, you need to list the extra IPs) you want to signal on the server side (192.168.7.2 I suppose) and the one you can use to create new subflow on the client side (192.168.7.1 I suppose).

So:

server # ip mptcp endpoint add 192.168.7.2 dev wg1 signal
client # ip mptcp endpoint add 192.168.7.1 dev wg1 subflow

[matttbe]

Hi @celevra

Did you have a chance to look at my previous comment?

Note that if you use mptcpize (or similar), maybe this is due to the way the server is behaving by closing the listening socket and not allowing new subflows to be created, similar to issue #203.

[celevra]

thank you for your response.
I'm actually on holidays and had no chance, i will look in it in around 7 days, thank you!

[matttbe]

Hi @celevra

Did you now have a chance to look at this issue?

[celevra]

@matttbe the tutorial at tessares was exactly what i was looking for
Thank you very much, it is working now as expected :-)