WAN Bonding with ROOter and OpenMPTCProuter

[xdavidx]

Wow, now that was one heck of a long and detailed test.

Tell me about it. And that was just the beginning! I figured someone might benefit from knowing what is possible and some of the hurdles faced.

I've noticed that a VOIP connection's reliability depends on several factors, which includes the VOIP provider's platform/proxy, the client device being used, and of course the internet connection. A call can drop at any one of those points of fault, and not necessarily due to the internet connection alone.

I've been testing with the Linphone application running on my windows 7 laptop. It is pretty resilient. Further testing has shown that when I have the OpenMPTCProuter serving my entire LAN network, and I'm connected over wifi with my laptop, I'm able to reboot the DSL router or the LTE router, and I only get maybe a split second robotic sound glitch in audio quality. The call and conversation continues fine. This is exactly what I hope will happen for the person in the house who is on the phone all day.

I've also shut down the VPN connection (running as a client on my laptop), and Linphone waits long enough for Windows to re-establish its IP address and routes, so the call isn't dropped. I didn't time it, but it was several seconds. Obviously, the audio is silent during that time period. Starting up the VPN client in the middle of a call results in just a momentary gap of audio.

Side Note: For anyone else wanting to test SIP, my setup is:

- Linphone app
- Free Linphone sip account
- Call this sip endpoint: sip:thetestcall@sip.linphone.org

Some of the features that test call system provides are:

- record yourself and play it back
- do a real-time echo test, where everything you say is played back immediately (to test latency)
- play music constantly.

That setup, without any commercial VPN, uses UDP, which OpenMPTCProuter then hands off to the Glorytun UDP tunneling module. I can see this in the UI: Status->Realtime Graphs->Traffic->tun0

realtime_graphs_linphone.png

I thought that was pretty efficient for VOIP, until I saw what the corporate VOIP was doing.

For the person in the house who is on the phone all day for work, their setup is:

- Company laptop connected over wifi to LAN
- Corporate VPN
- Citrix virtual desktops for all the applications (only thing that runs locally (at least I think it is local) is the Avaya telephony system). They lock it down so no other apps can access the internet, not even through the VPN. Everything must be done on the remote desktops. Doesn't seem like the most efficient approach, but that's how they have it set up.

I finally turned on OpenMPTCProuter when they were on a break from working. After they re-established their VPN connection (which may have been due to a timeout, not due to the network switch), everything worked fine over the OpenMPTCProuter system. VPN, Citrix and Avaya all worked.

They said that voice quality was fine. They have always had some latency with the above system, so even if this added a little bit more latency, it wasn't noticeable.

I wasn't able to test rebooting one leg of the WAN connections to see if the VPN and Avaya handled that. I spent most of my time trying to track down which interface their data was using in OpenMPTCProuter. Using some linux tools (iptraf-ng helped a lot with this), I was finally able to watch increases in packets, but *only* when they were talking or the person on the other end was talking. With Linphone, silence is sent, not just voice (from my end, not from the other end, with what I was connecting to).

It would appear the corporate VPN tunnel and the Avaya voip inside that were going over the same Glorytun UDP tunnel in OpenMPTCProuter. But the bandwidth was averaging between 1 Kbps and 10 Kbps. Hardly anything. I couldn't believe it was that efficient.

This is the traffic in the iptraf-ng for Linphone, *not* for Avaya, going through Glorytun:

linphone_iptraf-ng.png

Next step will be to reboot the LTE router and the DSL router (separately), when they are not on a call to see if the VPN can handle that. If that test passes, then I'll cross my fingers and do it when they are on a call.

Some of my early testing was calling a phone number that runs that same test call system, over wifi calling. My wifi calling is poor quality to begin with (combination of phone software and t-mobile probably), so I think most of my glitching was due to the wifi calling, not due to the OpenMPTCProuter setup. And I think I did most of that with only the DSL link in the equation, which is a slower link with only 0.8 Mbps upload speed. I don't think I've tried it yet over the dual WAN setup.

Seem like running a VPN on top of OpenMPTCProuter is a little over complicated, giving you a hit on both speed and latency. There has to be a better way to bypass the streaming provider's IP security without having to use a clusterfuq of VPN's on top of VPN's. lol

This whole thing is very complicated. This isn't for the faint of heart. I mean, some of the setup isn't too bad, but if you need to dig into it more, there is a *lot* going on under the covers. On the other hand, it provides redundancy/resiliency, in real-time, plus the extra speed of multiple WAN links. So what a person loses in speed of the VPN on top of it, they more than make up for in the extra WAN link, I'm guessing. I'm not sure how much latency the VPN adds on top. I don't think it is that much, since I've picked a VPS and VPN provider in the same large city, which is also where my LTE data enters the internet.

You can bypass OpenMPTCProuter by going to: Services->OMR-Bypass. Depending on your network setup, this may or may not be easy. I have double NAT going on, so that complicates it, since the OpenMPTCProuter doesn't see my LAN IP addresses -- it sees them as the router's address between OpenMPTCProuter and the rest of my network. In my case, however, I don't want to bypass the OpenMPTCProuter solution for streaming. I'd have to set it up for every streaming device (TVs, Rokus, etc.), and then I'd have to choose either a difficult destination based rule for putting streaming content out a regular WAN link, or a simpler, source IP based rule, which would mean phones and computers wouldn't benefit from the added speed of the multi-WAN connection. And streaming wouldn't benefit from the multi-WAN connection either (at least not in an aggregated sense).

Having said that, I'm very open to other ideas, if there is some other way to fake out the streaming providers. I'm guessing there are probably some VPSes they haven't caught onto yet, so that might be one option, but I'm also guessing they don't advertise this, for obvious reasons.

Having a VPN client running on the VPS server, so the multi-path stuff is chained to the VPN, instead of them being used over the top of each other, would seem to be more efficient. However, that would it much more difficult to route something through the multi-path link, but *not* through the VPN. It would all be based on target addresses, not source addresses, since i'd be very divorced from my LAN by the time the packets came out the other side of the VPS software. I did open a ticket on github for OpenMPTCProuter, to see if I could get help chaining it this way, however.

I tried running the VPN client on the OpenMPTCrouter device, but I couldn't get it to work properly. The VPN came up, but the firmware didn't route my traffic over that VPN tunnel, on top of the multi-path link. I'm thinking maybe it only has openvpn capabilities so that it can use that as a different option for its own tunneling, not for a commerical vpn to be used like the way I'm trying to do. I asked about this in the ticket too.

I did get the commercial VPN tunnel running in my router that is running Tomato, that sits between OpenMPTCProuter and the rest of my LAN. But that is an old linksys E3000. It doesn't have the horsepower. My speeds were very slow on speed tests and I traced it back to maxing out the CPU in that router. It did work, however, and Netflix, Hulu, etc. were happy.

I have to decide if I want to run a VPN client on a device that can handle it, on the LAN side of OpenMPTCProuter, or if I want to try to get the VPN client to chain together with the OpenMPTCProuter server software in the VPS. There are pros and cons to each.

Also, upgrading to a 5G modem would not only improve your speed and throughput for your family, but also reliability. You may not have 5G service in your area yet, but rest assured it's coming soon to a tower near you (in that creepy movie announcer guy's voice... muaahh!).

I probably won't benefit from 5G, directly, for some time, where I'm located. The higher frequencies won't reach my house, so I'd need them to roll it out for the lower frequencies here. I could possibly benefit from the other 4G features that the 5G modems have, however, regarding greater carrier aggregation, more MIMO, etc. But given the costs of the modems right now, I think I'll invest in better antennas and a less expensive 4G modem for my second LTE connection. When 5G modems come down in price, I'll look into upgrading to see if I can take advantage of the better 4G features. That will give the providers time to add support for greater CA and MIMO in the towers too.

If you're saying that a different modem might not drop connections, so I wouldn't need all this other stuff for reliability, that is something I've considered. I may go with a Quectel modem (currently using a Sierra). That, combined with a beefier router based on different hardware might make a difference. Some of the disconnects are due to router firmware crashes (more rare). Others are due to the modem being disconnected and the router having to start it up again. Whether that is due to the tower doing the disconnect, or modem firmware, or router firmware, or router hardware, is an unknown.

*If* my second LTE system doesn't show this problem, then I can route the work computer through its connection and use the multi-path stuff for the rest of the house for the added speed.

At the end of the day, the multi-path speed does come with a lot of other complexities and side-effects that people need to be willing to live with.

[xdavidx]

I've tried to figure this out, but haven't been successful: How do you run all the data through a VPN between the two? I've got two ATT and one Verizon plan.

What is the purpose of the added VPN in this case? There should already be encryption in the OpenMPTCProuter tunnel.

If it is so you can access devices at home, when you are at the lake house, that might just be a matter of network routing.

To answer your question, however, see my above posts. You would be doing a similar thing. The difference is that you'd run a VPN server in your home, rather than using a commercial VPN service. If I'm understanding the question correctly, your path would be:

Lake house LAN devices -> VPN client (either running on multiple end devices or in a router) -> OpenMPTCProuter router -> ATT+ATT+Verizon -> Internet -> fiber -> Home router -> VPS Server (either running in the current home router or in another router) -> Home LAN devices

What kind of speed do you currently get out of the ATT+ATT+Verizon connection? What kind of modems and antennas are you using? (Nothing to do with the VPN question, but just general curiosity.)

[docderwood]

What is the purpose of the added VPN in this case? There should already be encryption in the OpenMPTCProuter tunnel.

If it is so you can access devices at home, when you are at the lake house, that might just be a matter of network routing.

To answer your question, however, see my above posts. You would be doing a similar thing. The difference is that you'd run a VPN server in your home, rather than using a commercial VPN service. If I'm understanding the question correctly, your path would be:

Lake house LAN devices -> VPN client (either running on multiple end devices or in a router) -> OpenMPTCProuter router -> ATT+ATT+Verizon -> Internet -> fiber -> Home router -> VPS Server (either running in the current home router or in another router) -> Home LAN devices

What kind of speed do you currently get out of the ATT+ATT+Verizon connection? What kind of modems and antennas are you using? (Nothing to do with the VPN question, but just general curiosity.)

******

My setup is pretty much as you described, and I can get to my main house network from the Lakehouse w/o issue.

I have two WireEng Yagi's with Sierra Wireless for Verizon, this is an old grandfathered unlimited data plan.

For ATT: One Pyongting flat panel (2 N female out); The other ATT uses The Wireless Haven large dual ant with the router in the Ant box with POE. Both ATT use EM20

My bandwidth is highly variable....likely related to tower congestion. I'll struggle to get 15-20 down between all three connections in the evenings, presumably from streaming. During the day, I'll get 50-100 down (presumably everyone at work). This is a very rural location.

Latency really sucks no matter what I've tried, fast.com typically 200-300ms. I can't figure out why.

I've got a new 4X4 MIMO The Wireless Haven antenna and an EM160 that I'm going to try next, probably aim at a different ATT tower (after trying Verizon 4X4).

With good aiming, I typically get good signal.

I've used Pepwave before and continue to think about trying to pick up one of their speed fusion units....but very pricey. Their software is really well designed, though.

[xdavidx]

My setup is pretty much as you described, and I can get to my main house network from the Lakehouse w/o issue.

What would the purpose be of adding the VPN to this? Happy to help get it set up.

That's a cool combination of equipment you have.

My bandwidth is highly variable....likely related to tower congestion. I'll struggle to get 15-20 down between all three connections in the evenings, presumably from streaming. During the day, I'll get 50-100 down (presumably everyone at work). This is a very rural location.

Latency really sucks no matter what I've tried, fast.com typically 200-300ms. I can't figure out why.

Is that speed and/or latency from the lake house without OpenMPTCProuter, or is that what you get when you are going through OpenMPTCProuter?

I've got a new 4X4 MIMO The Wireless Haven antenna and an EM160 that I'm going to try next, probably aim at a different ATT tower (after trying Verizon 4X4).

I'll be interested to hear if AT&T supports 4x4 MIMO. I know they were really pushing for that in 2020, along with greater carrier aggregation. Speaking of which, since you have the EM20 modem, how much carrier aggregation do you get out of that?

I've used Pepwave before and continue to think about trying to pick up one of their speed fusion units....but very pricey. Their software is really well designed, though.

I looked at their stuff a long time ago, but it didn't make money-sense for me at the time, because I was only dealing with two 6 Mbps DSL links. I used a build of Tomato in my router that load balances WAN links. Not the same as aggregating, but it got us by for a period of time, until the phone company upgraded the DSL capabilities at the other end of the phone lines.

[docderwood]

What would the purpose be of adding the VPN to this? Happy to help get it set up.

As much as anything, to figure it out. I see the open VPN option etc but am not sure what that gets me over what I have now. Verizon fails the MPTCP check, but if I enable VPN for it no data flows.

To be quite honest, I've never understood what is going on under the hood of OpenMPTCP. For example, the status page (below) shows only some data going thru the VPN. What data flows thru the VPN and how is that decided? The bandwidth does not seem to be additive, it seems to be balanced between all three connections.

Cropped.jpg

That's a cool combination of equipment you have. Thanks!


Is that speed and/or latency from the lake house without OpenMPTCProuter, or is that what you get when you are going through OpenMPTCProuter?

That speed and latency is what I get going through OpenMPTCP (both the VPS and OpenMPTCP are running on I7 boxes with plenty of excess processing power) with all three connections together. The latency under load is the worst on fast.com (300-500ms). DSL reports has lots of buffer bloat mirroring fast.com. Google speediest shows the best throughput, in general.


I'll be interested to hear if AT&T supports 4x4 MIMO. I know they were really pushing for that in 2020, along with greater carrier aggregation. Speaking of which, since you have the EM20 modem, how much carrier aggregation do you get out of that?


I typically seem to get 2XCA (don't watch super close). It will be interesting to see what the 4X4 MIMO with EM160 does on Verizon vs ATT (but will be three weeks until I get back out there).


I looked at their stuff a long time ago, but it didn't make money-sense for me at the time, because I was only dealing with two 6 Mbps DSL links. I used a build of Tomato in my router that load balances WAN links. Not the same as aggregating, but it got us by for a period of time, until the phone company upgraded the DSL capabilities at the other end of the phone lines.

Peplink is what I started with @ the lake with a single Version LTE connection. ATT wanted $120,000 for fiber buildout. A WISP has been telling me "soon" for two years. Maybe Starlink will fill a void at some point. Charter won the FCC bid for subsidies in the area (probably not great).

[latic]

Hoping someone can give me a little advice on OMR-Bypass, because I've been stuck in compiling my own version to get OMR + Mwan3 headache mode for the last 24 hours.

Last week I moved from my trusty EdgeRouter X over to a mini PC (celeron + 6gb ram) running OpenWRT. Initially I found Mwan3, replicated my load balanced setup with edgeos and everything was running great. My aim was to run MPTCP, but only for one or two devices.

From my understanding you can use OMR-bypass to stop individual IP addresses from using the tunnel, does bypass enable you to set any load balance type structure of ips that are bypassing?

My groups are generally Main (2xSprint + ATT Backup) TV (ATT + 1xSprint backup) Work (2xSprint + ATT + DSL) Gaming (DSL only), ideally I want to use the MPTCP for work and to gain a static IP, but dont want to lose out on the overly complicated (but headache saving) load balance groups I've always used.

[mtl26637]

Last week I moved from my trusty EdgeRouter X over to a mini PC (celeron + 6gb ram) running OpenWRT. Initially I found Mwan3, replicated my load balanced setup with edgeos and everything was running great. My aim was to run MPTCP, but only for one or two devices.

Confused on exactly what software and setup you are trying to accomplish? Openwrt with mwan3 is very different than 'OMR'. OMR is a complete software/firmware standalone package that gets flashed to your miniPC. Once installed you can setup everything you are looking for through the qos and bypass menus. Don't think there is a need for using mwan3. Under the omr-bypass menu you can set up to bypass using several methods such as: Domain/IP and Networks/Ports/MAC/ASN/Protocols and Services.

[latic]

Completely understand OMR is different than Mwan3. I'm looking to have OMR on top of my existing load balanced groups, so only one group/device will using the OMR tunnel.

It looks like OMR-bypass can select a particular connection, but then I'd lose the ability to have failover / sticky load balancing on the groups that bypass OMR, without Mwan3.

Was hoping someone had already done this type of setup.

[mtl26637]

I did have similar setup before but had OMR separate from mwan3 rather than both integrated together. Had some devices connected to mwan3 router and some connected to OMR router but this was tricky if any of them need to talk to each other on the separate networks.

Fyi, not sure about the load balancing portion if ran through the OMR-bypass but failover would be taken care of if you leave the interface unselected. It would then default to 'master' and if it was to fail the master interface is automatically modified so it would automatically switch.

[latic]

Did you end up just pushing everything out of OMR and let QOS deal with devices?

My big thing right now making sure that we still have a failover on the TV group as if I'm out the country and the ATT connection breaks the TVs need to work.

[mtl26637]

Initially I had a double NAT situation where OMR was behind mwan3 router but that was a headache to say the least. Ultimately I caved and made the OMR device the main router and pushed all devices through it. To bypass say a netflix tv, I set the netflix tvs ip in OMR-bypass and left the interface set to 'none'. By doing this the tv uses the 'master' interface at all times. If the OMR router's master interface goes down it automatically switches the master interface to another connection, therefore, switching the tv's interface to a working connection. The only thing this doesn't account for is individual ip 'load balancing' on the tv's connection but I wasn't too worried about that. That being said, overall load balancing is already controlled under system/openmptcprouter/other settings/master interface selection/balancing. If you needed further fine grain control you can take it a step further and look into OMR'S SQM and DSCP settings. I believe you would then be able to manage and balance the different type of traffic and services over the wan interfaces.

[mtl26637]

I just took another look at my OMR-bypass settings since it had been awhile. It looks like I mainly avoided using bypass settings via individual IP as that is where you can run into issues. Most of my 'bypass' settings are via 'Domain' such as amazon domain. Out of a year or more of using OMR I only have around 5-10 domains that were needed. Netflix was the bitch of them all as I had to enter all of there domains and also set the 3 "ASN" for netflix to work without hiccups.

By using domain bypass rather than via IP the only traffic that gets bypassed is traffic to that domain rather than all traffic for that IP/device. I also just ran a quick test to see what the result would be if the wan interface that a certain domain was set to went down. To test I set the domain of "ifconfig.co" to use wan interface 'eth2'. I then removed 'eth2' and tested the domain. The result was the ip of the master interface so it seems if your specified wan interface does go down it defaults back to the master interface.

Sorry if you might know all/most of this just thinking out loud in case it may help others down the road.

[latic]

All your replies have been hugely helpful.

I guess I've always had to manage the bandwidth on the network (to make it usable) that Im scared about changing for no reason!

Sounds like I need to let go. Put OMR in place, Let QOS do its thing and put together some domain overrides WITHOUT controlling where they go!

[mtl26637]

LOL, I know exactly what you mean, it took me forever to give in and remove my nice new shiny netgear router as my main router and replace it with what was a raspberry pi 4 at the time. Now that I have, i'd never go back! Kind of unrelated but just as an fyi to anyone wanting to test OMR, that netgear was a supported WRT32X router for OMR and I used it for a long time. I seemed to have several issues with settings and firewall etc, and my top speeds were limited to around 200mbps. I finally switched to running a 'Protectli Vault 4' minipc with OMR installed and its been stable since day one and also no more top speed limits!

[latic]

I managed to pay $165 for:

Mini PC Intel Celeron J3455 6GB DDR4, 128 GB SATA SSD, Mini Desktop Computer, AWOW AK34 Micro PC, Windows 10, Quad Core, Dual Gigabit Ethernet NIC, Dual HDMI, 5X USB3.0, 4K UHD, Bluetooth

Before hand running via edgeos I'd consider good, but it looks like now the amount of load balancing that was needed was actually causing it to choke a little (3LTE + DSL).

[latic]

All looking good today, Using Vultr which is in Seattle and pings aren't bad.

Connection (even with Sprint stopping band 41 CA) is at 65mbps down and 20mbps up.

Did anyone ever figure out what "Residential" VPN/VPS were being used?

[cagordon1985]

Been tinkering with this a bit.

200mbps and under, anything that is in the supported devices list will work more or less. PI4 can get upto 250mbps.

For up to 400mbps throughput from WANs you want a dedicated 1gb NIC and gb switch backplane and modern dual core PC dedicated to OMR. At worst.

For 500mbps +, you are going to need at worst a 2.5gb NIC and switch backplane. Also very quality VPS.

It is very easy to go unnoticed, OMR seems to double xmit almost all traffic on both ends locally by default. (you can cause even more multi-xmits by modifying mptcp settings) So once you get to 250-400mbps + overhead you start to saturate 1gbps equipment. 500mbps+ WAN you have started to exceed gig capabilitys in many instances of transmission.

Turn on vray and bbr2 you are going to need a even beefier VPS and home/router CPU.

[mtl26637]

It is very easy to go unnoticed, OMR seems to double xmit almost all traffic on both ends locally by default.

What exactly are you seeing here? If it is happening to that extreme, yes it has been unnoticed. Now on the other hand it seems in the beginning when I started messing with OMR I was seeing much more traffic such as double local, however, at the time I think I had my OMR setup behind my main router and chalked the extra traffic up to that. I'm gonna look into this further since it does explain the speed limits I've seen with the different pieces of hardware.

[cagordon1985]

Using my VM is where I noticed the loopback traffic, If my WAN recieve side was 390-400mbps, I was seeing 100% nic utilization. The connection causing WAN saturation was external to the hypervisor.

Also it seems this router can cause tons of switch level saturation as I have noticed WAN speed fall off, if I start streaming a fairly high quality local file. (75-100mbps HVEC) The media-server and hypervisor are separate hosts sharing switch.

Changed to a 8GB capacity switch, still only 1gb ports, WAN speed back to normal, router NIC still 100% utilized.

Combined WAN speed from 300mbps-450mbps, I started seeing my gig equipment struggle.. I have also confirmed my VPS is capable of saturating all three WAN's directly at once. Also the VPS has 900ish MBPS throughput available.

I will reply once I slap in a 10gbps card and switch. Speedtest should start reflecting hopefully 80%+ of my combined WAN capability.

[latic]

Had a great working system and then got issues. It ended up being Vultr! Swapped to RackNerd and its been much better since then.

Really wish I'd checked this straight away. Does anyone know a decently priced provider that has residential IPs?

[kalistian]

I am currently trying to set up a mini desktop as my local OMR, and whenever i plug the device into my switch (using eth0), it doesn't seem to connect to the network at all. the port is blinking orange continuously. any idea what I need to do / what the issue is? all the other devices on the switch connect just fine.

Any advice would be appreciated.

[jppawlak]

Has anyone used their residential internet to set up the VPS? I have gig fiber at home but travel in the RV so wouldn't mind hosting it at the house (and avoiding the fun with the domain blocks). Hadn't had much luck making getting to to connect to a local Ubuntu instance with port forwarding.

[bnhf2]

Has anyone used their residential internet to set up the VPS? I have gig fiber at home but travel in the RV so wouldn't mind hosting it at the house (and avoiding the fun with the domain blocks). Hadn't had much luck making getting to to connect to a local Ubuntu instance with port forwarding.

Yes, I have an OMR VPS running using Debian 10.8 Server in a VM under Windows 10 at a residence. Ports 65500, 65001, 65101 and 65222 need to be forwarded for admin, glorytun, shadowsocks and ssh respectively. When I set mine up (earlier this year), a static IP was required for the VPS -- though support for a dynamic IP was planned and may be possible by now.

[bnhf2]

I am currently trying to set up a mini desktop as my local OMR, and whenever i plug the device into my switch (using eth0), it doesn't seem to connect to the network at all. the port is blinking orange continuously. any idea what I need to do / what the issue is? all the other devices on the switch connect just fine.

Any advice would be appreciated.

When you say "my local OMR", are you trying to deploy OMR as your primary router -- or are you trying to create an OMR VPS to be accessed from an OMR router located elsewhere?

[kalistian]

When you say "my local OMR", are you trying to deploy OMR as your primary router -- or are you trying to create an OMR VPS to be accessed from an OMR router located elsewhere?

appreciate the response, i was talking about it being the primary router, but i figured it out -- was a DHCP issue. i think i got it working, though my vultr Los Angeles VPS keeps getting clogged for some unknown reason. few times a day, ping goes up from an average of 40 to 300-600ms for a minute or so. every other part of the system at those times seems to be operating just fine, so i'll prolly have to find an alternative VPS to see if that fixes it.

[iceburglar]

This is like a DIY version of speedify? Why choose this over speedify or vice versa?

[bnhf2]

This is like a DIY version of speedify? Why choose this over speedify or vice versa?

It's been a couple of years since I tested Speedify, but the performance was not nearly on par with what you can do with OpenMPTCProuter. Speedify is also much more oriented towards running on an individual client computer (i.e. a single device gets the benefit of multiple WANs). It is possible to setup Speedify as a router, though I believe it's not often used that way. OMR is designed to become your primary router (and is based on OpenWRT), so your entire network benefits from the aggregation of multiple Internet connections, along with many other benefits of an OpenWRT derivative.

BTW, stock OpenWRT is much more DIY than OMR -- OMR is a more complete solution from the start. The only thing one might consider "DIY" is the setting up of a VPS, which you don't need to do with Speedify. However for those familiar with the concepts of a virtual machine, the OMR provided installation script makes the setup trivial on supported OS and VPS platforms.

[iceburglar]

It's been a couple of years since I tested Speedify, but the performance was not nearly on par with what you can do with OpenMPTCProuter. Speedify is also much more oriented towards running on an individual client computer (i.e. a single device gets the benefit of multiple WANs). It is possible to setup Speedify as a router, though I believe it's not often used that way. OMR is designed to become your primary router (and is based on OpenWRT), so your entire network benefits from the aggregation of multiple Internet connections, along with many other benefits of an OpenWRT derivative.

BTW, stock OpenWRT is much more DIY than OMR -- OMR is a more complete solution from the start. The only thing one might consider "DIY" is the setting up of a VPS, which you don't need to do with Speedify. However for those familiar with the concepts of a virtual machine, the OMR provided installation script makes the setup trivial on supported OS and VPS platforms.

Cool! I've been experimenting with running speedify on my laptop to bond my ATT Tablet plan ( Quectel RM502Q-AE ) , T-Mobile One Plus International ( Telit LM960 ) and T-Mobile One Plus International ( Smartphone tethering ). I might look into doing this eventually, it would be cool to have it all handled on a router instead of the computer like you said.

Say my ATT modem was giving me a downstream of 50Mbps in a given location, if I were to add another like modem and plan should I expect it to also simultaneously deliver another ~50Mbps connected to the same tower/bands or would there be some sort of congestion/saturation be expected to cause poor scaling?

In speedify it seems common that each device will only contribute maybe 1/2 or 3/4 of their solo throughput capability to the bonded connection and I'm not sure if its their server limitation, bonding overhead, limitations of my PC, or maybe some kind of RF saturation/interference.

[bnhf2]

Cool! I've been experimenting with running speedify on my laptop to bond my ATT Tablet plan ( Quectel RM502Q-AE ) , T-Mobile One Plus International ( Telit LM960 ) and T-Mobile One Plus International ( Smartphone tethering ). I might look into doing this eventually, it would be cool to have it all handled on a router instead of the computer like you said.

Say my ATT modem was giving me a downstream of 50Mbps in a given location, if I were to add another like modem and plan should I expect it to also simultaneously deliver another ~50Mbps connected to the same tower/bands or would there be some sort of congestion/saturation be expected to cause poor scaling?

In speedify it seems common that each device will only contribute maybe 1/2 or 3/4 of their solo throughput capability to the bonded connection and I'm not sure if its their server limitation, bonding overhead, limitations of my PC, or maybe some kind of RF saturation/interference.

OMR is about a close to linear (in most cases) as one could hope for, with typical WAN aggregation of 90% or better, compared to the sum of the individual link speeds. Having said that, you do need to be careful with cellular connections that you're not trying to combine two identical devices from the same carrier. Two identical modems on the same data plan, hitting the same tower, will aggregate poorly -- and likely will be no faster than they would be individually. Using directional antennas to hit different towers, using modems that'll connect on different bands, or (best of all) different carriers, will keep speeds fast. OMR supports combining up to 8 different WAN connections.

[mtl26637]

Having said that, you do need to be careful with cellular connections that you're not trying to combine two identical devices from the same carrier. Two identical modems on the same data plan, hitting the same tower, will aggregate poorly -- and likely will be no faster than they would be individually. Using directional antennas to hit different towers, using modems that'll connect on different bands, or (best of all) different carriers, will keep speeds fast. OMR supports combining up to 8 different WAN connections.

One might think this the case but in my experiences it hasn't seemed to matter much. I have had at one time 5 different devices pulling all from the same tower (3 att, 1 vzw, 1 tmo) and they all aggregated nicely. All of them even pulling from same mixture of bands (mostly b66). It all has to do with the scheduling at the tower. I'm thinking as far as the tower sees things, these are 5 different customers and has to allocate resources accordingly. I'm in a rural location so the tower isn't as congested as something in the city, however, this tower serves a large area for 3 different carriers so I'm sure it still keeps busy. All in all, I'm sure it would be better if spread across providers and towers but on the other hand if not feasible then even though the aggregated devices are all using the same bands and such, it still gives each device its fair share of bandwidth, at least in my area.

[packlet]

Now that I have several working Rooter/modem/SIM setups (on VZW & AT&T,) I'm looking to try bonding with OpenMTCPRouter. I'd run the VPS on my own hardware behind our home gigabit fiber from Sonic (tested over 800mbps in both directions), and looking to aggregate up to 300mbps from 2-3 cell connections, or more in the future.

I have an old spare 'media' mini PC sitting around with these specs

Device name AOpen_DE965-HG
Processor Intel(R) Core(TM)2 Duo CPU T6600 @ 2.20GHz 2.20 GHz
Installed RAM 3.00 GB (2.87 GB usable)
System type 32-bit operating system, x64-based processor

Could this support those speeds as VPS (possibly even replacing my main router, currently a LinkSysy EA8500 with Gargoyle) or would such a setup need more horsepower?

Can one assume you'd need pretty much the same horsepower at both ends of OMR? or should the VPS be 'beefier' to handle the VPN de-coding/de-aggregation than the cell-side?

TIA, for any folks experienced with this... Paul.

[bnhf2]

I have an old spare 'media' mini PC sitting around with these specs

Device name AOpen_DE965-HG
Processor Intel(R) Core(TM)2 Duo CPU T6600 @ 2.20GHz 2.20 GHz
Installed RAM 3.00 GB (2.87 GB usable)
System type 32-bit operating system, x64-based processor

Could this support those speeds as VPS (possibly even replacing my main router, currently a LinkSysy EA8500 with Gargoyle) or would such a setup need more horsepower?

Can one assume you'd need pretty much the same horsepower at both ends of OMR? or should the VPS be 'beefier' to handle the VPN de-coding/de-aggregation than the cell-side?

The specs on your media mini PC seem like they would be adequate for a VPS, given that many are running theirs on one virtual core with 1GB of RAM -- including me. You'll need to be able to install either Debian or Ubuntu server on it, but if you can do that, you should be in business. You would not want to use this device as router, and in fact it's not recommended to use it for anything else. When I set mine up, a static IP was required, but that may have changed by now.

On the OMR side, you can definitely use whatever device you setup as your primary router, and in fact I'd say it's recommended (though not required). Options here are more limited hardware-wise, and you'll need to choose from the list of supported routers, RaspberryPi SBCs or use an x86. Horsepower requirements are higher on this end, but I've had success getting speeds in the range of what you're looking for on an RPi4 and on an i3 industrial PC.

[packlet]

cool, thanx for that input! Any benefits/preference for modems in their own router boards (eg: mikrotik rbm33g or 11g connected to the OMR via ethernet) Vs. USB modems only connected directly to the OMR (using the OMR's USB port/s ala Rooter)? I read you've used the ethernet/vlan route, but wondering if one can directly add USB modems to the mix ? or bad idea?...

[bnhf2]

cool, thanx for that input! Any benefits/preference for modems in their own router boards (eg: mikrotik rbm33g or 11g connected to the OMR via ethernet) Vs. USB modems only connected directly to the OMR (using the OMR's USB port/s ala Rooter)? I read you've used the ethernet/vlan route, but wondering if one can directly add USB modems to the mix ? or bad idea?...

If you're in an area with strong cellular signals, then USB modems connected directly to an OMR router work great. Often, however, one of the motivations for aggregating cellular connections is in areas where the cellular isn't so great. In those situations, being able to place the cellular routers outdoors, facilitates keeping coax antenna runs runs short which allows for the use of high-gain antennas with minimal cable loss.

The MikroTik boards mentioned can run ROOter, have gigabit ports (one with 24V PoE support) and have slots for one or two mPCIe or m.2 modems. They work well in waterproof enclosures, and the price is certainly right. I've done many builds using these boards, with zero failures to date.

Below is an example of an RBM33G in a vented waterproof box, pole mounted, with cross-polarized directional antennas. The low-loss coaxial cables are only about a foot long for MIMO and Carrier Aggregation goodness.

IMG_20200220_120028672_HDR (resized).jpg

[packlet]

That's thing of beauty! I totally get the benefit of keeping those antenna leads as short as possible, especially in a low-signal area. I'm getting over 200mbps with an RBM33g and on-board EM7565 with AT&T 'tablet' plan SIM and similar antennas (a pair of LPDA's @ 90 degrees from each other). I'm using ~15m antenna leads, coiled up, as that's what I had lying around for testing. Then again, the tower is (currently) about 400 feet away in direct LOS. At those speeds, there's still CPU headroom available on this processor. I've had 2 modems working in the unit as well, but have yet to grok how to properly 'route' the traffic, either via load balancing or (preferably) one modem each to separate ethernets (or VLANs) to be bonded externally, as this board certainly does not have the grunt to do VPN at anywhere near those speeds...

[packlet]

oh, @bnh2, I've been pondering the 'network of gateways' thing you do with each modem/router on it's own subnet using VLANs.

if all the WAN r00ters are on the same physical ethernet, couldn't one also just give each r00ter a separate ip all on the same subnet? eg, 192.168.1.2, 1.3 and 1.4. instead of separate subnets using VLANs. i understand the VLANs separate the broadcast domains, but is that needed if the only things on the subnet are the WAN r00ters and the openmtcprouter?

thanx for any further details or clarification on how/why that's beneficial for this setup... Paul.

edit: maybe this is needed because the openmtcprouter implementation assumes each WAN gateway is on it's own separate subnet and there's no simple/obvious way to make it work with all IP's on the same subnet?...

[bnhf2]

oh, @bnh2, I've been pondering the 'network of gateways' thing you do with each modem/router on it's own subnet using VLANs.

if all the WAN r00ters are on the same physical ethernet, couldn't one also just give each r00ter a separate ip all on the same subnet? eg, 192.168.1.2, 1.3 and 1.4. instead of separate subnets using VLANs. i understand the VLANs separate the broadcast domains, but is that needed if the only things on the subnet are the WAN r00ters and the openmtcprouter?

thanx for any further details or clarification on how/why that's beneficial for this setup... Paul.

What you are suggesting might work, if you shutoff the DHCP server on each of the gateways, and set its IP address as per your example.

It's been a few years since I did my first of these "Network of Gateway" setups. I was originally using a Cradlepoint MBR1400, and my recollection is that the only way there to have multiple WAN Gateways on the same Ethernet port was to use VLANs. That became my architecture, and I've used it to good effect ever since -- and I know it works. The WAN side of your typical router is much more locked-down than the LAN side (for obvious reasons), and the VLAN approach keeps one from running afoul of WAN-side restrictions.

VLANs aren't really any more difficult to setup than the changes you're suggesting to each ROOter box, and it's a proven approach with no performance penalties. You may find with everything on the same subnet, there's a tremendous amount of rejected traffic on gateways that aren't the destination of that traffic. It wouldn't be difficult to test out though, as you're setting things up. Speedtests would be crucial to be certain you're not killing your performance.

And of course, all of this is only applicable for those wanting to have multiple gateways on the same physical Ethernet network, dedicated to WAN devices.

[packlet]

thanx for the reply. "And of course, all of this is only applicable for those wanting to have multiple gateways on the same physical Ethernet network, dedicated to WAN devices." yes, thinking of 2-3 rooter setups in external enclosures (near the antennas!) and only running one ethernet down/in to the openmtcprouter. RBM33g has 3 ethernet ports, after all. Will report here once I've got my new fanless x86 pc to use as OMR, currently on a slow boat from CN...

[islander261]

Hello

I am looking for a little advise and help. We have an LTE connection for internet right now. WE-826-T, MC7411, Calyx sim, connected to the house through 100Mbs PtP link. Tower is 3km away and my antennas are in the Fresnel zone just at LOS with lots of trees in the way. The tower has bands 2, 4, 12, 66, 71. Usual connection is band 2 with band 71 for CA. We get 3 to 50 Mbs down and 0 to 5Mbs up. We have very slow service on weekends and holidays when the tower (one and only within range and topography) is clogged with part time residents coming from the city. We just received Starlink and want to integrate it into our network while keeping our LTE connection to smooth over Starlink dropouts.

The cost of a Peplink or Speedify is really too high for us to use so an open source solution is the only thing practical for us. Is OpenMPTCP the only open source way to combine the the two connections into one? Can you change the IP address of the open source router from 192.168.100.1 as this conflicts with the Starlink user connection? Can I run this on an Rpi4 and get >50Mbs reliably? TIA

John

[bnhf2]

We just received Starlink and want to integrate it into our network while keeping our LTE connection to smooth over Starlink dropouts.

The cost of a Peplink or Speedify is really too high for us to use so an open source solution is the only thing practical for us. Is OpenMPTCP the only open source way to combine the the two connections into one? Can you change the IP address of the open source router from 192.168.100.1 as this conflicts with the Starlink user connection? Can I run this on an Rpi4 and get >50Mbs reliably? TIA

John

Your objectives should be very achievable with OpenMPTCProuter (OMR). And yes, OMR is about the only true WAN bonding solution in the open source world. No problem to change the IP address of OMR, if you can't change Starlink. An RPi4 will work for you, though you may find its single Ethernet port something of a limitation.

I'm signed up for Starlink as well, though it'll be mid next year before I get my kit (was supposed to be this year, but just got an e-mail saying there's a delay for my area). Which leads me to a couple of questions: I'm assuming here that the Starlink box has an Ethernet port? Also, does it broadcast WiFi, and if so, can that be disabled? And, the final item, which you probably won't be able to answer until you test it out, does it block the MPTCP protocol? (I don't think there's much of that happening anymore, but Starlink is new, and I haven't heard of anybody using it with OMR -- yet!)

[islander261]

OK, this is my third try to reply.
Thank you for the info.

The new V2 rectangular dish that is shipping to all U.S. customers now does not have a wired ethernet port. You must buy a wired ethernet dongle that installs between the dish and the router to get a wired port. The problem is that they haven't shipped any of the dongles yet so nobody outside of Starlink actually knows how they will work. The only user accessible settings are the SSIDs (2.4g and 5g radios), security type and passwords. The wifi is on all the time.

[bnhf2]

OK, this is my third try to reply.
Thank you for the info.

The new V2 rectangular dish that is shipping to all U.S. customers now does not have a wired ethernet port. You must buy a wired ethernet dongle that installs between the dish and the router to get a wired port. The problem is that they haven't shipped any of the dongles yet so nobody outside of Starlink actually knows how they will work. The only user accessible settings are the SSIDs (2.4g and 5g radios), security type and passwords. The wifi is on all the time.

Bummer. This means you'll need to use some sort of "WiFi as WAN" approach to make the Starlink router into something you can use as a gateway on another router (in this case OMR, which will be your primary router). Perhaps somebody else that's been monitoring this thread has done something like this with OMR and can add to the discussion.

There are a number of ways to approach this, but if I were setting it up, I'd use a Ubiquiti NanoStation M5 Loco in "Station" mode. You'd connect the NanoStation to an Ethernet port on your OMR router (there'd need to be a 24V PoE injector in-between), and configure the NanoStation to connect to the Starlink router via WiFi. I've done this type of thing many times using OMR in my RV, to use campground WiFi as one of my WAN gateways combined with whatever cellular links are available.

Another, simpler approach equipment-wise, would be to use the Raspberry Pi's onboard WiFi in client mode -- the problem is that I'm not sure if this is possible with OMR. The onboard WiFi on an RPi has never been very good in AP mode, so this could make good use of this chip -- if it's possible. Hopefully somebody else has done this.

[islander261]

Thank you.

My first task is to get the Starlink cable run into the house. Presently the router is by a window on a covered porch. So far it is passing the most critical test, providing the wife's social media connections, with ease.

I was considering trying to use either a bridged router or Rpi4 to connect to the wifi once I get the Starlink router indoors (today hopefully).

The race is on as to which gets here first, the Starlink ethernet dongle or the 802.3bt passive POE injector. There have been a couple of reports about a couple of dongles sighted in wild but no pictures and the dates are highly suspect. I am looking forward to cutting the cable to verify the supposed non standard 802.3bt POE++ wiring.

We will be leaving on our annual snowbird journey in a couple of weeks to the SW so I will not be back on the OMR project seriously until late March. Squishy will stay at home because I don't want to loose my home connection when moving. We use our LTE modem and router on the road with great success.

[islander261]

I am back from the SW and ready to start migrating to an OpenMPTCProuter install. Sorry I know my questions are silly to a network engineer but I have read most of the documentation for using vlans and Mwan3 on line and they are clear as mud to me.

Presently my network is connected to Starlink using the Starlink Ethernet adapter (dongle). I am using a RPi-4 as the router via a USB3 Ethernet adapter with the stock Ethernet port connected to my LAN (8 port switch). The installation of OpenWRT and startup was anything but smooth but is now working. The first order of business is to get Starlink and my LTE modem connected to the input of the Pi at the same time. I am assuming that I will need a WAN side switch to connect every thing together physically and then use vlans to complete the data connections. The LTE modem seems pretty straight forward because all the examples I have seen use at least one. The Starlink not so much. The Starlink modem is connected to a port using DHCP and gives your device one of their CGNAT address to connect to, no fixed IP address here. Is it even possible to use this with a vlan? The next question is in regards to the WAN side switch. Most of the reviews of the lower cost managed switches rant about the lack of "proper isolation" of vlans by the devices firmware. Is this even an issue here where the switch is outside of the LAN? Is a true managed switch even required in this case? TIA

John

[bnhf2]

Presently my network is connected to Starlink using the Starlink Ethernet adapter (dongle). I am using a RPi-4 as the router via a USB3 Ethernet adapter with the stock Ethernet port connected to my LAN (8 port switch). The installation of OpenWRT and startup was anything but smooth but is now working. The first order of business is to get Starlink and my LTE modem connected to the input of the Pi at the same time. I am assuming that I will need a WAN side switch to connect every thing together physically and then use vlans to complete the data connections. The LTE modem seems pretty straight forward because all the examples I have seen use at least one. The Starlink not so much. The Starlink modem is connected to a port using DHCP and gives your device one of their CGNAT address to connect to, no fixed IP address here. Is it even possible to use this with a vlan? The next question is in regards to the WAN side switch. Most of the reviews of the lower cost managed switches rant about the lack of "proper isolation" of vlans by the devices firmware. Is this even an issue here where the switch is outside of the LAN? Is a true managed switch even required in this case?

Sorry for the delay in responding -- I don't seem to be getting notifications about this thread anymore. Probably something to do with the website name change.

You won't be able to do a vlan with Starlink. I recently got my Starlink up-and-running as well, and was a little surprised at the total lack of configuration capability -- I suspect they're trying to limit the variables until their software is further along. A vlan can only be used when both sides are configured for it.

Using vlans is mostly for those that have multiple outdoor cellular routers and want to run a single ethernet cable to support them all. With Starlink, you'll want to have an Ethernet port on your OpenMPTCProuter (an RPi in your case) that's dedicated to it, so you'll probably need another Ethernet dongle. That'll allow you to use the Starlink's DHCP without issue. Don't forget to change the subnet for OMR -- they both can't be 192.168.100.0.

[JimHelms]

Sorry for the delay in responding -- I don't seem to be getting notifications about this thread anymore. Probably something to do with the website name change.

You will need to update your NOTIFICATION settings.

[islander261]

Thank you.
Sorry I had a brain fart over the VLAN for the modems. I have a second USB3 dongle that I will install in a few days when life calms down again. I am going to have a go at MWAN3 fail over before setting up a VPS just to be sure the RPI 4 will handle two modems sources ok.

John

[islander261]

Well I got OpenMPTCProuter up and running with minimal drama. The main problem was I couldn't get the Rpi 4 OMR box to open secure coms with the VPS. At some point I managed to set a port correctly and everything started working.


So far no problems, web surfing, email, social media, Google dots and TV all working. Next chore is to add an Ooma box for Voip.
John

[islander261]

Well it was a good try with OpenMPTCProuter. We have lost all streaming services because they block VPN/VPS connections. The wife has demanded that the OMR experiment end. I have tried to use the OMR Bypass function using domain names, ASN numbers, source MAC address, source IP address all with no success (no bypass). I have tried everything I could find on the web with no success. I really wanted this to work but have grown accustom to living in my own house.

[bnhf2]

We have lost all streaming services because they block VPN/VPS connections. The wife has demanded that the OMR experiment end. I have tried to use the OMR Bypass function using domain names, ASN numbers, source MAC address, source IP address all with no success (no bypass).

It's true the streaming services block connections from commercial data centers -- mostly to prevent geo-relocation by people who are trying to watch content not available in their home market.

To deal with this, OMR Bypass is one way to go. I've tried most of the variations on this, as you did, and the only one that has worked consistently for me is the "Source LAN IP address or network". Configure any devices you want to go directly to a given interface by IP address (DHCP reservations recommended), and designate which WAN port you'd like that traffic directed. A simple Speedtest.net will show the change in the network you're using. I just tested it to confirm, and I went from using "Choopa" (Vultr's network) to AT&T immediately after doing a "Save & Apply" in OMR.

screenshot-192.168.100.1-2022.04.14-09_20_12.png

The ultimate way to deal with this, if you have really good broadband available at another residence (family, friend or second home), is to setup your own VPS. Just make sure both upload and download speeds exceed what your needs would be based on your potential maximum bonded speeds. Nothing super fancy required here -- I created an Ubunutu virtual machine for mine, and then forwarded the required ports from my router. This approach works really well, with no streaming service issues. Not an option for everyone obviously, so it's nice to have the OMR Bypass approach too!

[islander261]

Thank you for the tip. I used the IP of my workstation and put it into the Source lan IP address field to use the Starlink interface (eth2) and saved it. I then opened a new tab on my browser and started a speed test. Sure enough it was showing the Starlink server and not my VPS as the outlet to the internet. I started the speed test and the speed was about what my OMR aggregate speed usually was, not the Starlink alone speed. When the download portion finished the upload portion failed to start with a warning about not being able to reach the speed test server. Then I discovered that I had no internet connection through OMR at all. Both interfaces were still up but no connection to the VPS, both modems indicated they where still on line. On further investigation it appears the VPS is not functioning now, this also prevents the router from doing things like passing out IPs using DHCP on the lan.

I have reached the end of domestic tolerance for all the OMR funny stuff. I have switched back to plain old OpenWRT and will have another try at Mwan3 until I get some time alone at home to work on this. I am not sure about how to restart the VPS but will give it a try this weekend. I will also reflash the card with OMR on it for the Rpi 4 so I get a clean start.

[bnhf2]

I have reached the end of domestic tolerance for all the OMR funny stuff. I have switched back to plain old OpenWRT and will have another try at Mwan3 until I get some time alone at home to work on this. I am not sure about how to restart the VPS but will give it a try this weekend. I will also reflash the card with OMR on it for the Rpi 4 so I get a clean start.

I'd suggest that you don't use OMR as your primary router until you have things running the way you want them. It's easy enough to do a temporary setup, which will allow you to experiment and get familiar with the concepts, without triggering family disharmony. For example, you could have the household use the Starlink WiFi, while you use the Starlink Ethernet port and your cellular setup to create a separate bonded OMR network for testing.

If you're using a Raspberry Pi for OpenMPTCProuter, I would not rely the RPi's onboard WiFi to create an AP -- performance will likely be poor and will result in unreliable test results. Best bet is to use Ethernet for your lab station, and ultimately deploy a high-quality WiFi AP, like something from Ubiquiti for final testing and rollout.

[islander261]

Thanks for all the tips. I have disabled the wifi on the RPi 4 because of the known issues. I am using an old High Power 1G router/switch/access point for my wifi with all the routing disabled, covers the whole house easily. On a side note before I received the Ethernet dongle to use with my V3 (rectangular) dish I did use the Rpi 4 wifi right next to the Starlink router as a wifi bridge and it worked very well.

[islander261]

I have now managed to get OpenMPTCProuter working with Starlink and my LTE connection AND maintain domestic peace. At this time all of our current streaming and social media needs are being met.

To do this I had to use the latest snapshot builds of OpenMPTCProuter and the latest matching VPS script. My first attempts at this didn't work correctly. Trafic from both devises did go through the VPS but there was no channel bonding happening. After much pulling of hair and knashing of teeth I managed to get things working. What I finally did was to delete the openmptcptrouter_config.txt file in the root user directory and rerun the VPS install script. This caused a new set of keys to be generated. I then reran the setup wizard, System > OpenMPTCProuter and put in the new VPS key and checked Show advanced settings box then under Retrive settings from server checked the Force retrive settings box. After hitting Save and Apply the Network Overview tab came up showing everything working as expected, no pesky error messages for the router or VPS.

This latest software version has a working OMR-Bypass function. I have tested bypass by domain, by ASN, by source lan IP, MAC-address on both devices. I have no reason to think that the other filters will not work as well.

[jimmyle1511]

take a look at the project https://github.com/TalalMash/SmoothWAN
its build base on OpenWRT and using Speedify bonding more than 2 wans together,
i tested on ras pi 4 and r4s nano pi, speed is over 500Mbps.

[Chris29]

I have now managed to get OpenMPTCProuter working with Starlink and my LTE connection AND maintain domestic peace. At this time all of our current streaming and social media needs are being met.

Please can you share some of your settings etc?

Eg:

Is this Starlink V2 (rectangular) ?
With Ethernet adapter?
In Bypass mode ?
How did you connect Starlink to the Pi ?
What wizard settings did you use ?
Have you got the Starlink app to work, ie 192.168.100.1 (static routes)
What speed are you getting (eg max on a good day)?
Does it smooth over interruptions in Starlink?

[islander261]

I am using a V3 (rectangular) Starlink dish. I have always been behind the Starlink CGNAT wall, no public IP, IPV4 only.
I am using the the Starlink Ethernet Adapter.
I am running in bypass mode.
The Starlink is connected to the RPi using an USB3 to Ethernet dongle, TP-Link UE300, do not use the newer UE306 version.
Not sure what you mean here as everyone is a bit unique. WAN1 (Starlink) is eth1 set to DHCP, WAN2 (LTE modem) is eth2 set to a static address, eth0 (the native port) is set to a static address an is my LAN connection.
I only use the Starlink web connection. I have a static route set to 192.168.100.1.
The Starlink speeds here are low due to over subscription. I get anywhere from 10 to 110Mbps depending on the day and time. Most of the time Starlink averages 50 to 75Mbps and my LTE connection 15 to 50Mbps. Typical aggregate speeds of 75 to 120 Mbps. On Saturday and Sunday evening I may get aggregate speeds as low as 25Mbps.
The VPS keeps calls from dropping because the IP doesn't change when Starlink stutters. I have my LTE modem set as the master because it is always works. I use the OMR-Bypass function to get the streaming services and location (VPS, VPN) blocked websites to work.

openmptcprouter_status.png

[Chris29]

The Starlink is connected to the RPi using an USB3 to Ethernet dongle, TP-Link UE300, do not use the newer UE306 version.

Thanks for that, interesting.

Why not the UE306 out of interest ?

Here (UK) Starnet is (currently) over 200M and have seen 260M so I wanted to try and get some use of that (currently limited to 100M by my hardware)
I'm surprised to hear it's that low for you considering it's price ...
This may explain why some people are bonding two Starnets ...

I have 3M ADSL on WAN1
Starlink on WAN2
And flaky 0-30M LTE on WAN3

It's behaving oddly, the ADSL has similar latency to Starlink and OMR seems to flip between the two.
If I set ADSL to backup it all goes belly up ...
I will try your set up.

Thanks again.

[Chris29]

TP-Link UE300 is working and here are the setting I used:

Screenshot 2022-05-23 at 14.09.09.jpg

[islander261]

I am glad the UE306 is working for you. I found that the longest the interface would stay up was about 2 days when using one. It required a full reboot of the router (RPi4) to reconnect. The UE300s have been running for four weeks now without a problem.

We don't have good internet service outside urban areas in the U.S. If you are lucky you can get slow flaky ADSL or a LTE connection but for many nothing. So there are many more Starlink users here than on other continents. The slower speeds we see are a direct result of the number of users online at any time. For most rural users Starlink is still far and away the highest performing most reliable choice for internet access there is.

I found that using Starlink as the master link didn't work well. Every time Starlink did one of it's little hiccups all links would go down until the router was restarted. This is likely an OpenMPTCProuter problem but I don't have the knowledge to try and fix it. So I use a LTE connection from my well house (10m tower) 200m up the hill from my house (no cell service at my house due to terrain) so I can see the only tower 3.5km away as my master link. This gets overloaded on holidays and weekends when all the part timers show up.

[Chris29]

I am glad the UE306 is working for you.

I'm sorry I did take your advice and purchase TP-Link UE300 (older model).
I just typed it wrong (fixed).
Thanks for the additional info.
My ADSL is reliable but slow. My 4G is flaky as hell. So not quite the same as you. Currently I have Starlink as Master.
I am still monitoring. See also : https://github.com/Ysurac/openmptcprout ... sions/2357

[islander261]

My next challenge is to get IPV6 working with OpenMPTCProuter. Starlink is only IPV4 behind CGNAT but my LTE connection gives both IPV4 and IPV6 addresses out. This is getting really pressing because my LTE connection will go to IPV6 only next month if you want to avoid throttling. This is the whole Calyx/T-Mobile/Sprint change over causing this.

I have tried several times to get IPV6 working on OMR but don't have the networking knowledge to get it going. I know that my VPS is setup for IPV6 because it shows both an IPV4 and IPV6 address. I also know that my LTE modem (Rooter/ GO) will pass out IPV6 addresses when it is used as my network router so most of the pieces are here. I am going to post a detailed request for help on this topic on https://github.com/Ysurac/openmptcprouter in the next few days.

[packlet]

per instructions at: https://github.com/Ysurac/openmptcprout ... te-the-VPS

trying to install OpenMTCPRouter VPS on my (local, not cloud) x86 with totally stock Ubuntu 20.0.4, i get this:

wget -O - https://www.openmptcprouter.com/server/ ... -x86_64.sh | sh
...
Hit:5 http://us.archive.ubuntu.com/ubuntu focal-backports InRelease
Ign:6 https://repo.openmptcprouter.com buster InRelease
Err:7 https://repo.openmptcprouter.com buster Release
Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Error in the certificate verification. [IP: 68.70.205.3 443]
Get:8 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
...
Get:18 http://archive.ubuntu.com/ubuntu bionic/universe DEP-11 64x64@2 Icons [29 B]
Reading package lists... Done
E: The repository 'https://repo.openmptcprouter.com buster Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

not a big Unix weenie, so not sure how to work around. Any ideas? thanx, Paul.

edit: man page for apt-secure says:

"You can force all APT clients to raise only warnings by setting the configuration option Acquire::AllowInsecureRepositories to true. Individual
repositories can also be allowed to be insecure via the sources.list(5) option allow-insecure=yes. Note that insecure repositories are strongly
discouraged and all options to force apt to continue supporting them will eventually be removed. Users also have the Trusted option available to
disable even the warnings, but be sure to understand the implications as detailed in sources.list(5)."

so for my next dumb Unix question: How do i "allow-insecure=yes" ? Do I edit some config file somewhere? is the there a UI? typical Unix documentation that assumes you know everything already...

[islander261]

This is a question for over on the OpenMPTCProuter GIT issues or on a Ubuntu forum.

I am using a VPS (Vultr) running Debian 10 and the scripts have just worked for both the stable release and the snapshot I am now using. If the scripts had not worked I would have been SOL. Sorry Ubuntu isn't my thing even though it is a Debian derivative. You can try a VPS pretty cheap as mine costs only $5.80 a month with no multi month contract needed.

[packlet]

Got it working, with some input from the dev (YSurac). Needed to update all the Ubuntu packages BEFORE running the VPS installer script, was all. For some reason, i thought 'update during install' (of the OS) would take care of that, but apparently not... Anyways, Multipath Goodness is almost a reality. I have a bottleneck somewhere i need to track down, as I'm hitting a very hard limit @70mbps. ( Hoping it's just a 100Mb switch on the path somewhere - as mentioned, for now VPS is local bare metal)

[JimHelms]

@packlet

Got it working, with some input from the dev (YSurac). https://flic.kr/p/2noLzyr

Please read the rules regarding posting links to images....

[packlet]

sorry about that. removed external link and adding (more recent) images directly here.

Using Ubuntu on Linode for VPS, not (currently) blocked by Netflix...

[Rituxan]

Does anyone know the exact steps to bypass Netflix? Using Vultr + Debian 10.

Can’t finish my setup until Netflix is bypassed on firestick devices because my wife will kill me.

[mtl26637]

Quickest way is to just add the streaming device MAC address under OMR bypass. Leave "interface" set to "unspecified" and it will use the master interface and if that goes down it will switch to a working interface. If you set it to a specific interface it will only work if that interface is up. The other (harder) way is to add all of the different netflix domains under "domains" and add all of the netflix "ASN" under ASN settings. Both ways work but the first way will bypass all traffic to that specific device rather than just netflix traffic.