My Apartment internet is usually pretty fast. I get around 150 Mbps down and 90 Mbps up. But lately, it’s gotten quite slow in the evenings: down to around 1 Mbps.

“Slow internet” is not a useful term. I first noticed that videos wouldn’t load. I went to speedtest.net and… the speeds seemed fine? ~150 Mbps down. Totally normal, and yet I couldn’t stream my favorite anime (crucial for any programmer’s productivity).

Testing

I confirmed a few things. Everything was slow: all websites, Wi-Fi and Ethernet, and all my devices. Cellular data worked fine. The issue was definitely the apartment’s network.

Unfortunately, I can’t actually do anything about the network itself. It’s run by the apartment complex I live in. It’s basically one giant mesh network so I can’t even reboot the router.

There are two tools I’d recommend anyone who deals with networks learn: MTR and iPerf 3. MTR is a tool that combines ping and traceroute. It traces all the nodes between you and a destination and pings all of them. MTR is great for finding where a connection is having issues. iPerf 3 is basically like running your own custom speed test, you run it on a server and send packets between them and measure various network performance metrics.

What I found was quite fascinating. First, using MTR to ping jimmyhoke.net showed that I was getting a large amount of packet loss, around 13%! This is of course, very bad. iPerf 3 tests showed that downloads, but not uploads, were very slow. However, while single connections were slow, using multiple connections worked pretty fast (what speedtest.net does by default).

The Problem

The issue, as it turns out, was packet loss. There was plenty of bandwidth on the network, but due to packet loss the entire complex was stuck at slow speeds. Normally, if there’s packet loss that means data is being sent too fast for the network to handle so servers slow down their transmission speed until there is very little packet loss (at least, for TCP. some protocols do other stuff). This normally works well and ensures data is sent at a reasonable rate and that no single connection hogs the network.

This is also why using multiple connections for a speed test seemed fast: a lost packet only affects one. But while that’s a good way to measure bandwidth for a connection, most of what people actually use the internet for uses a single connections (such as streaming a movie). And those single connections were getting slowed to 1 Mbps. Even though there was technically enough bandwidth for it to work just fine, servers were sending the data slowly because of the packet loss.

The Solution

Since packet loss makes TCP connections slow, there solution was to change the way those connections were made. Most of my important internet traffic is over HTTP.

This is good, as I can control how to the HTTP data gets sent to me via an HTTP proxy server. I set up Tinyproxy on a VPS and connected by phone to it. Now, all the TCP connections (for HTTP anyway) were going through my server. This meant that my server was managing the TCP connection that sends data to me. Ordinary, this would do essentially nothing.

However, now that I was sending the TCP packets for the last chunk of the connection, I can change the congestion control algorithm. While Cubit, which slows down with packet loss, is the default, you can change it to BBR. BBR estimates the bandwidth of a network and sends at that speed. Since it doesn’t use packet loss to determine bandwidth it will just keep re-transmitting the data at full speed. So all I had to do is run:

sudo modprobe tcp_bbr
sudo sysctl -w net.ipv4.tcp_congestion_control=bbr

on the server. This restored my download speeds back to the normal ~150 Mbps.

A few notes on BBR. Many people don’t like it because it can drown out and slow down other people on the network. This is

• 1. Not really a concern with a large network like this.
• 2. Not applicable, as nobody had any bandwidth for me to take anyway. The network was essentially unusable.

That being said, I wouldn’t use BBR on your systems unless you have a good reason to. It also is very unlikely to fix your slow internet.

A Few Notes On Proxies

First: why a proxy and not a VPN? Well a VPN wouldn’t do anything. I have my own Wireguard VPN, but that just encapsulates packets and forwards them.

[Website] --> [My Server] --> [Me]

But with a proxy, the TCP connection is terminated at my server:

[Website] --> [My Server] (Sent normally)
[My Server] --> [Me] (sent using BBR)

Another issue is UDP. This only fixes TCP connections. Fortunately most important stuff I was doing uses TCP. While HTTP does sometimes use UDP with HTTP3/QUIC, it can always be proxied over TCP. Another benefit of the HTTP proxy is that DNS lookups (which mostly use UDP) happen on the proxy server so local network conditions won’t affect them. For non-HTTP traffic, such as a lot of video games, some other type of proxy would be desired but I was just trying to watch anime and do homework (the most important things in life).

Another roadblock is proxy support. It is bad. Very bad. Especially on phones. Firstly, iOS doesn’t support using SSL to connect to a proxy, which is generally desirable. Also, if your proxy requires a password to log in iOS can use it, but it breaks a lot of apps. The only built-in way that works well is to use a plain HTTP proxy with no authentication. Of course, setting up a completely open proxy is an incredibly bad idea, so I had to set the proxy to only allow connections from my IP address (not a terribly secure solution, but good enough for a temporary fix). For iOS, another good option is to use a SOCKS proxy over SSH, which is much more secure. You can use the (paid) Shadowrocket app for this, but I wanted something both me and my friends could easily use.

Browser support is pretty good. For desktops/laptops, you can add any kind of proxy you want in your browser settings. I actually setup a Proxy Auto Configuration (PAC) script that the browser downloads to get the proxy configuration. This is useful, as the script only enables the proxy if your IP is on the allowlist (this is done by having NGINX serve a different PAC file depending on the IP address).

Conclusion

When the winds of packet loss disrupt your connection, the solution is to scream louder (re-transmit faster) into the wind (lossy connection). For a guide on how to do this yourself, see this article on the Spectavi technical blog: https://spectavi.me/how-to-make-a-bbr-http-proxy/