AnyCast CDN with only VPS
In order to keep this project under my very low budget of $100, I decided to use only VPS. There are a few VPS providers that offer BGP sessions, but I'm primarily using Vultr and Xenyth for AnyCast.
Using a VPS provider for AnyCast limits our ability to pick IP transit. But IP transit is expensive. And when you want a presence all over the world, it's insanely expensive. These providers have multiple tier 1 transit providers in each datacenter. So, this works out to our advantage.
That said, it would be nice if we could have more control over how our packets are routed. The public internet can be a noisy place. Your packets can take wild routes, sometimes even stupid routes. Those stupid routes can really slow you down. But if we look back in our toolbox, we see BIRD.
If we put our edge routers on a WireGuard mesh, then have them talk to each other using Babel, we can actually route around inefficient (dumb) routes. We can even utilize VPS providers, like DigitalOcean and Linode, that don't support BGP as leaf nodes. I put a Droplet in NYC and another in London, and my edge nodes in Europe will use that transatlantic route over other slower routes. Adding these leaf nodes increases our performance and resiliency.
Utilizing each VPS provider firewall is a great way to limit what ports are reachable from the dirty outside internet. But what about all of those bad actors out there? They can still touch my ports that I do have exposed! Well, we're using firehol to block known bad actors and Coraza WAF to block any malicious attempts. And we have fail2ban analyzing the WAF logs to look for repeat offenders; if they are up to no good, then we block them.
I'm still building, figuring things out, breaking things, over-engineering, and rolling back. For now, there are two static sites hosted by a5t.
A CDN for static sites is relatively simple. I'm working on supporting API requests next.