/ Load Balancing

A Developer's Introduction to Geotraffic

Many moons ago, when you connected to a website, things were relatively simple. You would resolve a domain name into an IP address, then your requests would travel across the internet to the physical host associated with that IP address.

The modern internet is much more complex. You have visitors arriving from every-which-place to applications that are hosted from varied and exotic locations. At a high-level, we call this geotraffic distribution. Within this article, we’ll look at the basics of geotraffic distribution that every developer should know: how modern web applications handle international visitor traffic.

In particular, we’ll...

  • Provide an overview of geotraffic distribution.
  • Peek into some of the distribution methods:
    • DNS “Load Balancing”
    • A Content Delivery Network
    • Anycast
    • An Application Delivery Network

After reading, you will have a clear picture of how a massive, distributed web application may be able to provide a fast and pleasant experience to users from around the world.

Geotraffic Distribution: Visitors and Hosts

There are two halves to the geotraffic equation. The first half: visitors access your application from anywhere in the world.

Let’s explore this, briefly.

If we live in Frankfurt, Germany and want to access an application in Chicago, here is what happens:

goodroot@flyiofrankfurt:~# traceroute 184.154.83.119
traceroute to 184.154.83.119 (184.154.83.119), 30 hops max, 60 byte packets
 1  207.154.208.253 (207.154.208.253)  1.488 ms 207.154.208.254 (207.154.208.254)  0.875 ms  0.865 ms
 2  138.197.250.156 (138.197.250.156)  0.886 ms 138.197.250.142 (138.197.250.142)  0.846 ms 138.197.250.156 (138.197.250.156)  0.772 ms
 3  ffm-b2-link.telia.net (213.248.75.41)  0.935 ms ffm-b2-link.telia.net (213.248.103.25)  1.165 ms ffm-b2-link.telia.net (213.248.75.41)  0.916 ms
 4  ffm-bb4-link.telia.net (62.115.125.214)  1.476 ms ffm-bb4-link.telia.net (62.115.118.201)  1.234 ms ffm-bb4-link.telia.net (62.115.125.216)  1.450 ms
 5  prs-bb2-link.telia.net (62.115.122.134)  10.439 ms prs-bb2-link.telia.net (62.115.122.124)  10.463 ms prs-bb2-link.telia.net (62.115.122.96)  10.418 ms
 6  nyk-bb4-link.telia.net (62.115.124.148)  82.084 ms nyk-bb4-link.telia.net (62.115.137.78)  82.182 ms  82.231 ms
 7  chi-b21-link.telia.net (62.115.137.59)  112.157 ms chi-b21-link.telia.net (62.115.125.23)  105.656 ms chi-b21-link.telia.net (213.155.136.19)  104.460 ms

We hopped through the ISP, then hit…

Germany 1 (ffm) - > Germany 2 (ffm) -> Paris (prs) -> New York (nwk) -> Chicago (chi)

As the traceroute unfurls, the further away from the server we hop, the greater the average latency becomes. As you likely expected, the further you are from the host the slower your browsing experience will be.

If our server were located in Frankfurt, nearer to where we live:

goodroot@flyiofrankfurt:~# traceroute 62.115.137.141
traceroute to 62.115.137.141 (62.115.137.141), 30 hops max, 60 byte packets
 1  207.154.208.253 (207.154.208.253)  2.719 ms  2.696 ms 207.154.208.254 (207.154.208.254)  1.938 ms
 2  138.197.250.150 (138.197.250.150)  1.975 ms 138.197.250.146 (138.197.250.146)  1.971 ms 138.197.250.156 (138.197.250.156)  1.893 ms
 3  ffm-b2-link.telia.net (213.248.75.41)  2.034 ms  2.022 ms ffm-b2-link.telia.net (213.248.103.25)  2.273 ms
 4  ffm-bb4-link.telia.net (62.115.125.216)  2.467 ms ffm-bb4-link.telia.net (62.115.125.218)  1.896 ms ffm-bb4-link.telia.net (62.115.125.216)  2.447 ms
 5  ffm-b10-link.telia.net (62.115.137.141)  1.912 ms  1.902 ms  3.958 ms

… We would see a significantly speedier roundtrip time: ~107.56ms vs ~2.57ms. Given the world wide nature of the web, we should expect — and be able to offer optimal service for — users from all corners of the world.

The second aspect of geotraffic distribution involves threading together application infrastructure that exists within various locales. If you are hosting your applications in several regions, like Western USA, Eastern USA, and Germany, how do you determine which user goes where? How do you coordinate and manage distributed system architecture?

Geotraffic distribution, as we will explore it, is the quest to solve the
performance and logistical problems of operating at scale across nations.

DNS for “Load Balancing”

We’ll take a look at a few different application delivery scenarios. The first method we’ll look involves leveraging the Domain Name System (DNS).

In order to provide a better experience for our global visitor base, imagine that we have two front-end servers running Rails, Ubuntu and Nginx. We have an IP address for each: 55.55.55.55 and 55.55.55.54. One is located in Germany and the other is located in the United States.

With these two IP addresses, we would head to our DNS host. After that, we’d create two A records:

oursite.com		A	   55.55.55.55
oursite.com		A	   55.55.55.54

As each visitor accesses oursite.com, they alternate in Round Robin format between our two IP addresses. If one is down, most modern browsers will deploy browser retry or client retry to pick the next available record after failing for a short period of time. If no active address exists, then your site is deemed inaccessible.

It is common to see this in the wild. If we dig google.com for example, we’d see several A records within their answer section:

;; ANSWER SECTION:
google.com.             224     IN      A       74.125.28.139
google.com.             224     IN      A       74.125.28.100
google.com.             224     IN      A       74.125.28.102
google.com.             224     IN      A       74.125.28.101
google.com.             224     IN      A       74.125.28.113
google.com.             224     IN      A       74.125.28.138

But, hang on — how does this help with geographic traffic distribution? While some DNS providers may have value-added GeoDNS functionality, the answer is that on its own it does not. Using multiple A records simply alternates traffic from one host to the other without paying any mind to where the visitor is located relative to the host.

Not only that, but these records are cached within the browser and have a life-span on each nameserver tied to its ‘Time to Live (TTL)’ . If one of your hosts goes down or you change host IP addresses, anyone resolving onehostname.com could be resolving a ‘broken’ record for a significant amount of time.

While this may have helped our two distributed servers join forces, our average user does not see any benefit. Next, we’ll look at how a Content Delivery Network might prove to be more beneficial.

Content Delivery Networks

When you signup for a Content Delivery Network (CDN), part of the configuration process involves changing your nameservers or DNS records to those provided by your CDN.

Let’s consider an example. If we have a load balancer available to the public at 97.97.97.97 and we connect it to the nameservers of our CDN, it becomes available at a new address: 29.29.29.29 . The CDN will then leverage a more robust implementation of the Anycast routing method to associate that single IP address with their full array of international ‘edge-servers’ or ‘Points of Presences(PoPs)’. A CDN may have hundreds of these servers spread around the world.

Typically, these edge servers / PoPs will act as a cache for static assets. A static asset could be your JavaScript, CSS, and HTML files. No matter where your application is located, the cached assets are consumed by your visitors at the edge nearest to them. The end result of a shortened trip to these cached assets is that less connection overhead is spent finishing the trip to your application; your users receive your application faster.

Somewhat troubling, most CDNs will also perform SSL termination at the edges, routing your data unencrypted over the back-half to your application. We've written more about HTTPS if you'd like to explore it in greater depth.

In summary, a CDN:

  • Uses modified Anycast to shorten the network route taken to perform SSL termination and resolve your application.
  • Caches static assets to improve rendering performance by reducing visitor distance to the server where the static assets are hosted.
  • Traffic is secured over HTTPS from client-to-edge-server. Traffic is exposed after SSL termination in most cases.

CDNs look like a step in the right direction, but what if you want to roll your own geotraffic solution? Let’s expand quickly on one of the underlying technologies found in most CDNs, the aforementioned Anycast.

About Anycast

With Anycast, one IP address represents many different servers. Users connect to this single IP address and Anycast will choose the route that has the shortest amount of hops. At Fly.io, we use Anycast to connect you to our network of multi-region edge servers.

For example, when reaching app.fly.io from Vancouver…

goodroot@flyiovancouver: ~ $ traceroute app.fly.io
traceroute to app.fly.io (206.51.242.1), 64 hops max, 52 byte packets
 1  10.33.0.1 (10.33.0.1)  154.710 ms  171.486 ms  167.112 ms
 2  162.221.207.252 (162.221.207.252)  171.680 ms  172.399 ms  172.047 ms
 3  xe-1-2-0-510.ie1.van-hc.unet.ca (97.107.179.16)  172.026 ms  170.684 ms  223.152 ms
 4  xe-3-2-0.cr0-van1.ip4.gtt.net (173.205.56.221)  287.871 ms  189.346 ms  190.702 ms
 5  xe-1-1-6.cr0-sjc1.ip4.gtt.net (89.149.130.226)  305.648 ms  185.229 ms  218.732 ms
 6  as23352.xe-5-2-0-101.cr1.sjc1.us.as4436.gtt.net (69.22.153.59)  190.293 ms  207.396 ms  190.646 ms
 7  ge0-2.aggr20.sjc1.us.scnet.net (64.202.104.58)  191.586 ms  182.985 ms  187.932 ms

... Compared with Frankfurt…

root@flyiofrankfurt: ~# traceroute app.fly.io
traceroute to app.fly.io (206.51.242.1), 30 hops max, 60 byte packets
 1  207.154.208.253 (207.154.208.253)  1.249 ms 207.154. 208.254 (207.154.208.254)  1.156 ms  1.143 ms
 2  138.197.250.156 (138.197.250.156)  0.944 ms 138.197.250.144 (138.197.250.144)  1.416 ms 138.197.250.156 (138 .197.250.156)  0.826 ms
 3  83.231.214.45 (83.231.214.45)  1.118 ms  1.259 ms  1.114 ms
 4  ae-6.r20.frnkge04.de.bb.gin.ntt.net (129.250.6.248) 7.742 ms ae-2.r20.frnkge04.de.bb.gin.ntt.net (129.250.5.217)  7.618 ms ae-6.r20.frnkge04.de.bb.gin.ntt.net (129.250.6.248)  7.728 ms
 5  ae-9.r25.amstnl02.nl.bb.gin.ntt.net (129.250.3.77)
9.906 ms  6.570 ms  7.669 ms
 6  ae-2.r02.amstnl02.nl.bb.gin.ntt.net (129.250.2.159) 9.045 ms  10.069 ms  8.642 ms
 7  83.231.213.78 (83.231.213.78)  7.730 ms  9.494 ms  6 .830 ms
 8  41.vlan.aggr203.ams3.nl.scnet.net (205.234.220.242) 19.305 ms  13.838 ms  12.324 ms

... We see that both connections to app.fly.io resolved the same IP address (206.51.242.1) but went to different edge servers. Our Vancouver connection wound up in San Jose (ge0-2.aggr20.sjc1.us.scnet.net), while our Frankfurt connection wound up in Amsterdam (41.vlan.aggr203.ams3.nl.scnet.net).

Anycast, which is part of the Border Gateway Protocol allows one-to-many IP-address-to-host communication. Anycast is powerful but a tremendous challenge to both configure and maintain. It includes such hurdles as acquiring a reserved block of IP addresses and dealing with the dangers of maintaining stateful TCP connections over routing paths that are subject to change. One major Anycast caveat is that it will choose the path with the least hops. The hops may be lesser, but they could be slower hops; 5 100ms hops is a longer trip than 7 50ms hops.

Unless you are creating your own global network of servers to host your application, Anycast is likely not a good fit. The last option is the one seen within Fly, the new-fangled Application Delivery Network.

The ADN

An ADN is CDN for developers, one they can control. While a CDN will shorten the resolution path to your application and improve performance through closer static asset caching and more performant TLS handshakes, an ADN will do that and allow you to intelligently route based on HTTP header information.

An example: After verifying your hostname, you can connect any Docker, Heroku, or GitHub Pages backend to your hostname. From there, activating the Geo IP Lookup Middleware will inscribe each of the following request headers with information pulled from the visitor IP:

  • Fly-IP-Country-Code
  • Fly-IP-Country-Name
  • Fly-IP-City-Name

Now, the magic kicks in when you can use this header information to route your requests to specific backends.

For example, if you receive headers from Germany with Fly-IP-Country-Code: DE you can route them to your application instance hosted in Frankfurt. If you receive US headers with Fly-IP-Country-Code: US you can route them to your San Jose instance - or, route them more specifically using Fly-IP-City-Name.

Performance increase through more efficient routing is but one avenue of benefit. Imagine that your application hosted in Germany is also German localized and your San Jose site is tweaked to appeal to groovy West Coast types; there is immense potential in intentioned geotraffic distribution.

The "exposed after the edges" problem seen in most CDNs can be mitigated by the open-source utility Wormhole. Wormhole creates a secure tunnel between two end-points. You can, for example, connect to a private Kubernetes cluster without needing to configure how you deal with ingress traffic.

To summarize, an ADN:

  • Uses modified Anycast to shorten the network route taken to perform SSL termination and resolve your application.
  • Caches static assets to improve rendering performance by reducing visitor distance to the server where the static assets are hosted.
  • Traffic is secured over HTTPS from client-to-edge-server. Traffic is encrypted after termination via Wormhole.
  • Allows developers to write code against the edge, enabling features like geo IP header routing.

Summary

Geotraffic distribution can be a whole kettle of fish. As your application grows in appeal and attracts an international audience, it is in your best interest to optimize the user experience for each region from which you receive visitors.

DNS, unless heavily adjusted by a third-party, will not help you much. Anycast is way out of scope for your average development team. A CDN is a step in the right direction, but they don’t do much to stimulate the imagination of your friendly neighbourhood developer and expose your connections through the 'back-half' of their network route.

Fly, the Application Delivery Network, is an intriguing step into the world of secure, fast and intelligent geotraffic distribution. It's free to sign-up as are your first 2,000,000 requests and 100GB of transfer every month.

Kellen Evan Person

@thegoodroot

Kellen Evan Person

A polite, forest-dwelling Canadian who enjoys coding and writing. He's spent near two decades building web applications and strives to keep development fun and light-hearted.

North Vancouver, Canada