Dynamic Request Routing with fly-replay
The fly-replay
feature gives you fine-grained control over request routing in your Fly.io applications. It allows you to dynamically route requests between regions, specific Machines, or even different apps within your organization.
How fly-replay Works
When your app responds to a request with a fly-replay
response, Fly Proxy will automatically replay the original request according to your specified routing rules. This enables advanced patterns like:
- Routing write operations to primary database regions
- Load balancing between specific Machines
- Cross-app request routing within your organization
- Implementing sticky sessions
Replay Header Format
Your app can add a fly-replay
header to its response. The fly-replay
header accepts the following fields:
Field | Description |
---|---|
region |
The region(s) to route the request to. Accepts comma-separated list of region codes |
instance |
The ID of a specific Machine to route to |
prefer_instance |
The ID of a specific Machine to route to, if possible |
app |
The name of another app (in same organization) to route to |
state |
Optional string included in fly-replay-src header on replay |
elsewhere |
If true , excludes responding Machine from next load-balance |
Example Usage
Route to a specific region:
fly-replay: region=sjc
Route to one of the given regions, in order of preference:
fly-replay: region="iad,ord,us,na"
Route to a preferred region, or fallback to any available machine:
fly-replay: region="sjc,any"
Route to specific Machine:
fly-replay: instance=00bb33ff
Route to another app:
fly-replay: app=app-in-same-org
You can combine multiple fields:
fly-replay: region="sjc,any";app=app-in-same-org
Note: A comma-separated list of regions must be quoted.
Geographic groups and aliases
When replaying to a region, you can use geographic aliases like us
, eu
, or sa
to target a broader area.
Alias | Area |
---|---|
apac |
Asia-Pacific |
eu |
Europe |
na |
North America |
sa |
South America |
us , usa |
United States |
any |
Earth |
Replay JSON Format
Your app can set the response content-type to application/vnd.fly.replay+json
and include replay instructions in the response body.
For a complex replay the JSON body is easier to compose than the header format, and also supports more functionality.
JSON Structure
The application/vnd.fly.replay+json
replay body accepts the following fields:
Field | Description |
---|---|
region |
The region(s) to route the request to. Accepts comma-separated list of region codes |
instance |
The ID of a specific Machine to route to |
prefer_instance |
The ID of a specific Machine to route to, if possible |
app |
The name of another app (in same organization) to route to |
state |
Optional string included in fly-replay-src header on replay |
elsewhere |
If true , excludes responding Machine from next load-balance |
transform.path |
Rewrite the path and query parameters of the request |
transform.delete_headers |
Delete headers from the request, hiding them from the replay target |
transform.set_headers |
Set new headers on the request, overwriting headers of the same name |
cache.prefix |
Cache the replay for matching requests (see Replay Caching) |
cache.ttl |
Cache the replay for this many seconds (see Replay Caching) |
cache.invalidate |
Invalidate the cache for the current route (see Replay Caching) |
Example Usage
Route to another app, and modify the request:
{
"app": "target-app",
"region": "iad,us",
"transform": {
"path": "/new/path?param=value",
"delete_headers": ["x-unwanted-header", "cookie"],
"set_headers": [
{ "name": "x-custom-header", "value": "new-value" },
{ "name": "authorization", "value": "Bearer token123" }
]
}
}
Replay Caching
In cases where the replay target does not change often, you can use the fly-replay-cache
mechanism to relieve the original app of some load. Here’s how it works:
- Issue a
fly-replay
header as usual - Set
fly-replay-cache: /some/path/*
- This needs to be a path-matching pattern ending with a wildcard
/*
. This is where the cached replay is applied. - The cached replay is unique to the request domain. The domain may also be set explicitly:
example.com/some/path/*
- The domain part should not include ports; use
example.com
, notexample.com:80
. - The pattern must also match the current request.
- This needs to be a path-matching pattern ending with a wildcard
- Set
fly-replay-cache-ttl-secs: number_of_seconds
If the replay target does eventually change, the replay target may proactively invalidate the cache by
- Issuing a
fly-replay
back to the origin - Setting
fly-replay-cache: invalidate
For apps using the JSON format, specify a cache
object:
{
"app": "target-app",
"cache": {
"prefix": "/some/path/*",
"ttl": 60
}
}
{
"app": "origin-app",
"cache": {
"invalidate": true
}
}
Note: Replay caching is an optimization, not a guarantee. Your app should not depend on this mechanism to function.
The app issuing fly-replay
still serves as the ultimate source of truth, and we may decide to consult that app at any moment even if a replay cache has previously been set.
To ensure reliable operation, the app issuing fly-replay
should still have multiple instances deployed in multiple regions.
Implementation Details
Requirements and Limitations
- Your app must use the http handler
- Requests larger than 1MB cannot be replayed
- Field combinations must be logically valid (e.g., don’t specify both app and instance if instance isn’t in that app)
For large uploads that exceed the 1MB limit, consider:
- Using direct-to-storage uploads where possible
- Using the fly-prefer-region header instead
For fly-replay-cache
, the following limitations apply:
- The
state
field cannot be set in thefly-replay
intended to be cached - Transformations for the headers or path cannot be defined.
- The TTL needs to be a minimum of 10 seconds
- Only one step of lookup is performed in the cache; as such, if the target app issues another
fly-replay-cache
, the caching behavior in this case is undefined - The
fly-replay-src
header (described below) will not be set for requests replayed through the cache
The fly-replay-src Header
When a request is replayed, Fly Proxy adds a fly-replay-src
header containing metadata about the original request:
Field | Description |
---|---|
instance |
ID of Machine that sent fly-replay |
region |
Region request was replayed from |
t |
Timestamp (microseconds since Unix epoch) |
state |
Contents of original state field, if any |
This header is useful for tracking request paths and implementing consistency patterns. See the official Ruby client for an example of using these fields to prevent read-your-write inconsistency.
This header is not set when the request is replayed through a cached fly-replay
entry (fly-replay-cache
).
The fly-preferred-instance-unavailable Header
If you replay with prefer_instance
set, Fly Proxy will attempt to route to this Machine. This may not happen for a number of reasons, for example the Machine may not be found, or found but at its configured hard_limit.
In these cases, the request will be delivered to a different Machine that matches the remaining fields in your replay. Along with the other Fly.io-specific headers, a fly-preferred-instance-unavailable
header will be set containing the ID of the instance that could not be reached.
Alternative Routing Headers
For cases where fly-replay
isn’t suitable, Fly.io provides two alternative request headers:
The fly-prefer-region Header
fly-prefer-region: ams
Attempts to route directly to specific region(s). You can specify multiple regions as comma-separated values for fallback preferences:
fly-prefer-region: iad,ord,us
Falls back to the next region in the list, or to the nearest region with healthy Machines if none of the specified regions are available. Useful for large uploads that can’t be replayed.
The fly-prefer-instance-id Header
fly-prefer-instance-id: 90801679a10038
Attempts to route to a specific Machine. Falls back to any matching Machine if the target Machine cannot be found, or is unable to accept requests.
If the request is delivered to a different Machine, the fly-preferred-instance-unavailable request header will be set.
The fly-force-instance-id Header
fly-force-instance-id: 90801679a10038
Forces routing to a specific Machine. The Fly Proxy will attempt to reach the Machine multiple times if the Machine is unhealthy. No fallback if Machine is ultimately unavailable.
Note: Get Machine IDs using fly status
or fly Machines list
.
Common Use Cases
Multi-Region Databases
When using global read replicas, use fly-replay
to ensure write operations go to the primary region:
fly-replay: region=sjc
See our blueprint for Multi-region databases and fly-replay for a complete implementation guide.
Cross-App Routing
Use fly-replay
to implement routing layers or FaaS-style architectures:
fly-replay: app=customer-function-app
This allows you to build router apps that can dynamically route requests to other apps in your organization.