Volumes

We're rolling out V2 of the Fly Apps platform, running on Fly Machines. New customers' organizations now default to deploying V2 apps, and docs put Apps V2 in the foreground. Much remains the same! But where there's a difference between Apps V2 and the original Nomad-orchestrated Fly Apps, we'll supply Nomad-specific information either inline or at the end of the doc.

You can check if your org defaults to Apps V2 deployments with fly orgs apps-v2 show <org-slug>. Get your organization slug using fly orgs list.

If you're not a new customer but want to ride the wave of the future, start deploying your new apps to Apps V2 with fly orgs apps-v2 default-on <org-slug>.

Volumes are local persistent storage for Fly Machines. They allow an app to save its state, preserving configuration, session or user data, and be restarted with that information in place.

A Fly Volume is a slice of an NVMe drive on the physical server your Fly App runs on. It is tied to that hardware.

The first rule of Fly Volumes is: Always run at least two of them per application. If you don't, at some point you'll have downtime.

Fly Volumes are a lot like the disk inside your laptop, with the speed and simplicity advantage of being attached to your motherboard and accessible from a mount point in your file system. And the disadvantages that come with being tied to that hardware, too.

Things to consider before settling on volume storage:

You'll need to run as many volumes as there are Machines.
There's a one-to-one mapping between VMs and volumes. You can't share a volume between apps, nor can two VMs mount the same volume at the same time. A single VM can only mount one volume at a time.
Volumes are independent of one another; Fly.io does not automatically replicate data among the volumes on an app, so if you need the volumes to sync up, your app has to make that happen.
If your app needs a volume to function, and the NVMe drive hosting your volume fails, that instance of your app goes down. There's no way around that.
If you only have a single copy of your data on a single Fly Volume, and that drive fails, data is lost. Fly.io takes daily snapshots, retained for 5 days, but these are meant as a backstop, not your primary backup method.

Explore further options for data storage in Databases & Storage.

Fly Volumes and Flyctl

Volumes are managed using the fly volumes command.

fly volumes is aliased to fly volume and fly vol for convenience.

Create a Volume

Create a volume for an app using fly volumes create. The default volume size is 3GB. See fly volumes create in the flyctl reference for usage and options.

The following command creates a new volume named "myapp_data" with 40GB of storage in the lhr (London Heathrow) region, for the application whose fly.toml file is in the working directory. To specify a different app, use the -a or --app flag.

fly volumes create myapp_data --region yyz --size 1

        ID: vol_kgj54500d3qry2wz
      Name: myapp_data
       App: myapp
    Region: yyz
      Zone: acc6
   Size GB: 1
 Encrypted: true
Created at: 04 Mar 23 03:32 UTC

Volumes are, by default, created with encryption-at-rest enabled for additional protection of the data on the volume. Use --no-encryption to instead create an unencrypted volume for improved performance at deployment and runtime.

Volumes are bound to both apps and regions. A volume is directly associated with only one app and exists in only one region. No other app can see this volume and only an instance of the app running in the LHR region can access it.

Most people use volumes for databases, so for high availability, we default to putting each of your app's volumes on different hardware (equivalent to using --require-unique-zone=true with fly volumes create). This setting does limit the number of volumes your app can have in a region.

When you create a volume on a legacy (Nomad) Fly App, its region is added to the app's region pool to allow app instances to be started with it.

Mount a Volume

In the fly.toml for the app, there should be a section that mounts a volume into the app, like so:

[mounts]
source="myapp_data"
destination="/data"

When a Fly App is deployed with this configuration file, the data from a volume named myapp_data appears under the /data directory of the application. With this present, if an app instance is started and cannot find an unused volume named myapp_data, it will not be started.

Also, if you have specified a mounts section in fly.toml and forgotten to create a volume, your deployment will fail.

There can be multiple volumes of the same volume name in a region. Each volume has a unique ID to distinguish itself from others to allow for this. This allows multiple instances of an app to run in one region. Creating three volumes named myapp_data would let up to three instances of the app start up and run.

Volumes are independent of one another; Fly.io does not automatically replicate data among the volumes on an app.

List an App's Volumes

You can get a list of all volumes created for an app using the sub-command list.

fly volumes list

ID                      STATE   NAME          SIZE    REGION  ZONE    ENCRYPTED       ATTACHED VM     CREATED AT    
vol_xme149kke8ovowpl    created myapp_data    1GB     iad     7806    true                            2 minutes ago
vol_od56vjpp95mvny30    created myapp_data    1GB     lhr     79f0    true                            2 minutes ago
vol_kgj54500d3qry2wz    created myapp_data    1GB     yyz     acc6    true                            9 minutes ago

The unique ID can be used in commands that reference a specific volume, such as the show or delete sub-command. For example, the show command can display the details for a particular volume:

fly vol show vol_kgj54500d3qry2wz

        ID: vol_kgj54500d3qry2wz
      Name: myapp_data
       App: myapp
    Region: yyz
      Zone: acc6
   Size GB: 1
 Encrypted: true
Created at: 04 Mar 23 03:32 UTC

Extend a Volume

Volumes can be extended, but cannot be made smaller. To make a volume larger, find its ID with fly volumes list, then use:

fly volumes extend <volume-id> -s <new-size>

where <new-size> is the desired size in GB.

The Machine attached to the target volume must be restarted to allow the file system to be resized. On legacy Nomad-orchestrated Apps, the instance is automatically restarted, so this step isn't needed.

Get the id of the Machine that has the volume mounted (find it under ATTACHED VM).

fly volumes list

ID                      STATE   NAME    SIZE    REGION  ZONE    ENCRYPTED       ATTACHED VM     CREATED AT     
vol_od56vjp5pzmvny30    created data    2GB     yyz     acc6    true            4d891de2f66587  36 minutes ago

Restart the Machine:

fly machine restart <machine-id>

You can check that the new volume size is reflected in the Machine's file system by running df via fly ssh console. If there's more than one Machine on the app, fly ssh console -s allows you to select the correct one:

fly ssh console -s -C df

? Select VM:  [Use arrows to move, type to filter]
> yyz: 4d891de2f66587 fdaa:0:3b99:a7b:ef:8cc4:dc49:2 withered-shadow-4027           
Connecting to fdaa:0:3b99:a7b:ef:8cc4:dc49:2... complete
Filesystem     1K-blocks   Used Available Use% Mounted on
devtmpfs          103068      0    103068   0% /dev
/dev/vda         8191416 172752   7582852   3% /
shm               113224      0    113224   0% /dev/shm
tmpfs             113224      0    113224   0% /sys/fs/cgroup
/dev/vdb         2043856   3072   1930400   1% /storage

Here, the volume is mounted under /storage in the Machine's root file system and has been resized to 2GB.

Snapshots and Restores

We take daily block-level snapshots of volumes. Snapshots are kept for five days. These may not have your latest data. You should implement your own backup plan for important data.

Find the snapshots belonging to your target volume with fly volumes snapshots list <volume-id>:

fly volumes snapshots list vol_wod56vjyd6pvny30

Snapshots
ID                  SIZE        CREATED AT
vs_MgLAggLZkYx89fLy 17638389    1 hour ago
vs_1KRgwpDqZ2ll5tx  17649006    1 day ago
vs_nymJyYMwXpjxqTzJ 17677766    2 days ago
vs_R3OPAz5jBqzogF16 17689473    3 days ago
vs_pZlGZvq3gkAlAcaZ 17655830    4 days ago
vs_A9k6age3bQov6twj 17631880    5 days ago

Restoring from the snapshot to a new volume is a matter of:

fly volumes create <volume-name> --snapshot-id <snapshot-id> -s <volume-size> [-a <app-name>]

A volume snapshot can be restored into a volume that's the same size as, or larger than, the source volume, but not a smaller one. If you don't specify a size with the -s flag, fly volumes create will request a 3GB volume.

fly volumes create pg_data --snapshot-id vs_0Gvz2kBKJ28Mph4y -a cat-pg

? Select region: Chennai (Madras), India (maa)
        ID: vol_mjn924o9l3q403lq
      Name: pg_data
       App: cat-pg
    Region: maa
      Zone: 180d
   Size GB: 3
 Encrypted: true
Created at: 02 Aug 22 21:27 UTC

The flyctl output shows the details of the new volume, including its size.

Delete a Volume

The delete sub-command allows you to delete a specific volume.

fly volumes delete vol_2n0l9vlnklpr635d -a myapp

Deleting a volume is not reversible.
? Are you sure you want to delete this volume? Yes
Deleted volume vol_2n0l9vlnklpr635d from myapp

Edit this page on GitHub