Add Volume Storage

Volumes are local persistent storage for Fly Machines. Volumes allow an app to save its state—to preserve configuration, session or user data—and then be restarted with that information in place. You can access and write to a volume on a Machine just like a regular directory.

The first rule of Fly Volumes is: Always run at least two volumes per application. The way to do this is to run at least two Machines per app. Also note that volumes don't sync up by themselves; your app needs to take care of that. Refer to Fly Volumes for details about how volumes work.

Launch a New App With a Fly Volume

Launch a new app with one Machine and an attached volume, and then clone the Machine to scale out.

Launch the App, but Don't Deploy It Yet

The app has to exist for the volume to be created. The volume has to exist before you can mount it to a Machine. The shortest path to a Machine with a mounted volume begins by launching the app, and saying N to "deploy now?":

fly launch 

...
Wrote config file fly.toml
? Would you like to deploy now? No
...
Your app is ready! Deploy with `flyctl deploy`

Configure the App to Mount the New Volume

Add a [mounts] section in the app's fly.toml. The fly deploy command you'll run in the next step will create the Machine and the volume during the deploy process. The following configuration exposes data from a volume named myapp_data under the /data directory of the application.

[mounts]
source="myapp_data"
destination="/data"

Deploy the App

fly deploy 

On the first deployment, you'll get one Machine. You can confirm this with fly status.

fly status

App
  Name     = myapp
  Owner    = personal
  Hostname = myapp.fly.dev
  Image    = image:latest
  Platform = machines

Machines
PROCESS ID              VERSION REGION  STATE   CHECKS  LAST UPDATED
app     5683606c41098e  1       lhr     started 1 total 2023-04-26T20:40:48Z

Confirm the Volume Is Mounted

You can check on all the volumes in your app using fly volumes list. The ATTACHED VM column lets you know which Machine, if any, the volume is mounted on.

fly volumes list

ID                      STATE   NAME    SIZE    REGION  ZONE    ENCRYPTED       ATTACHED VM     CREATED AT    
vol_zmjnv8m81p5rywgx    created data    1GB     lhr     b6a7    true            5683606c41098e  3 minutes ago

You can also see the volume in the Machine's file system (there's only one Machine so far):

fly ssh console -s -C df

? Select VM: lhr: 5683606c41098e fdaa:0:3b99:a7b:7e:3155:9844:2 nameless-feather-6339
Connecting to fdaa:0:3b99:a7b:7e:3155:9844:2... complete
Filesystem     1K-blocks   Used Available Use% Mounted on
devtmpfs          103068      0    103068   0% /dev
/dev/vda         8191416 172748   7582856   3% /
shm               113224      0    113224   0% /dev/shm
tmpfs             113224      0    113224   0% /sys/fs/cgroup
/dev/vdb         1011672   2564    940500   1% /data

Voilà: our 1GB volume is mounted at /data. You can access and write to a volume on a Machine just like a regular directory.

Clone the First Machine to Scale Out to Two VMs

fly machine clone 5683606c41098e

Check what that did:

fly volumes list

ID                      STATE   NAME    SIZE    REGION  ZONE    ENCRYPTED       ATTACHED VM     CREATED AT     
vol_ez1nvxkwl3jrmxl7    created data    1GB     lhr     4de2    true            91851edb6ee983  39 seconds ago
vol_zmjnv8m81p5rywgx    created data    1GB     lhr     b6a7    true            5683606c41098e  7 minutes ago

fly machine clone creates an additional Machine with the same configuration as the source, and creates and mounts a volume for the new Machine, if the source Machine has one. fly machine clone doesn't write data into the new volume.

At this point there are two identically configured Machines on the app, each with a volume of the same size. You take it from here, setting up whatever data replication you need.

On a V1 (Nomad) App

Create an additional volume in the desired region, then use fly scale count to add a VM to use it.

Add Volumes to an Existing App

You can add a volume, or volumes, to an existing app by adding [mounts] to the app's fly.toml file and creating the volumes using flyctl.

Configure the App to Mount the New Volume

Add a [mounts] section in the app's fly.toml. The following example configures the app to expose data from a volume named myapp_data under the /data directory of the application.

[mounts]
source="myapp_data"
destination="/data"

Create the Volume

Create the volume (or volumes) in the same region as your app.

fly volumes create myapp_data -r lhr

If you configure [mounts] in fly.toml and then run fly deploy without first creating enough volumes, then you'll get an error similar to this one:

Error: Process group 'app' needs volumes with name 'myapp_data' to fullfill mounts defined in fly.toml; 
Run fly volume create myapp_data -r REGION for the following regions and counts: lhr=1.`

The error indicates that the app needs one more volume in the lhr region.

Deploy the App

fly deploy 

Run fly volumes list to verify that your volumes now have ATTACHED VM populated.

fly volumes list

ID                      STATE   NAME    SIZE    REGION  ZONE    ENCRYPTED       ATTACHED VM     CREATED AT    
vol_zmjnv8m81p5rywgx    created data    1GB     lhr     b6a7    true            5683606c41098e  3 minutes ago

Add a Volume to a Machine Clone

If you're not using the fly.toml with fly deploy workflow for some Machines, you can still add a volume by cloning a Machine with no volume and attaching a new volume to it. You can also clone a Machine with a volume already attached, which gives you a new Machine with a new, empty, volume attached.

Create the Volume

Create the volume in the same region as your app.

fly volumes create myapp_data -r lhr

Clone the Machine

Clone one of your app's Machines (with no volume) and attach the volume you just created:

fly machine clone <machine-id> --attach-volume <vol-id>:<destination-mount-path>

destination-mount-path is the directory where the volume should be mounted on the file system.

For example:

fly machine clone 148eddeef09789 --attach-volume vol_8l524yj0ko347zmp:/data

Destroy the Machine Used to Create the Clone

fly machine destroy 148eddeef09789

Run fly volumes list to verify that the volume you just created has the newly-cloned Machine's ID listed under ATTACHED VM.

fly volumes list

ID                      STATE   NAME    SIZE    REGION  ZONE    ENCRYPTED       ATTACHED VM     CREATED AT    
vol_8l524yj0ko347zmp    created data    1GB     lhr     b6a7    true            5683606c41098e  3 minutes ago

Repeat the process for each machine that you want to add a volume to.

Remove a Volume From an App

You can't currently unmount a volume from an existing Machine. If you need to remove volumes from your app, you can scale it down to zero Machines, destroy all the volumes, remove the mounts section from fly.toml, and redeploy. Only destroy a volume whose data you no longer need access to.

On a V1 (Nomad) App

Remove the mounts section from fly.toml and redeploy. This will create a new Machine without a volume. Destroy any unused volumes, only if you no longer need their data.